Although not entirely off the beaten path since it is on the rue St-Antoine which continues from the rue de Rivoli, this church is certainly not anything I've ever read about in guidebooks.
In fact, I'd never even noticed it before this trip!

This old Renaissance edifice (which lies in the midst of the Marais at the corner of rue St-Antoine & rue Castex)was the old Marais parish church known as the Ste-Marie temple.
It is well-noted for being built by Mansart between 1632-34 under the initiative of Francois de Sales and Jeanne de Chantal.
After the Revolution, it was thankfully spared and reformed into a Protestant church on December 3, 1802 and has been a place of worship since May 1, 1803.
It is classified as an historic monument with tours occurring one Sunday out of the month.

Worship services every Sunday at 10:30am.

http://eglisereformee.dumarais.fr/