The usual models for dissipative environments involve a bath of harmonic oscillators, producing Gaussian fluctuations. However, modern experiments on dephasing in qubits and electronic interferometers indicate strong coupling to non-Gaussian quantum noise. Most strikingly, the coherence (interference contrast) may oscillate as a function of time, voltage, and other control parameters. We present the theory behind a recent "controlled dephasing" experiment involving an electronic Mach-Zehnder interferometer strongly coupled to the non-Gaussian shot noise of a detector edge channel. Visibility oscillations are predicted (and experimentally found) for the coupling strength between system and bath exceeding a certain threshold. We argue that this is a genuine and generic feature of non-Gaussian noise.