Off-manifold robustness in synthesizer inversion with joint distribution flow matching

Ben Hayes
Sony Computer Science Laboratories Paris
Paper Supplementary

Abstract

Recent work on synthesizer inversion shows that generative models outperform deterministic approaches by explicitly modeling the ambiguity in mapping audio to parameters. Training such models, however, requires audio-parameter pairs, which are typically obtained by rendering sampled or preset parameters through the synthesizer itself. This creates a train-test mismatch that can degrade performance on off-manifold real-world recordings, for which ground-truth parameter annotations do not exist. To circumvent this obstacle, we propose to model the joint distribution of audio and parameters with a multi-modal continuous normalizing flow using independent noise schedules for each modality. This formulation allows us to train joint and conditional densities with paired synthesizer data, while unpaired real recordings train the audio marginal alone, exposing the model to off-manifold signals without requiring parameter labels. Further, because the model learns to map from audio to parameters at all noise levels, we find that partially noising the audio reference at inference improves real-audio reconstruction, consistent with reducing sensitivity to distribution-specific detail while preserving coarse structure. Evaluating on Surge XT and Dexed, we find our method substantially improves inversion of real-world audio while remaining competitive in-domain.

On-Manifold Audio Examples

In these examples, the target audio was rendered by the synthesizer itself.
We invert reconstructions for two synthesizers: Surge XT, a full-featured open-source subtractive synth used in real-world production, and Dexed, an open-source emulation of the Yamaha DX7 FM synthesizer. Targets are rendered from parameter settings sampled randomly from each synthesizer's parameter space, so a setting that exactly reproduces the target always exists; the task reduces to recovering it.

Each cell shows a mel spectrogram and waveform; press play to listen. Press “Fetch more samples” to load a different batch.

Off-Manifold Audio Examples

In these examples, the target audio did not come from the synthesizer.
The model must produce parameter estimates for audio that no setting of the synthesizer can exactly reproduce, and ground-truth annotations do not exist.

BibTeX

@inproceedings{hayes_offmanifold_2026,
  author    = {Hayes, Ben},
  title     = {Off-manifold robustness in synthesizer inversion with joint distribution flow matching},
  booktitle = {Proceedings of the 27th International Society for Music Information Retrieval Conference},
  address   = {Abu Dhabi, UAE},
  year      = {2026},
}