Please use the youtube doubler to view this, it’ll make more sense:
The only sounds in this video are from this sound file: http://softwareprocess.es/b/fmout.ogg
The original video is here: http://www.youtube.com/watch?v=4fndeDfaWCg
Mostitch tries to interpret the audio by running metrics on the input audio (backstreet boys) and then matches it to the closest audio from the fm synthesizer. Then it reports the index of the best match in the synthesizer. This best match aligns with a set of parameters that we feed back into the original synthesizer.
To give you some idea of the roundabout path we take here’s an example of how everything starts:
python print_stitch.py fm/fmout.wav | perl fm/fmmapper.pl fm/map.dump | csound -L stdin -o devaudio -+rtaudio=jack -dm6 ./fm/fm.orc ./fm/fmlive.sco
Source code here: https://github.com/abramhindle/mostitch/tree/fm
This version is 3 bands of FM synthesis stitched together with some hard stereo panning.