Morph 2 FFT

[eternith15]

-

[martinvicanek]

Hey eternith15,

welcome to the forum! You could start here to get some inspiration:
viewtopic.php?f=2&t=3972&p=22927

[eternith15]

-

[Spogg]

Hi eternith15

This idea fascinates me.

The subject of morphing has come up a few times on the forum. One difficulty is that there seems to be varying definitions or understanding of the term. Some are content with simple cross-fading and I guess this actually is a kind of morphing.

Then there's spectral morphing whereby each partial's frequency, phase and amplitude are effectively "individually" cross-faded to give a smooth transition range between what could be very different signals.

What I've never yet come across is what you describe, whereby you have a change of each partial's frequency between the 2 signals. If you have for example one signal described instantaneously by 20 partials and the other signal by 25 partials how would the system determine which intermediate frequencies to create?
Using the suggested scheme, if you morphed between a deep male vocal and a high female vocal what would the 50/50 morph sound like and would it be useful, because the fundamental pitch would be shifted?

If you and/or Martin could achieve what you suggest I'd love to hear the results!

Cheers

Spogg

[eternith15]

Ok, so I spent the day implementing this, I looked over at FS first, but coming from a traditional coder point of view it looks quite tedious to implement there. I also looked at Protoplugin but couldn't figure out how to sidechain it. So I ended up using Harmor to export the FFT in high-precision mode, and created the algorithm for the pixels per each column to morph between. It worked as I expected, but I need to turn up the particles further in number for an even cleaner transition (I used 40,000 particles). I wrote it in C# and just let it process.

Here are the sources:

SourceA

SourceA 1 Partial

sourceA.png (1.55 KiB) Viewed 32074 times

SourceB

SourceB 2 Partials

sourceB.png (1.73 KiB) Viewed 32074 times

SourceC

SourceC Noise

sourceC.png (230.91 KiB) Viewed 32074 times

[eternith15]

And here are the morphed results:

SourceA 1 Partial to SourceB 2 Partials

SourceA to SourceB

result1.png (10.23 KiB) Viewed 32073 times

SourceA 1 Partial to SourceC Noise

SourceA to SourceC

result2.png (156.58 KiB) Viewed 32073 times

It seems to morph anything pretty well so far, these are just the basics. I'll put up sounds samples likely tomorrow.

[tulamide]

Coming from game development, I sure have not very much to say here. But, unless you or martin find some very tricky shortcuts, I'm afraid we would have to wait for more powerful CPUs.
In gaming, where I can outsource everything related to drawing to the graphics card, and can make use of parallel processing (cpu calculates data for next frame, while gpu draws current), even with 100% processor usage for particles I never could process more than approx. 10,000 per frame (running at 60 fps). That's 60,000 per second.

If I understand your idea a bit, you would need to fft every sample (44,100 per second) and then morph some 40k particles? That's too much for any current CPU (again, unless you find tricky shortcuts).

Btw, I also have a dream. A real vector oscillator! I have no clue how this could be done, but the idea is a realtime oscillator (calculating at runtime), that uses vector information to build up the waveform. With the vectors always starting where the last stopped, there's always a smooth transition in the interpolation, but with modulating the end points one could create endless waveforms without the need for wavetables, morphing techs, etc. (I'm almost sure, nobody than me understands what I'm talking about, so it will stay a dream, but it fascinates me just thinking about it)

[Spogg]

Jeez I can't wait to actually hear the samples! Can you morph a male vocal into female with this?

@tulamide: Your dream sounds achievable to me. I imagine Martin could do this in DSP then optimise it. I guess you'd need to decide the number of nodes in advance and make a clever correction for overall period. Fascinating idea!

Cheers

Spogg

[BobF]

Hello eternith15,

This sounds very interesting and intriguing,

I do not know if this will help but it has to do with partials. I posted it before , but you may not have seen it. Here you go,

http://www.klingbeil.com/spear/

I asked Martin about this once and he said it would take a collaboration. Since I am not proficient in coding maybe you would be interested in it.

Can not wait to see more!

Later then, BobF.....

[tulamide]

@tulamide: Your dream sounds achievable to me. I imagine Martin could do this in DSP then optimise it. I guess you'd need to decide the number of nodes in advance and make a clever correction for overall period. Fascinating idea!

I'm shocked. You seem to really understand my thoughts, because indeed, a fixed number of nodes is all you need to predefine! Everything else, their position in time and amplitude is totally free and depends solely on the modulated values (length, angle) of the previous vector, the modulation of the current one, and the applied interpolation math.

@eternith15, I'm sorry for having interrupt the thread. I will now keep my mouth shut about this idea!