Support

If you have a problem or need to report a bug please email : support@dsprobotics.com

There are 3 sections to this support area:

DOWNLOADS: access to product manuals, support files and drivers

HELP & INFORMATION: tutorials and example files for learning or finding pre-made modules for your projects

USER FORUMS: meet with other users and exchange ideas, you can also get help and assistance here

Morph 2 FFT

DSP related issues, mathematics, processing and techniques

Morph 2 FFT

Postby eternith15 » Mon Aug 08, 2016 5:45 am

I'm looking to morph two signals, that is interpolate both frequency and amplitude. I'm looking for some help in generally developing this idea. I own FS. I'm aware of MMorph and Zynaptiq Morph 2 and played with the demos and all it's algorithms, none achieve what I am looking for.

The idea I have is to FFT each of the incoming two signals (assuming mono).

For each FFT output:
1. accumulate the amplitudes of all bins
2. create a large set of 'interpolation particles' (assume say 100,000 particles per fft)
- per fft, the particles all share one amplitude dividing the accumulated amplitude evenly
- the particles are distributed across the frequency bins recreating the spectral shape exactly
3. pair each particle from one fft with the other (at random)
4. interpolate particle amplitude and frequency given an input between 0 to 1
5. ifft the result to output
- the one exception is a signal with zero amplitude, just fade out

If there are enough particles dividing up the spectrum, the energy should simply move around the spectrum fairly fluidly. Phase may still need addressing.

I got the idea after watching this demo and visualizing the particles moving around by interpolation between 2 points.
https://www.youtube.com/watch?v=qDoV6NVbNbY

If you have some ideas or examples, please discuss.
eternith15
 
Posts: 7
Joined: Sun Mar 22, 2015 1:55 am

Re: Morph 2 FFT

Postby martinvicanek » Mon Aug 08, 2016 6:53 pm

Hey eternith15,

welcome to the forum! You could start here to get some inspiration:
viewtopic.php?f=2&t=3972&p=22927
User avatar
martinvicanek
 
Posts: 1122
Joined: Sat Jun 22, 2013 8:28 pm

Re: Morph 2 FFT

Postby eternith15 » Wed Aug 10, 2016 5:33 am

martinvicanek,
Very very impressive. If that were on live material it would have many applications. Like a new kind of vocoder or trackspacer, just to name a couple ideas of the top of my head. So many applications for that. Thank you for the idea and the inspiration.

The morph idea I have, the way I see it is like this, if you have channel A with a single partial in it, and channel B (to morph to) with a single partial in it as well (at different frequencies).. the resulting morph would be a simple pitch shift between the 2 as the morph occurs. Another example, if channel A has a single partial, and channel B has two partials in it, A will morph to B by splitting in the transition and vice versa. This can be done with the particle concept I have, of course even with the 2 partials in the first example, there would still needs to be many particles composing each partial This idea however would always do a clean transition of both amplitude and freq no matter what the spectral shapes are in A and B.

Back to your idea though, thank you for the inspiration, a live version of your idea would be a gem in every audio/music toolkit. I'd imagine it being like the invention of the microwave, since it's its inception, we can't imagine life without it.
eternith15
 
Posts: 7
Joined: Sun Mar 22, 2015 1:55 am

Re: Morph 2 FFT

Postby Spogg » Wed Aug 10, 2016 10:24 am

Hi eternith15

This idea fascinates me.

The subject of morphing has come up a few times on the forum. One difficulty is that there seems to be varying definitions or understanding of the term. Some are content with simple cross-fading and I guess this actually is a kind of morphing.

Then there's spectral morphing whereby each partial's frequency, phase and amplitude are effectively "individually" cross-faded to give a smooth transition range between what could be very different signals.

What I've never yet come across is what you describe, whereby you have a change of each partial's frequency between the 2 signals. If you have for example one signal described instantaneously by 20 partials and the other signal by 25 partials how would the system determine which intermediate frequencies to create?
Using the suggested scheme, if you morphed between a deep male vocal and a high female vocal what would the 50/50 morph sound like and would it be useful, because the fundamental pitch would be shifted?

If you and/or Martin could achieve what you suggest I'd love to hear the results!

Cheers

Spogg
User avatar
Spogg
 
Posts: 2687
Joined: Thu Nov 20, 2014 4:24 pm
Location: Birmingham, England

Re: Morph 2 FFT

Postby eternith15 » Thu Aug 11, 2016 9:41 am

Ok, so I spent the day implementing this, I looked over at FS first, but coming from a traditional coder point of view it looks quite tedious to implement there. I also looked at Protoplugin but couldn't figure out how to sidechain it. So I ended up using Harmor to export the FFT in high-precision mode, and created the algorithm for the pixels per each column to morph between. It worked as I expected, but I need to turn up the particles further in number for an even cleaner transition (I used 40,000 particles). I wrote it in C# and just let it process.

Here are the sources:

SourceA
sourceA.png
SourceA 1 Partial
sourceA.png (1.55 KiB) Viewed 18797 times


SourceB
sourceB.png
SourceB 2 Partials
sourceB.png (1.73 KiB) Viewed 18797 times


SourceC
sourceC.png
SourceC Noise
sourceC.png (230.91 KiB) Viewed 18797 times
eternith15
 
Posts: 7
Joined: Sun Mar 22, 2015 1:55 am

Re: Morph 2 FFT

Postby eternith15 » Thu Aug 11, 2016 9:44 am

And here are the morphed results:

SourceA 1 Partial to SourceB 2 Partials
result1.png
SourceA to SourceB
result1.png (10.23 KiB) Viewed 18796 times


SourceA 1 Partial to SourceC Noise
result2.png
SourceA to SourceC
result2.png (156.58 KiB) Viewed 18796 times


It seems to morph anything pretty well so far, these are just the basics. I'll put up sounds samples likely tomorrow.
eternith15
 
Posts: 7
Joined: Sun Mar 22, 2015 1:55 am

Re: Morph 2 FFT

Postby tulamide » Thu Aug 11, 2016 10:02 am

Coming from game development, I sure have not very much to say here. But, unless you or martin find some very tricky shortcuts, I'm afraid we would have to wait for more powerful CPUs.
In gaming, where I can outsource everything related to drawing to the graphics card, and can make use of parallel processing (cpu calculates data for next frame, while gpu draws current), even with 100% processor usage for particles I never could process more than approx. 10,000 per frame (running at 60 fps). That's 60,000 per second.

If I understand your idea a bit, you would need to fft every sample (44,100 per second) and then morph some 40k particles? That's too much for any current CPU (again, unless you find tricky shortcuts).

Btw, I also have a dream. A real vector oscillator! I have no clue how this could be done, but the idea is a realtime oscillator (calculating at runtime), that uses vector information to build up the waveform. With the vectors always starting where the last stopped, there's always a smooth transition in the interpolation, but with modulating the end points one could create endless waveforms without the need for wavetables, morphing techs, etc. (I'm almost sure, nobody than me understands what I'm talking about, so it will stay a dream, but it fascinates me just thinking about it)
If you feel like gifting: https://paypal.me/tulamide
tulamide
 
Posts: 2379
Joined: Sat Jun 21, 2014 2:48 pm
Location: Germany

Re: Morph 2 FFT

Postby Spogg » Thu Aug 11, 2016 12:38 pm

Jeez I can't wait to actually hear the samples! Can you morph a male vocal into female with this?

@tulamide: Your dream sounds achievable to me. I imagine Martin could do this in DSP then optimise it. I guess you'd need to decide the number of nodes in advance and make a clever correction for overall period. Fascinating idea!

Cheers

Spogg
User avatar
Spogg
 
Posts: 2687
Joined: Thu Nov 20, 2014 4:24 pm
Location: Birmingham, England

Re: Morph 2 FFT

Postby BobF » Thu Aug 11, 2016 3:34 pm

Hello eternith15,

This sounds very interesting and intriguing,

I do not know if this will help but it has to do with partials. I posted it before , but you may not have seen it. Here you go,

http://www.klingbeil.com/spear/

I asked Martin about this once and he said it would take a collaboration. Since I am not proficient in coding maybe you would be interested in it.

Can not wait to see more!

Later then, BobF.....
BobF
 
Posts: 592
Joined: Mon Apr 20, 2015 9:54 pm

Re: Morph 2 FFT

Postby tulamide » Thu Aug 11, 2016 10:26 pm

Spogg wrote:@tulamide: Your dream sounds achievable to me. I imagine Martin could do this in DSP then optimise it. I guess you'd need to decide the number of nodes in advance and make a clever correction for overall period. Fascinating idea!

I'm shocked. You seem to really understand my thoughts, because indeed, a fixed number of nodes is all you need to predefine! Everything else, their position in time and amplitude is totally free and depends solely on the modulated values (length, angle) of the previous vector, the modulation of the current one, and the applied interpolation math.

@eternith15, I'm sorry for having interrupt the thread. I will now keep my mouth shut about this idea!
If you feel like gifting: https://paypal.me/tulamide
tulamide
 
Posts: 2379
Joined: Sat Jun 21, 2014 2:48 pm
Location: Germany

Next

Return to DSP

Who is online

Users browsing this forum: Majestic-12 [Bot] and 6 guests

cron