Impulse Response

[martinvicanek]

Here is a little convolution reverb demo. It is derived from the stream FFT project here:
viewtopic.php?f=4&t=1510&start=60#p6660
The reverb duration is limited by the FFT size (32k). Have fun.

[tulamide]

THIS.IS.THE.BOMB.

Simple as that. That's just about the limit. Awesome. Awesome. Crazy. Genius. Awesome.

I don't think people realize what this module, you've done here, really means. I wonder where you got the floats from that represent the IR (inside module "FFT'd IR")?
Wouldn't it be possible to load longer IRs, say with a tail of a few seconds?

Because, if that last one could be realized then the following should be done:
We could bring a Impulse Response Processor on the market. I would call it IRP because with IRs you can not only represent reverb. There's guitar amps, for example. I even have IRs from the inside of a violin, giving violin sounds that extra realism. It works fantastic as an effect engone as well.
If we all would bring our powers together, for example Martin and Myco for the ASM/DSP, people like me for Ruby and GUI design, people with microphones (Spogg?) for recording Impulse Responses, we could publish a professional standard convolution engine and sell it.
Not for our direct profit, but to give the income DSPRobotics, exclusively for the developing of Flowstone's audio part, for example paying MyCo, or buying material needed for upgrading the audio engine.

You might think I'm totally crazy now, but I am not. A convolution engine, an impulse response processor, is state-of-the-art. There aren't many out there. For a good reason. It is a cpu-heavy task. So only highly optimized code and the quickest of audio engines are capable of it.

In audio forums, I often read something along the lines of "get away with those slow, cpu-heavy synthmaker modules". A professional convolution engine, both in DSP- and GUI-Design, would increase Flowstone's reputation a lot!

And we are close, very close! The demo already runs flawlessly. We just need high quality IRs and a way to allow for any size IRs.

I hope all of you read this. Think about it. Think about it a second time. And then let's do it!

[martinvicanek]

Tulamide,
I got the IR by sampling the Freeverb.

From my point of view, before we can consider this more than a demo we have to:
1. replace the serial inputs/outputs by shared mems to avoid extra delay. My experience with memin has been rather frustrating so far.
2. extend the FFT size to at least 256k. Notice that FFT implements circular convolution in the first place. For linear convolution (the one that we need for reverbs) only half of the data is used (the other half is simply thrown away). That makes it 128k equivalent to 3 seconds reverb duration. The demo has 0.5*32k, i.e. 0.35 seconds.
3. implement a partitioned convolution as pioneered by Gardner http://www.cs.ust.hk/mjg_lib/bibs/DPSu/ ... s/Ga95.PDF. This means increased complexity and CPU load.

I am not saying it is impossible - I don't know - but those three steps are not easily solved.

[KG_is_back]

Finally I've found it...

Some time ago, I've made a convolution reverb (it works but crashes often). The schematic is very hard to read...
It works kind of like this:
Assuming window-size is 64 and FFT size is twice (to leave space for tails - explained forther). I will include C-ish code example for readability.

stage 1 - impulse preprocessing:
1.separate first 64samples. They will be convolved in time-domain. Let's call that a nose.
2. FFT each 64sample segment (with added 64samples of silence) with 128FFT and store them (they will take 4x as much space as original impulse). Let's call that a IMPULSE_STACK
stage 2 - convolution.
To process the convolution in frequency domain, you need to load 64sample chunk of input, so the convolution part is actually 64samples delayed. We've already taken care of that - the frequency domain convolution will use impulse trimmed from first 64samples (which will be convolved via time domain convolver).

IN_ARRAY[256] = input buffer
IN_FFTED = stack of FFTed input buffer. 4x the original impulse length. It will be circular buffer with offset N which will be incremented in 128 steps.
ACCUM[512]= accumulator buffer. This one will accumulate the multiplied spectra. it will then be iFFTed and also read from. Because after iFFT the accumulator will also contain additional 64sample reverb tail, you need two. They will be used alternatively. ALT will determine, which accumulator to use - it will flip between 0 and 256 to switch them after each window.

1. buffer 64samples of input into IN_ARRAY and add 64samples of silence. Don't forget imaginary parts.

Code: Select all

streamin in;
int i=0;
IN_ARRAY[i]=in; //even are REAL
in_ARRAY[i+1]=in; //odd are IMAG
//set the second half of buffer to zero
IN_ARRAY[i+128]=0;
IN_ARRAY[i+129]=0;

//this should be at the very end of code, but I include it here for clarity
i=(i+2) % 256; //circulate i.



2. FFT the IN_ARRAY with 128FFT and push it into the IN FFTED starting at position N.

Code: Select all

// fft(dest,source); -both destination and source are arrays - even elements are REALs odd elements are IMAGs
// hope you know how pointers work in C.

N= (N+256) % impulseLength; //N is circulated around impulse length
dest=IN_FFTED + N; //dest is IN_FFTED offset by N 
fft(dest,IN_ARRAY);


3. iterate trough IN_FFTED and multiply it with IMPULSE_STACK. Remember IN_FFTED is ofset by N.

Code: Select all

acc=ACCUM+ALT; //this will switch between accumulator parts for odd and even windows.
ALT=(ALT+256) % 512;
//first empty the accumulator array
for(k=0; k<256; k++)
{
acc[k]=0;
}
for(k=impulseLength-2; k<=0 ; k-=2) //iterate over complex frequency bins
//Note the iteration happens in reverse, to minimize rounding errors (end of impulse will most likely contain smaller values)
{
//load real and imaginary parts from IMPULSE_STACK and IN_FFTED
imp_R=IMPULSE_STACK[k];
imp_I=IMPULSE_STACK[k+1];
in_R=IN_FFTED[ (N+k) % impulseLength ]; // the "%impuseLength" is there to wrap the reading
in_I=IN_FFTED[ (N+k+1) % impulseLength ];

//Do complex multiplication and add the results to the acumulator
acc[k %256]= imp_R*in_R - imp_I * in_I; //real part
acc[k %256]= imp_R*in_I + imp_I * in_R; //imaginary part
}


4. iFFT the accumulator (the part that was collected).

Code: Select all

fft(acc,acc);


5.read the values from the accumulator at samplerate. Note that since accumulator has two parts (reverb tail from one window should overlap with the start of next window), you need to read from two halves:

Code: Select all

streamout out;
out = ACCUM[i] + ACCUM[i+256]; //only real parts are relevant - imaginary parts are zero anyway.


The attacked schematic implements that, but the processing part is completely written in assembler and hard to read (also names of variables, inputs and outputs are misleading).

[martinvicanek]

Some time ago, I've made a convolution reverb

Convolution, yes, but where is the reverb? I can see a lowpass filter...

(it works but crashes often).

Confirmed.

The schematic is very hard to read...

Confimed!
Thanks for sharing, KG! According to your description you have implemented a straight convlolver and the first overlap-add-stage of the partitioned convolver described in the Gardner paper (and later work building on that concept). Cool!
So in essence you have a zero latency 128-point convolver. To proceed from there, you can implement the next 128 samples FFT convolution via FFT and add it to the current result. It will fit seamlessly to your existing 128 convolved signal in the same way that your first 64 FFT'ed samples fit to your 64 direct convolution. Then the next 256 saples, and so on, until either you achieve a decent reverb szize or your CPU evaporates. I have yet to find out which happens first.

[KG_is_back]

To proceed from there, you can implement the next 128 samples FFT convolution via FFT and add it to the current result.

Unnecessary... Remember, 64sample segments (convelved via 128-FFT - so technically they are 256floats) are stored in IN_FFTED, which is circular buffer. To clarify IN_FFTED is a circular buffer of FFT windows. Previous window is multiplied by next segment of impulse (which is preprocessed (FFTed) in the same way). The accumulator I've mentioned sums all previous windows (which are multiplied by appropriate segments of impulse) and then iFFT the sum.
All in all, the convolver should work with any impulse of any length (the length is extended to be multiple of 64).

Convolution, yes, but where is the reverb? I can see a lowpass filter...

Yeah, the impulse loaded into the convolver in my example is 1000-sample long IR of lowpass filter. It should be able to load any impulse of any length (as long as it fits into the buffers ~ 2^17 samples long).

If you think about it, time domain-convolver is just a special case of algorithm I've described above, where window is of length 1. Since FFT and iFFT of one sample is itself. Time-domain convolver stores the input (which gets FFTed at this point) into circular buffer (which contains FFTed windows of input - but each window is just 1 sample long), then multiply each sample in the buffer by corresponding sample of the impulse (which technically also is a sequence of 1sample-FFTs), accumulating the sum. The accumulated result (which now gets iFFTed, so it stays the same, because it's 1sample long) is passed to output.

[tulamide]

Guys, this is what I'm talking about! Combining our powers. It was a long way from trog, exo, MyCo, Martin and KG. I doubt that people would have thought a stream fft would be possible 4 years ago. And now we even have the beginnings of a convolution engine!

KG, from what I read you pretty much made what WebAudio (part of html5) does as well, and which seems to be going back to Gardner. I linked to their explanations in a post above. Here it is again: https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/convolution.html,

Could you please have a look at it and confirm the similarity or disconfirm?

It crahes a lot and you used machine code. It seems obvious that the code violates some rule. What about the ringbuffer? Crashes could occur if read and write operations occur at the same time. Also, for such a buffer the memory needs to be contiguous and should be locked, to prevent fragmentation and re-addressing. Is this the case for memory that is requested through Flowstone's prims/code?

I know that isn't much, but I can't help more.

[KG_is_back]

It is very similar, with one marginal difference. The FFT of the input is only performed once and it's stored in a buffer. Also the summing of resulting frames is done in frequency domain and iFFT is only performed once. So in the end, I'd say my approach is actually more efficient.

Gardners convolver seems to be a parallel of FFTconvolvers described on the page Tulamide provided. First convolver buffers N samples and performs 2N convolution. Second buffers 2Nsamples and performs 4Nconvolution etc.
Mine buffers N samples, performs 2N FFT and then does series of 2Nconvolutions (which are now only multiplications of the pectra) and sums their results into one 2N buffer. The buffer is then 2N iFFTed and serialized to output.


I should upgrade to 3.0.9 beta. The new stuff that was added would make the implementation much easier - especially the FFT.

[tulamide]

I hope you guys are still crashing your heads to get this going? I found only 1 free convolution reverb and even that one is using exclusive IRs:
http://www.siraudiotools.com/sir2.php

I think it would be worth to prove FS is still competing with the standards!

[Spogg]

I wish I was able to help with this. It sounds fascinating!

Just a note to say there are loads of free IR waves out there. For example try

https://www.propellerheads.se/blog/free ... -responses

I did find a convolution reverb buried in the old Synthmaker forum remnants. It would only take IR wavs up to 8192 samples long. I played around with it but the results (on voice) were unusable. Maybe this would be enough samples for a guitar amp cabinet?

If there's any interest I could find it again and post it here...

Good luck to all of you who are working towards something rather special.

Cheers

Spogg