ESQ1 Project

[tulamide]

In the hope that more people are interested to add to the project, I create this thread. We should focus on the creation of an ESQ1 clone here, and create other threads for things that are only loosely connected to it.

My first contribution is a collection of the extracted 82 waveforms from the original ROM. Those are in their original format (I added a text file describing it shortly). They are binary files that can be opened in an audio editor that accepts RAW audio.

I also added the .fsm of a rough version of the extractor I programmed for this. The extractor can also be used to get all waveforms from the 4 SQ80 ROM files, that you can download from http://www.buchty.net/ensoniq/.

You just need to select the section of the ROM you're interested in from the Excel table, Spogg provided (http://www.dsprobotics.com/support/viewtopic.php?f=2&t=12389#p42868 SECOND TAB!), and export it as a csv file (Comma Seperated Values). I've used LibreOffice Portable for that, because I don't own Excel and the online version doesn't export .csv. Load that file and the rom file, then click on "split rom". The original stays untouched, a new folder named "waveforms" is created wherever the .fsm is saved to, and filled with the waveforms. If such a folder already exists, it gives out a warning and does no extracting.

[Spogg]

Absolutely totally effing brilliant tulamide!
That Ruby code is a work of art in itself, and I can now appreciate it more than ever.

To make life easier for others who, like me, want the SQ80 sounds as well, I've extracted the other ROMs and attached them, including your extraction of ROM 1, so they are all in one bundle.

I haven't tested them yet, or done anything with any of them at this point. That's next... unless you already have them all as Float arrays or WAVs.

Cheers

Spogg

[KG_is_back]

I think it might be possible to convert the raw files into wavefiles simply by adding wave file header.

Code: Select all

data=@raw[start,length] #data should be the string containing the raw data
sampleRate=44100 #set appropriate value
bitdepth=8
byteRate=sampleRate*bitdepth #note mono is assumed

filedata=["RIFF",36+data.length,"WAVEfmt ", 16, #RIFFchunk and formatchunk header
1,1, sampleRate,byteRate,1,8, #format data
"data",data.length, #datachunk header
data].pack("a4La8LSSLLSSa4La*")

#modify this line appropriately
File.open(path,"wb"){|file| file.write(filedata)}



The code assumes the audio is mono 8bit linear PCM.

[tulamide]

Sorry, I was destracted by the world cup. But now that I (unfortunately) have lots of time, I will continue.

Spogg, float arrays are almost ready.

KG, thanks for the code. Very clever! I have no doubt it works (if 8 bit in the chunk refers to unsigned integer byte), but I'm afraid Flowstone won't read in such a format. I once tried 32-bit float .wav and FS refused to load them (without any message of course, so that we have to guess what was going wrong). The same file as 16-bit word was loaded without issues. Also, the sample rate of these waveforms is a very odd one [ (1 MHz / 26) = lots of decimal digits ], FS will refuse to load it as well, I'm afraid. But I'll test it.

[KG_is_back]

Sorry, I was destracted by the world cup. But now that I (unfortunately) have lots of time, I will continue.

Spogg, float arrays are almost ready.

KG, thanks for the code. Very clever! I have no doubt it works (if 8 bit in the chunk refers to unsigned integer byte), but I'm afraid Flowstone won't read in such a format. I once tried 32-bit float .wav and FS refused to load them (without any message of course, so that we have to guess what was going wrong). The same file as 16-bit word was loaded without issues. Also, the sample rate of these waveforms is a very odd one [ (1 MHz / 26) = lots of decimal digits ], FS will refuse to load it as well, I'm afraid. But I'll test it.

Strange... I never had problems loading wavefiles of any format. Technically, the samplerate is non-issue. If you know what sample-rate the wave should be, you can save it with arbitrary samplerate and then simply ignore it in the flowstone schematic.
It is also not that hard to convert formats in ruby:

Code: Select all

data = #raw string of 8bit unsigned values
newData16bit= data.unpack("C*").map!{|v| (v-128)*256}.pack("s*") #converts to 16bit signed raw string
newData32float= data.unpack("C*").map!{|v| (v-128)/128.0}.pack("f*") #converts to 32bit float raw string
sampleArray=data.unpack("C*").map!{|v| (v-128)/128.0} #converts to ruby array of floats, that you can output
#really, the only tricky one is 24bit


Off course, you could potentially skip the raw file saving if the intent is to get the values into flowstone as arrays of floats.

[tulamide]

Strange... I never had problems loading wavefiles of any format.

Interesting. Did you ever encounter problems loading mp3s with the streaming prim? Because I had. Only half of my collection actually loaded, and it seems that vbr is one issue.

It is also not that hard to convert formats in ruby:

I never said it is. What takes time is to prepare a schematic in such a way that people without Ruby knowledge can make use of them (after all we're talking about 82 waveforms, and I'm not willing to create 82 float array prims or text prims). However, 'it takes time' doesn't mean 'it's hard'.

Code: Select all

sampleArray=data.unpack("C*").map!{|v| (v-128)/128.0} #converts to ruby array of floats, that you can output

Yep, that's pretty much what I use as well. Only that it's simpler. No need for unpack, it's a binary string and strings already have the method ::each_byte
Also the math actually is '(v - 127) / 128.0', because Ensoniq used only 255 values. Zero was used to stop/reset the loop, not as data, and 0x80 was the center.

Off course, you could potentially skip the raw file saving if the intent is to get the values into flowstone as arrays of floats.

Well, that's up to the guys who will do the real stuff (like building the oscillators). Ensoniq used multi-sampling, so for me it would be natural to work with .wav files and the wave array prim, but there are also other ideas. So why not offering as many formats as needed, if possible? Also, a lot of help came from Rainer Buchty's Ensoniq webpage. His help was crucial, so I thought to send him the .wav files to offer them on his page alongside the ROMs.

[KG_is_back]

Interesting. Did you ever encounter problems loading mp3s with the streaming prim? Because I had. Only half of my collection actually loaded, and it seems that vbr is one issue.

I never really played with mp3s in flowstone... The capabilities of the prim that processes them is very limited.

I never said it is. What takes time is to prepare a schematic in such a way that people without Ruby knowledge can make use of them (after all we're talking about 82 waveforms, and I'm not willing to create 82 float array prims or text prims). However, 'it takes time' doesn't mean 'it's hard'.

I was thinking more along the ways of Wave Array set prim...

[Spogg]

Here are the ROM 1 waves for the ESQ-1 in raw and wav format.
I've only listened to a few but they sound promising.

The ones I tried load into a stock wave player correctly. I tried 8 bit WAVs and they don’t load correctly. These are standard 16 bit 44100 and the number of samples is the same as the original.

I’ve done ROM 2 and 3 but I'm a bit puzzled why some seem to contain 2 sounds in one WAV so I’ll be looking into that today and converting ROM 4.

For others watching, ROM 1 is the only one used by the ESQ-1. The ROMs 2-4 were added for the SQ80.

Cheers

Spogg

EDIT: @ tulamide If the WAVs I created are “correct” and fit for purpose then maybe we could use the Mem2FA prim to create float arrays from the WAVs. This would be useful for creating wave tables of course, but may be a redundant step for reasons we spoke about before.

[tulamide]

Here is a library of all ESQ1 waveforms as float arrays.

Select a waveform by name, then click on "Send to float array". The waveform is now available at the float array output.

@Spogg: To be honest, now that we have more possibilities to do the wave files, I'm not sure if 16 bit is the way to go. Flowstone works with float32, so it will convert the bit depth anyway. Why converting to 16 bit and then again to 32 bit. Also, the external programs convert RAW audio over the whole range of values. In case of 8-bit that's 256 values. But Ensoniq only uses 255 values (1-255), while 0 never appears in the waveform data, but is send as a signal to stop/reset the loop. Which means that any conversion ignoring this behaviour slightly falsifies the original waveforms (they get a dc offset).

EDIT: Also, this comment

I’ve done ROM 2 and 3 but I'm a bit puzzled why some seem to contain 2 sounds in one WAV so I’ll be looking into that today and converting ROM 4.

This should not have happened. Are the csv files correctly created? If so, the code I use might introduce an error

[Spogg]

That’s an amazing tool you have tulamide!

I can see you are indexing data and name arrays but where have you hidden them? This is a Ruby magic trick I need to learn.

Regarding the question of rendering the wave data to useful audio I have to say that I don’t know what’s for the best. I don’t think that a dc offset of 0.39% (due to missing zero sample value) is at all critical though. What matters most is the authenticity of the sound, but without A/B/C testing and without clean examples of the actual instrument sounds it’s not easy to decide. I do think the 8 bit quantisation noise is an important characteristic and interpolating that out is probably not desirable in an emulation (but possibly OK in an “inspired by” project).

Now onto the problems with the other ROMs.

Yesterday I spent some hours on this and I’ve come to the conclusion, very reluctantly, that either the spreadsheet is messed up for ROMs 2-4 or the file contents don’t match the spreadsheet (more likely).

Here’s what I did:

I checked my method of csv extraction from the spreadsheet by re-extracting the ROM 1 files and got the same results as you, so I know the csv files for the other ROMs should be ok. Plus, your tool extracted the expected number of waveforms from each ROM, as per the spreadsheet, and the file sizes were correct, even though some extractions contained 2 or 3 three complete “waveforms”. So your tool and my csv files are not guilty.

The contents of the files were certainly not as expected! ROM 2 contained sounds that should not even be in ROM 2. So I assumed the file names for the ROMs were misleading me. So I used each csv file for all the ROM images 2-4 and none of them produced the range of expected sounds. I base this on the obvious sounds; one-shots and the Attack sounds, such as “bowing” and percussion. The single cycle sounds were not possible to identify clearly of course. Also the loop points chosen for some vocal sounds were not good, which is very unlikely for a commercial product. I was able to find better loop points myself!

As an example I found the 2 bowing sounds in ROM 3 image, not in ROM 2. I did this by manually extracting the obvious sounds from the whole ROM images, and also comparing with those AIFF files (these seem to be accurate BTW).

Also I wondered if the first wave in the extracted files was always correct and maybe it was my interpretation of the WSR value that was wrong, but even this isn’t the case. The first waveform, where there are 2 or more included, was wrong too, and that one is taken from the base address.

My suspicion is that the spreadsheet was obtained by reading the OS ROM (which is where the index values and names etc must reside) but the ROM images have not been made correctly. I plan to look more closely at ROM 1 today (jeez I hope that one is ok!).

Just ask if you need any more info about this…

Cheers

Spogg