Maschine code injection (custom compiler?)

[MyCo]

Hi,

just played a bit with some less used assembler instructions, and discovered that FS supports "call eax". With this and the recently added memory address stuff, you can do some weird hacking

I've attached a small example that shows, how you can write machine code into a memory and let the assembler primitive execute it. That's basically how FS stream system works internally...

Don't know how usefull this is, but with some creativity, we could build our own language in FS and compile it directly to maschine code running at sample rate.

Regards
Maik

[KG_is_back]

Don't know how usefull this is,

A game changer, I'd say...

I remember doing some tests with the call eax; myself, but at that time I had no idea what I'm doing.

There is another thing that comes to my mind. do only FS-available opcodes work or do any opcodes work?
If any opcodes work then it should be possible to run any code. This includes external functions form DLLs. All we need to do is to mod a DLL to dump the addresses of the individual functions and we can run DLLs on streams. It will not be very efficient though if called every sample. But perhaps it can be usable for things like FFT, which is called on once in a while.
Not to mention a compiler could be done using this.

[MyCo]

do only FS-available opcodes work or do any opcodes work?

Any opcodes work, it is written completely unchanged into memory and the assembler code jumps directly into the memory and executes it as it would be a program.
Problem is managing memory for variables. You'll have to allocate some memory for em, either a separate memory or you have to add some space after your program code, and then in your code you have to give the right address. It can get very difficult to do that by hand, because you'll have to calculate addresses all the time.

[KG_is_back]

do only FS-available opcodes work or do any opcodes work?

Any opcodes work, it is written completely unchanged into memory and the assembler code jumps directly into the memory and executes it as it would be a program.
Problem is managing memory for variables. You'll have to allocate some memory for em, either a separate memory or you have to add some space after your program code, and then in your code you have to give the right address. It can get very difficult to do that by hand, because you'll have to calculate addresses all the time.

Not necesairly. You may simply change the ebp (base pointer) to refference the first position in memory, where variables reside. If I recall correctly variables are read form memory by address in form [ebp + integer] (at least that's what we see when you connect something to analyser prim and observe the ASM code generated by it)
It may look something like this:

Code: Select all

//main program:
push ebx;
push eax;
mov ebx,addressOfVariables;
mov eax,addressOfCode;
call eax;
pop eax;
pop ebx;

//the called code:
push ebp;
mov ebp,ebx;
//your code
pup ebp;
ret;



[MyCo]

Yeah, I know that. But you have to keep the relative offsets in mind when you do that by hand. I would rather use only a single memory, and have the variable memory after the "ret" statement. That way you just have to care about one memory being valid.

Another "one memory only" option would be, to format the memory like that:

Code: Select all

jmp code;
variable memory
code:
...
ret

but this adds another useless jmp

[KG_is_back]

I considered both aproaches you mentioned above...

"jmp" takes only one cycle and processor easily predicts it, so it's not really a big deal.
Problem with one memory is that when mem is allocated it is not necesairly 128bit aligned, which is a must for SSE instructions (this is machine specific, as proven in another topic).
Only way to produce 128bit aligned memory is either to use DLL to create it, or to keep triggering "mem create prim"/float-array-to-mem prim until the memory is 128bit aligned by random. Perhaps Ruby frame may be used for the task too.
Also, to make the thing work in poly, you'd need to have unique mem for variables for each group of 4 channels. Preferable choice would be to create array in DSPcode/assembler and send it's pointer to the code as ebp. Flowstone would then manage the memory automatically for poly. That is not possible right now, unfortunately. and also we would have to put the values into it at stage0; (which I suspect is happening anyway for variables in stream).

[KG_is_back]

I have done some experiments with mem, and it is not possible to reliably produce 128bit aligned mem. I tried by creating a loop that retriggers "float array to mem" prim until its address is divisible by 8. It works like 50% times. Sometimes, the address simply oscillates between two values non-divisible by 16. Ruby frames behave in exactly the same way (I suspect they are identical to mems inside flowstone).

This may also be highly machine specific. Perhaps we should ask Malc to add optional 128bit alignment mode. Maybe a prim with boolean input, that would enable 128bit alignment vs. native alignment. This would also improve SSE processing of loaded waves.

[Youlean]

I have done some experiments with mem, and it is not possible to reliably produce 128bit aligned mem. I tried by creating a loop that retriggers "float array to mem" prim until its address is divisible by 8. It works like 50% times. Sometimes, the address simply oscillates between two values non-divisible by 16. Ruby frames behave in exactly the same way (I suspect they are identical to mems inside flowstone).

This may also be highly machine specific. Perhaps we should ask Malc to add optional 128bit alignment mode. Maybe a prim with boolean input, that would enable 128bit alignment vs. native alignment. This would also improve SSE processing of loaded waves.

Hey KG, what about mem DLL that I created? Can you actually use mem that was created with DLL?

[KG_is_back]

I have done some experiments with mem, and it is not possible to reliably produce 128bit aligned mem. I tried by creating a loop that retriggers "float array to mem" prim until its address is divisible by 8. It works like 50% times. Sometimes, the address simply oscillates between two values non-divisible by 16. Ruby frames behave in exactly the same way (I suspect they are identical to mems inside flowstone).

This may also be highly machine specific. Perhaps we should ask Malc to add optional 128bit alignment mode. Maybe a prim with boolean input, that would enable 128bit alignment vs. native alignment. This would also improve SSE processing of loaded waves.

Hey KG, what about mem DLL that I created? Can you actually use mem that was created with DLL?

I definitelly can... Actually I was just working on it, but I suck at C++ does your DLL produce 128bit (16byte) aligned memory?

[Youlean]

I have done some experiments with mem, and it is not possible to reliably produce 128bit aligned mem. I tried by creating a loop that retriggers "float array to mem" prim until its address is divisible by 8. It works like 50% times. Sometimes, the address simply oscillates between two values non-divisible by 16. Ruby frames behave in exactly the same way (I suspect they are identical to mems inside flowstone).

This may also be highly machine specific. Perhaps we should ask Malc to add optional 128bit alignment mode. Maybe a prim with boolean input, that would enable 128bit alignment vs. native alignment. This would also improve SSE processing of loaded waves.

Hey KG, what about mem DLL that I created? Can you actually use mem that was created with DLL?

I definitelly can... Actually I was just working on it, but I suck at C++ does your DLL produce 128bit (16byte) aligned memory?

I can edit it so that it does... Do you know why it actually needs to be 128bit aligned? Can this be connected to fact that different machines take different amounts of bytes to store float value?
Can you make me simple read from address in asm so I can test mem create?