Maschine code injection (custom compiler?)

[MyCo]

Having dealt with a lot of C-DLLs, I'm a bit concerned about memory allocation in DLLs. It's not as easy as you might think to manage memory the right way. The biggest issue is: How to make sure the memory gets deleted.
In your case, let's say you trigger the "mem_utility" twice, what happens with the memory of the first trigger? This get's a bit harder, when you use the "mem_utility" from more than one DLL-primitives, because then you'll have multiple memories that have to stay alive, and when you trigger one of them, you'll have to delete the original memory of that particular primitive instance.

One solution would be to input the previous memory address to DLL primitive. And the function deletes this before allocating a new memory. For optimization matters, you can reuse the old memory if the new requested size is smaller, too.

However, you'll still have the problem that you can't gracefully shutdown, because when you close the schematic/EXE/VST, you need a trigger that deletes all allocated memories.

[Youlean]

Currently mem gets created/deleted only if size is changed. I can add "about to close" primitive to release memory on closing...
I don't really understand what is the problem when using multiple "mem utility" dlls...

[tulamide]

Having dealt with a lot of C-DLLs, I'm a bit concerned about memory allocation in DLLs. It's not as easy as you might think to manage memory the right way. The biggest issue is: How to make sure the memory gets deleted.
In your case, let's say you trigger the "mem_utility" twice, what happens with the memory of the first trigger? This get's a bit harder, when you use the "mem_utility" from more than one DLL-primitives, because then you'll have multiple memories that have to stay alive, and when you trigger one of them, you'll have to delete the original memory of that particular primitive instance.

One solution would be to input the previous memory address to DLL primitive. And the function deletes this before allocating a new memory. For optimization matters, you can reuse the old memory if the new requested size is smaller, too.

However, you'll still have the problem that you can't gracefully shutdown, because when you close the schematic/EXE/VST, you need a trigger that deletes all allocated memories.

I know exactly what you mean and agree to everything. Memory is the weak spot, because it must be handled with immense care. Additionly it can also lead to unexpected crashes that will then be hard to track down and impossible to solve, since you can't access the dll.

What astonishes me is that what KG wants to see realized has already been done by Martin and me. The work I've done uses Flowstone's memory allocation via Ruby, it is safely freed if not in use anymore or on exit, it is 16 byte aligned, the address is converted to be readable exactly as KG needs it and the module converts arrays to mem for further use in assembler, just as demanded.

[KG_is_back]

Totally forgot about that...

Now that we have that sorted out, it's time to look at assembler (on how to convert assembly instructions into machine code). And how to make it available in Flowstone. I've done some quick search and heaven't found anything particularly useful...

[MyCo]

Compiling assembler is not a big deal. However it takes a lot of time to add instructions because they all have different structures. As far as I know there is no "one fits all" solution, you'll have to add every instruction seperately.

I've started a parser allready. But it's far from being ready. I want to support better addressing, so the parsing is a bit more complex. eg. you would be able to address memory like that:
movaps xmm0,mem[eax + ebx*4 + 1234];
That's actually a supported single instruction using "displacement" (mem + 1234 added at compile time) and "scaled indexes" (ebx*4 is a 4 times scaled register), however this means the compiler has to know the absolute address of "mem".

[KG_is_back]

cool MyCo...
it is a viable solution... suppose we create the mem before assembling the machine code and then we write it there. however, I do not see any benefits of that. the assembler would necesairly have to be part of the schematic, in order for it to work. Way I think about it is something like this:

Code: Select all

push ebx;//fist intruction - here the call will be pointing
mov ebx,/some number/; //the number will be chosen, so that ebx points to the first 16byte aligned position after ret;
//////
//the actual assembler code
/////
pop ebx;
ret;
///here may be some empty bytes if first byte after ret; is not 16byte aligned
//here should ebx be pointing and here the first variable will be.... sincerely mr.Yoda


in such case, the name of the variable will be replaced by [ebx+integer]. In case of arrays, index may be in "reg", "reg*2" or "integer" or any combination of two of them.
Do I make sense?

[MyCo]

Not sure if I understand it correctly. But as you pointed out, the memory must be allocated before compiling. This makes everything a lot easier as the memory doesn't have to be aligned at all.

So I would compile the code into the memory from the beginning, then I calculate how long the machine code is and insert spacing bytes (eg. machine codes for "ret" or "nop")before the variables are placed. The variables start at:

machine code length + ( 16 - ( (memory address + machine code length) % 16) ) % 16

maybe there are easier ways to calculate that.

[KG_is_back]

Correct... I also what to point out one thing. Perhaps we should keep code and variables in separate mems. and use ebp to point to the variable part. this way it will be possible to use the same code on multiple different sets of variables, like in poly.

[MyCo]

Here you go:
viewtopic.php?f=3&p=18767