Custom DSP Code 2 (comunity project)

[tulamide]

Very good, KG!

Since you didn't say something to my idea of a visual frontend I assume you are not interested in it. I will try then to create a keyword colored editor.

[KG_is_back]

Very good, KG!

Since you didn't say something to my idea of a visual frontend I assume you are not interested in it. I will try then to create a keyword colored editor.

frankly, it seemed like recreating flowstone in flowstone. Or did I got it all wrong?

[tulamide]

frankly, it seemed like recreating flowstone in flowstone. Or did I got it all wrong?

No, you didn't get it wrong. Of course, it wouldn't be a Flowstone, just graphical elements you could connect to each other. Currently there's no way to generate optimized ASM code without knowing it. Writng code in the DSP module is the closest you can get, but that again is not so easy for people without programming knowledge (it's too abstract).

But, I admit, writing a text editor would probably be a smaller project and could be realized faster, which this project then would benefit from. Also, it's just a frontend, so can be made anytime later.

[KG_is_back]

Hello friends!

The project is slowly coming together. Today I finished the lexer and parser (the front end).

I what to discuss some stuff regarding syntax of the DSPcode2, so that I wouldn't have to redo the stuff after it's finished. Here's sort of a manual:

Inputs, outputs and common operators (normal brackets + - * / % and comparisons) work the same way as in original DSP code. I've also added/planed several new operators

Code: Select all

a.+b //integer add
a.-b //integer subtract
a^b //the same as pow(a,b)
a&!b //logical and not


Also now you can use integers (both declare them using "int myVariable;" and use integer constants by preceeding the number by "?" example: " a=?1;"=assign integer 1 to a)
Stages work the same way except that now you must put ";" after the "}"

assignments may now be used in a middle of a line. That means this code is valid in DSPcode2:

Code: Select all

a=(b=2+3)*4; //assigns 5 to b and 20 to a
//it is the same as this in DSPcode1, only a little faster
b=2+3;
a=b*4;


Added "array.[ index ]" operator, which is basically a mono version of normal "array.[ index ]". It only uses data in channel 0 to define index reads/writes all values at that index to/from array. When you use "[]" or ".[]" with a variable (or a expression), instead of array, the variable will be considered a pointer. This way you may use real external mems by providing their address.

In DSPcode2 you can use codeblocks, which act as a single expression, but you can use multiline code inside them. They also return a value. By default the value of last expression, or you may use "return" statement to return specific value. "return" will also skip the rest of the codeblock (useful when combined with if/else statements). In codeblocks you may declare local variables (their names may match with other codeblocks, but their value will persist for each codeblock its own) which will also be available inside the codeblock (including its child codeblocks) but not outside it. Makes variable naming easier.

Code: Select all

a=11+{b=a^2; b=b*c; }+3;

b={ a=4; if(b==a) (return (a*b)); a=3;};


hop() and loop() may now take non-constant argument for changing loop/hop length on fly. You do not need to use "{ }" for singleline code. They will also return a value, so they can be used as expressions too (same as codeblock).
Also added while loop and repeat while loop. Only result in channel 0 of the condition has effect.

Code: Select all

a=4;
hop(a) b=a+b; //this will only execute the line every 4th sample.

a=hop(16) (5); //each 16th sample a=5, all other samples a=0

a=0;
b=loop(100) { // result is b=100.5
a=a+1;
return (a+0.5);
}

a=2;
while(a<100){
a=a*2;

} //stops when a==128 (above 100)

repeat() { //the "repeat" statement looks like a function call with no arguments
a=a+1;
}:while(a<100);



As mentioned, code branching is also possible - via "if(condition) expression;" statement (expression may be anything - an assignment, codeblock, value). It also has optional "elsif(condition) expression;" and "else expression;" statements, which must be separated from the previous ones via ":" instead of ";". "If" statement also returns a value - a value of the expression that was executed. If "else" statement is not provided and none of the conditions in "If" and "elsif" statements was met, it returns 0. Also only the first branch that met the condition is executed.

Code: Select all

a=1;
if(a==0) b=3:
elsif(a==1) b=5;
elsif (a<0) b=10;
else b=6; //result is b=5. "elsif (a<0) b=10;" is not executed because previous statement was true.

a=if(b==1) 4; //result will be a=4 or a=0 depending on the value of b


Functions will be provided in packs of ASM blueprints. To install a pack you simply copy it to text component and connect it to a string array builder inside the DSPcompiler. Functions will always be placed inline (no real function calls ATM ). It will also be possible to delcare a function directly in the code in this form:

Code: Select all

function myFunc(arg0,arg2[]) {
//some code
};


body of the function is a normal codeblock. Variables you use in the function are global (you must declare them yourself). Local variables will be made unique for each function call and will have value persistence (will keep their value to next samples).
Arguments for the function may be either variables or arrays (when input should be array, you just type in the name without [] in the function call). For functions declared as ASM blueprint packs the input(s) may specifically be a constant, or may generate different code when one or more inputs are constants. For example:

Code: Select all

pow(a,2)
//will generate this:
movaps xmm0,a;
mulps xmm0,xmm0;

//instead of lengthy and CPU heavy power function.




Creating ASM blueprint packs for functions will be relatively easy, but will require some basic Ruby skills. Unless you intend to do something completely crazy like unwrapped loop generator, it will be a matter of copy-pasting your ASMcode into a template and renaming a few things here and there.

functions that will be in a default pack:
All functions from original DSPcode (but most likely heavily optimized if possible)
MartinVicanek's stream math functions (includes htan(x) log2(x) sin(x) cos(x) ...all heavily optimized)
shuffle(var,int,int,int,int) - a shuffle function, that may be used to swap channels. Especially useful for "if" and "while" and "array.[]" statements, since they react to channel 0 only
any(var) - does logical OR of the 4 channels (basically results true if any channel is true) - again useful for "if" statements
all(var) - does logical AND of the 4 channels (results true only if all channels are true).
not(var) - logical NOT
...
please leave suggestions (FFT is also planned for (far) future)

DSPcode2 will no longer have the issue with only 8 successive operations and overall will manage registers more efficiently (including automatically using temporary variables when runs out of xmms).

Well, I believe that is all I wanted to say. Please express your opinions, submit suggestions and ask questions - there is most definitely some stuff I've missed/overlooked/done in a stupid way. Reason why I choose "?" to signify integer constants is, that they are used exceptionally rarely (because of lack of support in ASM). I've considered using "1.0" form for floats and "1" form for ints, but then everybody would be "why do no work when 1 and work when 1.0?" type of thing. (note for the noobs: SSE variables do not carry "type" - only binary structure. Every instruction assumes the inputs are in correct format and often you want to specifically use "incorrect" format (for example when using ints as bitmasks)).

[Perfect Human Interface]

Correct me if I'm wrong, but I don't believe you can simply Not something ('!variable') in current DSP code. Is that also planned for this?

Sorry I don't really have more input to offer. This all seems very ambitious though. It will be impressive if you can get all this up and running!

[KG_is_back]

Correct me if I'm wrong, but I don't believe you can simply Not something ('!variable') in current DSP code. Is that also planned for this?

I thought someone would bring this out. No, you can't NOT things in current DSPcode in flowstone. It is possible in assembler using andnps (ANDNOT), but that produces syntax coloring bug and makes debugging the assembler very hard. andnps also enables subtraction of integers, because (-a) = (1+!a).
Since NOT operator only takes one argument (and not two - one on left, one on right) I must've added it as a function, having the one argument in brackets (because I'd have to add checking for "!" all around the place). I'm not decided yet what syntax I'd use: two that I considered: not() !()

Sorry I don't really have more input to offer. This all seems very ambitious though. It will be impressive if you can get all this up and running!

As I've said, I have the lexer and parser done. That basically means the compiler can recognize and correctly interpret (or "understand" we can say) the code even right now (can detect operator priority in the line, differentiate between brackets and functions, etc.) and build intermediate representation. Last thing to do is, to write the back end, that can convert the IR into an assembler code. That means, it basically appends assembler blueprints (basically strings) in the order specified in IR (plus some magic stuff - changing names of xmm registers and names of labels, so they don't interfere with each other across the code).
A few posts back I've posted version 0.3 which worked (and produced almost indentical ASM to flowstones DSPcode).

No, you didn't get it wrong. Of course, it wouldn't be a Flowstone, just graphical elements you could connect to each other. Currently there's no way to generate optimized ASM code without knowing it. Writng code in the DSP module is the closest you can get, but that again is not so easy for people without programming knowledge (it's too abstract).

I had a look at the idea and I came to conclusion that it is hardly possible. In high level programing language you have obvious connections between the operations (like when you write "(x+1)*y" it is obvious, that result of x+1 is multiplied by y). In low level programing languages, there's no such thing. Each operation is a fullblown statement (it is used only for it's side-effects and doesn't return a value) reads some values form memory or registers and updates values in memory/register. Sometimes, you can't even specify which ones (for example in some FPU instructions like fadd; or fxch;), so you can't really make a "connect with lines" sort of thing without making it awfully inefficient.
What we can do though, we can make a "visualizer" that will draw a string (like a line) trough each xmm and each variable, to visualize where it is used and where it's value is changed.

[Exo]

Excellent KG!

Sounds like you have everything covered so far.

I cannot wait to use it this

Once it is done I will push Malc to add a component like you have mentioned that just takes ASM code as text as input. I'm sure he will add that when he sees how much effort has been put into this.

By the way Malc has not got back to me yet with the new opcodes to test, I will give him a nudge tonight and see where he is with that.

If you want a hand with the back end just let me know.

[martinvicanek]

Wow KG, you have quite some progress there!
Can't wait to try it out.

[Perfect Human Interface]

So if/when this is completed, assuming all goes according to plan, will there be any reason to use the original DSP code box instead of this?

[KG_is_back]

So if/when this is completed, assuming all goes according to plan, will there be any reason to use the original DSP code box instead of this?

I can think of few reasons:

1. DSPcode box has colorcoding, so it's easier to debug. DSPcode2 doesn't ATM - it's just a compiler, with green string input to which you send your code from text/string component. Hopefully someone will make a code editor for it in the future.

2. DSPcode is more stable (it doesn't crash under normal circumstances). DSPcode2 relies on several factors to be correct and may crash, if some of the used functions/operators has buggy blueprint or is used incorrectly. For example reading/writing of pointer ( "variable[index]" ) will crash, if the variable is not a valid address, or index is out of range. Both are impossible to check automatically with current version of Flowstone.

3. Legacy reasons ... some users might opt to stick with the DSPcode because of laziness or fear that DSPcode2 is more complicated (which isn't - it just has more features). Or they don't consider the new features and improvements worth a try. I myself still remember the times, when I was making a jungle of green/blue instead of using ruby/assembler, because I was lazy learning them.