Page 1 of 2

Packed data in Ruby?

PostPosted: Fri Nov 27, 2015 7:38 am
by martinvicanek
A question for the Ruby nerds out there: how do I create this simple function in Ruby:
pack.png
pack.png (14 KiB) Viewed 33998 times
Reason is I want to avoid unnecessary CPU load. The four inputs are basically constant, however they might change during stream excecution. I know how to code the Pack prim more efficiently in ASM, however that still involves unnecessary processing. When you have hundreds of packed data inputs in your schematic it does make a difference. :mrgreen:
Any ideas?

Re: Packed data in Ruby?

PostPosted: Fri Nov 27, 2015 3:41 pm
by KG_is_back
I do not understand what you are trying to do... Ruby can't output streams. Closest you can get to outputing packed data from ruby is the "alternative" mem that trog did a wile ago and some other guys refined.

Re: Packed data in Ruby?

PostPosted: Fri Nov 27, 2015 8:36 pm
by tulamide
What KG said. Trog's mem access we both refined a few months ago is the best you can do.
Packing is basically making sure that each step of data for the four streams is continously one after the other (plus a few other requirements, that we already solved in mentioned project) in RAM, so that SSE can calculate the four parallel.

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 12:51 am
by martinvicanek
Thanks guys. Yeah, that should be possible, although I was hoping that there would be an easier way. Like using the .pack("f") method or something. The inputs I have in mind are not stream data, they are more like triggered data. Sort of green, but with the four SSE channels being different. Like this:
greenpack.png
greenpack.png (15 KiB) Viewed 33963 times

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 11:20 am
by adamszabo
Why dont you do the packing in assembly and only use stage0 for that? Then I believe it will be only executed at the start and should not use any more cpu?

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 12:18 pm
by tulamide
Ruby's pack method is not the same as Flowstone's pack prim. In Ruby, .pack is just a format converter. You can get a string of 4 values, but the string is not guaranteed to be contiguous in RAM, nor that its address is divisible by 16, etc.

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 1:07 pm
by martinvicanek
adamszabo wrote:Why dont you do the packing in assembly and only use stage0 for that? Then I believe it will be only executed at the start and should not use any more cpu?
Yes, actually that's what I do. The drawback is if I do want to change the input I have to interrupt the audio stream to trigger stage0.
tulamide wrote:Ruby's pack method is not the same as Flowstone's pack prim. In Ruby, .pack is just a format converter. You can get a string of 4 values, but the string is not guaranteed to be contiguous in RAM, nor that its address is divisible by 16, etc.
Ah, I see. Probably the issue is not so much about generating the 4x32 bit sequence within the Ruby module but rather about getting it out transparently. We have only a small set of data types that we can use for Ruby output, and there is some implicit conversion taking place. For instance a green float output will provide 4 copies in single precision (even though internally floating point numbers are represented in double precision).

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 2:37 pm
by tulamide
martinvicanek wrote:Ah, I see. Probably the issue is not so much about generating the 4x32 bit sequence within the Ruby module but rather about getting it out transparently. We have only a small set of data types that we can use for Ruby output, and there is some implicit conversion taking place. For instance a green float output will provide 4 copies in single precision (even though internally floating point numbers are represented in double precision).

Yes, Ruby uses whatever the system provides, for example double precision floats. But Flowstone uses single precision in green and therefore converts any outputs from Ruby to green. I think this is also documented somewhere in the manuals.

To get a series of 4 different values in a certain format, you have to use .pack(), just as you assumed. But, the result will be a byte string. This string is a bit like the mem prim, but Ruby does not take care of the contiguous data requirement, etc. So, you have a byte string, whose content may be in different locations in RAM. That invalidates it for use in the unpack prim.

Btw.: The green to unpack link is a convenience, too. In the background Flowstone does a conversion of the data (it reserves a contiguous memory block with an address divisible by 16, then copies the float 4 times to that mem block, and finally passes it to the unpack prim)

I really don't see an easier way than the one we already did. It is all there, just the block of memory (in Ruby it is a "frame") must be reduced to 16 bytes, and the input single values instead of arrays. You then get the memory address that you can access from assembler

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 4:03 pm
by tulamide
Here's an example.

Re: Packed data in Ruby?

PostPosted: Sat Nov 28, 2015 4:44 pm
by martinvicanek
Thanks, tulamide, this mem create module looks quite familiar ;) Yeah, it works, but it is a vast overkill. And then you still need to check for a valid mem address in the ASM, which is about what a hop instruction would cost CPU wise. Thanks again.