DSP Robotics Support

Hi All,
There have been a few questions lately that bring up the subject of multi-dimensional arrays. This is something that we never had before Ruby, and is pretty tricky to implement using the 'green' primitives.
In fact, I would go so far as to say that, although the funky vector graphics, .dll access etc. are all very tasty Ruby features, the ability to build proper data structures is the real 'killer app' - whether you want to store a load of MIDI notes for a sequence, create a song database, or store tabulated results from your number crunching, Ruby will handle it like a breeze.

First a quick intro to Ruby's arrays and hashes for those who have not had a chance to play with them yet...

Arrays - Aside from the fact that there are zillions of different built-in methods for manipulating arrays, they have one great feature that 'green' arrays don't - they can hold just about ANY kind of data, and you can mix different kinds of data within the same array. No longer do you have to restrict yourself to all-integer, all-float, or all-string contents.
And this is crucial to multi-dimensional arrays...
Ruby doesn't really have multi-dimensional arrays at all - you can't for example, declare an array in the form "Array[10,20]", as you might in many other languages. But, Ruby arrays can contain other sub-arrays as data items - and you can keep nesting them inside each other as far as you want. This is the key to multi-dimensional arrays, and in some ways is more versatile than the more common kind, as you can easily extract whole "rows" of data and treat them as a single object (because an Array is a single object in Ruby). the structure ends up a little like this...
main_array = [ row1_array, row2_array, row3_array]
Where each 'row_array' holds a series of values representing each column of that row.
To get at a row, use its index...
this_row = main_array[x]
..and then get the column within the row...
result = this_row[y]
This can be simplified to...
result = main_array[x][y] # two sets of brackets! NOT [x,y]

Hashes - hashes don't exist in 'green', but once understood open up some great new possibilities. In a hash, there are no numbered indexes, and items are not stored in any particular order - but you can give each item a 'key' to reference it with, and Ruby is pretty flexible about what you can use as keys. Here's a very quick example using strings as the keys...

Why do I mention hashes when I am supposed to be showing you arrays? Well, because there is a sneaky way to make a Hash behave like a multi-dimensional array. It is not as versatile as the 'nested sub-array' method, but it needs a little less code and can be slightly more efficient.

And now for the examples - these are fully functioning modules, so can be plugged into your 'green' circuits if Ruby is not your cup of tea...

There are two methods shown in here for making a two-dimensional array - with a 'maker' and a 'getter' for each...

- The top one is what you might call the 'canonical' method, and is the one that most Rubyists would recommend for most typical cases. It is the one most compatible with Ruby's many array methods for sorting, filtering etc., and the array itself has a form which is intuitive to view.
The main point to watch out for is the way that empty elements are dealt with - making sure that each element in the outer array has a valid sub-array for each of its 'rows', and getting a valid 'default' value to come out for 'cells' which haven't been assigned yet.

- The lower one shows it done using a Hash - it's not a multi-dimensional array at all in reality, but the hash uses keys which encode the 'X' and 'Y' indexes in such a way that it behaves exactly the same as the top example, and is much leaner code. However, it does restrict the ways in which you can process the data - in particular, there is no simple way to access a whole array row or column, and counting through the array in order is much trickier without Ruby's cool Array handling methods. But still, it could be handy if you have a particularly large array that only requires the simplest of 'set' and 'get' functions, and you need the very fastest code possible.

Note also that you can't mix and match these two methods. Although all Ruby 'value' connectors look the same, they are able to carry any kind of Ruby object, so it is up to the programmer to be sure that the output from one Ruby primitive matches the input that the other one expects to see - giving the 'hash' style getter an array as its input will just confuse it!

I got to download the example posted ... but JUST as important was the text of your post.
I know its' been asked before, and I'll state it again ... Trog, I would wish you to write a book about topics
like this. You relay programming concepts, technique, and perspective in a manner that allows even the novice to feel that they can grasp. DSP Robotics would be wise to make an offer. If not them ... do a eBook. I would buy it just for the possible Cover design and Title. It'd be a best seller!

I only had a brief moment to peek at the example you posted. Without doubt, I have homework to do with this. Everything you describe in text touches many aspects of my first project. Even though my current project is nothing overwhelmingly complex, and I've greatly improved my schematic logic and layout, I can see being about to condense portions of the 'data' sections to further streamline the design.

I keep calling it my first project, but actually ... though being the first, it encompasses a 'core' that is the base for many other projects. As I learn, I'm focused on a future design concept that I'm building toward.

This is great stuff !!

Big THANKS!!!

Thanks, RJ,
Sadly, I doubt I'll ever have time to write that book - but it's true, the many Ruby resources available out there are great, but simply cannot put Ruby into the context of FS, or refer to things in terms that SM/FS users are familiar with (and I get sick of trying to filter out all the 'Ruby on Rails' stuff!) - we do need more Ruby lessons tailor made for our particular little community.
The forum does offer one advantage though - instant feedback from the readers! While it's always lovely to hear "Thanks, mate, that really helped", it's just as valuable when someone posts to say "Whoa, you just confused me even more!" - and then there's the chance for another user to post with a different perspective that might make more sense to some people, or for the original writer to make some tweaks. (...or just to explain what Ruby has to do with "..cups of tea" for our non-English readers!)
So, I think rather than a book, the dev's should be thinking about a little more "editorial control" on the forum itself - someone to collate the valuable stuff, categorize it, check for factual errors, and put it in a nice safe place that we can easily get at it. Of course, that involves some manpower which they surely cannot spare right now - but I work for a small company that makes gizmos for school/university science labs, and I know that the biggest attraction for many of our customers is not the hardware itself, but the steady supply of professionally authored and translated teaching materials that we give away on our website. We can see the jump in sales very clearly when a new product's tutorials and student work-books 'go live', and the work involved pays for itself many times over.

EDIT) Oops, I just noticed that there is a thread discussing exactly this topic - sorry folks!

Just re-upped the example schematic.
This adds a new module which finds the X,Y size of the 2D array.

It's also a good example of why you wouldn't use the Hash form of the 2D array very often...

Firstly - the '.max' method that it uses isn't available for Hashes - so that nice little one-liner of code would be out of the question!

Secondly - counting the size of a Hash wouldn't work right either. For an Array, 'cells' that you haven't assigned anything to yet are still held in memory, containing the 'nil' value. So if your first array assignment was to cell [0][10], cells [0][0] to [0][9] would also be created with empty contents.
That wouldn't happen in the Hash version, you'd just get a single entry named after the 'special' hash code for '[0,10]', and the length would be counted as one - empty items just don't exist! And you can't reverse the 'hash-code' to find out what indexes were originally used to make it, so the 'maximum' cell position would be forever lost unless you wrote a whole load of extra code to remember it.
Note though, that this tells us something else - if your X,Y matrix was very large, but also very "sparse" (only a few 'cells' with useful numbers), the Hash would use less memory by not storing 'cells' with the default value.

That is a great thing. Maybe we can do a piano roll.
No... Trog?
Cheers!

pall wrote:That is a great thing. Maybe we can do a piano roll.
No... Trog?

Oh yes, that would be a great little project - no arrays of MIDI notes in 'green'! I'll bet if you looked inside MyCo's cool new MIDI file classes, you would find nested data structures very similar to these - and many other funky Ruby ways to organise things too.
Even a MIDI event, although it is a class all of its own, in principle is just a "container" for a list of values, [status, channel, data1, data2]. That's something for a later tutorial - "wrapping" data into new classes of objects adds a whole new level of convenience and power!

I think I go check how to convert hebrew gematria system into modulator wavetable driver.

trogluddite wrote:I'll bet if you looked inside MyCo's cool new MIDI file classes, you would find nested data structures very similar to these - and many other funky Ruby ways to organise things too.

Yeah, it's really "funky". It's somehow nested, but the most important part is an extended Array, which means I've changed some methods that the Ruby standard Array class provides. The result is a self sorting Array.

tester wrote:I think I go check how to convert hebrew gematria system into modulator wavetable driver.

trogluddite wrote:...these are fully functioning modules, so can be plugged into your 'green' circuits if Ruby is not your cup of tea..

Not a tea drinker,, then? - no worries, like Chakl is doing with my XML hieroglyphics, and MyCo's filter kit (nice new update), we can still make Ruby useful to everyone with a bit of "modularising"!

MyCo wrote: The result is a self sorting Array.

trogluddite wrote:...a whole new level of convenience and power!