Logistics/Documentation

[deraudrl]

A two-part question, hopefully related:

1. How do you guys keep track of your modules? I'm familiar with the tag system, but with everything at one level (vs some sort of hierarchy), it seems like adding things to the toolbox can get out of hand rather quickly. I'm considering creating large meta-schematics full of related disconnected modules to cut/paste from, but was hoping for something a bit more elegant.

2. Are there any tools available for documenting new modules? I'm thinking in terms of something you'd use if you were trying to add new entries to the component reference and preferred some sort of fill-in-the-blank template rather than brute-force. Maybe an external app that looks at 'fsm files and extracts at least the top-level name, inputs, outputs, comments, (maybe) Ruby/ASM code blocks, and outputs it as text or XML.

[trogluddite]

Indeed; my personal preference would have been to allow nested folders on disk and a tree-view, and I often resort to the same "meta-schematic" solution, in combination with naming conventions to make toolbox searches more productive. Besides that...

The 'Tooltip help' primitive can be, erm.., helpful. This allows some multi-line documentation text to be added to its containing module. As well as showing up as a tooltip when you hover over the module in the schematic or toolbox, the text is also included in toolbox searches.

It's also possible to give inputs and outputs both short and long labels, which can be helpful documentation of their functions, values ranges, etc. When labelling them, write the short name first, followed by \n then the longer label (the backslash "n" represents a new-line character). The long label is only shown when the module is selected. However, this text can only be a single line and isn't searchable (and isn't always easy to read amongst the clutter of a schematic!)

Finally, local toolboxes can sometimes be helpful. Right-clicking on a module and selecting "Create Local Toolbox" makes every module nested within the chosen one (and only those) directly available from the toolbox. Selecting "Create Local Toolbox" again turns the feature off (so it's really "Toggle..." not "Create..."). It's not ideal, as it only applies to the schematic containing the module, nor can you switch between toolboxes without navigating to the module, nor even see any indication which module is being used in this way. To be fair, it's really intended to allow teachers to restrict which components their students can access, but for small collections of closely related components, I find them handy occasionally for speeding up tedious cutting and pasting.

[deraudrl]

The 'Tooltip help' primitive can be, erm.., helpful. This allows some multi-line documentation text to be added to its containing module. As well as showing up as a tooltip when you hover over the module in the schematic or toolbox, the text is also included in toolbox searches.

Ok, I knew about the tooltip thing, but that makes it a lot more useful.

[deraudrl]

Just an FYI, although probably everybody but me already knew it...

Just for jollies, I threw a couple of smallish schematics into a hex editor. As expected it looked like total gibberish, not a hint of useful readable text anywhere...until I noticed that the first 12 bytes or so of each were identical and they started with the ASCII characters "PK": yup, they're just zip files. ("PK" is Phil Katz, who co-created the ZIP file format.) So I renamed it "foo.zip", did an 'extract all', and got a file called "schematic.osm". Clicked on that, and FS started up and loaded it without a whimper. When I put that file into the hex editor, I found a bunch of text: component names, tooltips, assembly code, Ruby code etc. I also saw some embedded PNG files.

It's a small sample, but everything from ancient SM forum archive stuff (.osm) and brand-new v3.0.x .fsm files seem to be the same format. The .hom module files in the install directory also appear to be the same thing, just uncompressed.

I don't suppose there's any documentation describing the .osm/.fsm file internal format, is there?

[trogluddite]

Just for jollies...

So you're a fellow reverse-engineer! Good show!

I don't suppose there's any documentation describing the .osm/.fsm file internal format, is there?

Nothing definitive that I've ever seen or heard of. Minimal documentation and caginess about implementation details have usually been the norm, so FS/SM has long been a playground for reverse-engineers. Back in the SM days, there were certainly a few people who understood at least part of the format and tinkered with binaries. It might be arcane knowledge that's been lost over the years ("magic" to me at the time), but I'd be happy to be proved wrong.

The current dev' is a good guy, recruited from the forum, so it may be worth asking him; but I suspect that it wouldn't be for him to decide, and experience suggests that DSPr wouldn't be enthusiastic.

[deraudrl]

So you're a fellow reverse-engineer! Good show!

Yup...did enough forward-engineering over my career that I figure I need some of this for karmic balance.

Nothing definitive that I've ever seen or heard of. Minimal documentation and caginess about implementation details have usually been the norm, so FS/SM has long been a playground for reverse-engineers. Back in the SM days, there were certainly a few people who understood at least part of the format and tinkered with binaries. It might be arcane knowledge that's been lost over the years ("magic" to me at the time), but I'd be happy to be proved wrong.

The current dev' is a good guy, recruited from the forum, so it may be worth asking him; but I suspect that it wouldn't be for him to decide, and experience suggests that DSPr wouldn't be enthusiastic.

Didn't figure it would be easy...

I can't tell, from where I'm standing, what kind of organization DSPr actually is: to paraphrase Snoopy, "On the Internet, no one knows you're a two-man garage shop." Reading between some of the less-obvious lines, I'm thinking their main revenue stream is licensing to largish-corporate robotics/automation clients, with any development on the "SynthMaker" side relegated to a side-line at this point. (If nothing else, it seems to be taking longer to do the 64-bit port than the original development took, which is simply insane. Then again, I could be miss-reading the whole situation.)

[trogluddite]

I can't tell, from where I'm standing, what kind of organization DSPr actually is...

You're not too wide of the mark, I don't think. My impression is that there's one key person, who has various small collaborations on the go. So far as I know, SM/FS has never had more than one or two coders working on it, and even then, only intermittently. If what follows is a rather too long history lesson, please forgive me - I don't get to tell the old folk-tales very often!

The coder who initially wrote SM didn't hang around for very long, and development gradually slowed to a crawl over the next few years, though there was a push for a while in the direction of embedded-PC applications. As now, official documentation and examples were thin on the ground, so forum users put a lot of work into "adding value" to the product.

The change to FS came around the time that the UK education system was having one of its regular, and usually laughably pathetic, "teach all the kids coding and robotics" drives. I can understand that DSP-freaks aren't exactly the biggest market in the world, and why this opportunity led to development restarting, of course (I was working in the science education sector myself at the time). A new business partner popped up at this point. Then all hell broke loose!

The SM forums were to be closed with the loss of all content, there seemed very few new audio DSP features, and the new licensing system for "unlimited updates" wasn't trusted given previous performance. The discovery that the new business partner was running a sock-puppet account on the forums didn't exactly help matters! A few dedicated members saved what they could of the forum content, but a lot of the most valued forum members packed their bags. The school/college market that FS was intended for became saturated very quickly and those customers doggedly hang onto old kit for years, so I'd be surprised if FS has been a roaring success. Cue several more years of hardly any development and buggy releases, and new work-arounds to discover (in particular, the Ruby embedding, while clever, seems hastily gaffer-taped together at times).

More recently, the code-base was handed to a very talented and much respected former forum member and DSP fanatic. What the business arrangement is, I have no idea; but it isn't his full-time job, maybe not even a paid job at all. The alphas and commit notes that I've seen suggest that he is doing a fantastic job not only of the 64-bit conversion, but also of updating VST and Ruby to current standards, enhancing the application in general, squishing long-standing bugs, and refactoring a lot of ugly old code. Quite what will happen when it's ready for release, however, is anybody's guess.

So not quite abandon-ware (though the forum certainly is!), and some genuinely positive developments. However, I don't have total confidence that DSPr won't do something silly again on release day - fingers crossed!

[deraudrl]

You're not too wide of the mark, I don't think. My impression is that there's one key person, who has various small collaborations on the go. So far as I know, SM/FS has never had more than one or two coders working on it, and even then, only intermittently. If what follows is a rather too long history lesson, please forgive me - I don't get to tell the old folk-tales very often!

Ah, yes. I'm reminded of a time (late 1985 or so) when I purchased a C compiler for my first PC clone, only to discover that I had two copies of disc 3 and was missing disc 2. The phone call to "tech support" had a small delay in the middle so the "chief software developer" could hand off his infant daughter to his "shipping department director" for feeding. It was my first glimpse into the then-new non-corporate software cottage industry. (My extremely corporate employer eventually bought me a copy of Microsoft C when it became available...tech support interactions were decidedly less amusing after that.)

More recently, the code-base was handed to a very talented and much respected former forum member and DSP fanatic. What the business arrangement is, I have no idea; but it isn't his full-time job, maybe not even a paid job at all. The alphas and commit notes that I've seen suggest that he is doing a fantastic job not only of the 64-bit conversion, but also of updating VST and Ruby to current standards, enhancing the application in general, squishing long-standing bugs, and refactoring a lot of ugly old code. Quite what will happen when it's ready for release, however, is anybody's guess.

So not quite abandon-ware (though the forum certainly is!), and some genuinely positive developments. However, I don't have total confidence that DSPr won't do something silly again on release day - fingers crossed!

In a perfect world, an earlier (but useful) version would have gone open-source with your current Alpha Geek as curator...spread the load and all that. Worst case, DSPr could have held back non-essential things like external DLL linkage and continued to provide value-added custom versions to whoever makes up their customer base at this point.

Random background note: My own background was (now retired) aerospace SW development, about half of it concerning DSP and acoustics (sonar, not music). I seem to get the urge to dabble with synths every decade or so, starting with Roland hardware in the early 1980s, then PC soundboards in the mid-'90s, and playing with the VST SDK and associated tools around 2008 or so. I vaguely recall seeing SM then, tried SynthEdit instead (because it was cheaper) and was somewhat underwhelmed. My main issue this go-around is getting my head around the (rather baroque IMHO) interface between the working bits and the GUI; I have a lot of experience with "normal" GUI toolkits and there's a bit of culture shock kicking in. I tend to try to keep cosmetics well separated from functionality, and I'm not sure that's really possible here. We'll see how this goes.

[trogluddite]

In a perfect world, an earlier (but useful) version would have gone open-source with your current Alpha Geek as curator...spread the load and all that. Worst case, DSPr could have held back non-essential things like external DLL linkage and continued to provide value-added custom versions to whoever makes up their customer base at this point.

As much was mooted as far back as the SM/FS migration - and back then, there would have been plenty of users with both the product knowledge and the coding chops to have made the most of it. However, as their attitude to the forums shows, DSPr often seem blind to the value added to the product even by the application examples, tutorials, and mentoring of the user-base. If it's going to be popular, a coding language needs more than just an API reference (which is pretty much all that the documentation amounts to, IMHO.) I have seen many novices stumble when they reach the point where the features intended to make FS beginner-friendly reveal their complexities and they discover that programming a Turing machine graphically can require just the same conceptual understanding as any other way of doing it (not to mention unit-testing and refactoring!)

I tend to try to keep cosmetics well separated from functionality, and I'm not sure that's really possible here

It can be more difficult, especially if working with stock or downloaded modules, and it's not a common way of working. I do find it helpful at times to use separate modules for processing and GUI with wireless links or a bus used to share the control parameter data, but it's not always easy to arrange. A Ruby GUI framework is something that I've tinkered with over the past few years, but this has often been hampered by the way that Ruby and the GDI+ renderer are integrated into FS and the minimal support for external libraries (something else I've been looking into).

And finally; it's nice to get to know a bit more about you. If I haven't scared you off already with my horror stories, I'm going to enjoy having you around, I think.

[tulamide]

Don't want to sound over-dramatic here, but it is a fact that without MyCo, Flowstone would already be dead by now. Malc doesn't have any reserve assets and for years now does a fulltime job, to keep his family sorted. Maik jumped in as a volunteer, getting nothing for his work. His wish to keep Flowstone alive is the only reason he's hard at programming in his spare time (he has a fulltime job as well).

So, instead of dreaming about a global Flowstone success, I tend to be humble and appreciate MyCos work getting Flowstone up-to-date. It's way too easy for us to judge from our sofas what DSPR should or should not have done. They tried to make it profitable, they failed, and now it's in the hands of a volunteer to at least keep the VST part of Flowstone alive.