Support

If you have a problem or need to report a bug please email : support@dsprobotics.com

There are 3 sections to this support area:

DOWNLOADS: access to product manuals, support files and drivers

HELP & INFORMATION: tutorials and example files for learning or finding pre-made modules for your projects

USER FORUMS: meet with other users and exchange ideas, you can also get help and assistance here

Transfer array between instances

For general discussion related FlowStone

Transfer array between instances

Postby HughBanton » Sat Oct 31, 2020 3:45 pm

Does anyone know the way (there's got t' be a way ..) to transfer data from one FS instance to another?

I want to send a 'control' array from an .exe (probably) to VSTs (probably). Array size would be around 2k.

I've got such a thing working internally inside a schematic, but I reckon it would be rather neat to centralise and extract the control part and run it externally.

Am I looking at win32api or something? Assume I know nerrthing ...

H

(I'm also interested in running the control networked, from a different PC, but first things first 8-) )
User avatar
HughBanton
 
Posts: 185
Joined: Sat Apr 12, 2008 3:10 pm
Location: Evesham, Worcestershire

Re: Transfer array between instances

Postby deraudrl » Sat Oct 31, 2020 4:34 pm

Seems like we discussed 'shared memory' implementations here recently...seems like the place to start.
("Recently" defined as "since I got here" 8-) ).

I'm not finding it in a quick search. It may also have been in a tutorial or FS Guru post...
I keep a pair of oven mitts next to my computer so I don't get a concussion from slapping my forehead while I'm reading the responses to my questions.
deraudrl
 
Posts: 201
Joined: Thu Nov 28, 2019 9:12 pm
Location: SoCal

Re: Transfer array between instances

Postby trogluddite » Sat Oct 31, 2020 4:41 pm

The best solution will depend on how often and how promptly you need the data to be transferred.

The simplest, but slowest, way would be to save the data to a file. The receiver would then poll for changes to this file and load the data whenever the file has been saved/modified. This would be best done with Ruby, as it has the necessary methods for testing a file's modification time, and makes it relatively easy to save/load files containing arbitrary data. This method might not be as slow as you think - Windows uses "write behind" caching, meaning that the saved data can be loaded by another instance directly from memory before the file actually gets written to disk (it might not get written to disk at all if you delete it as soon as you've read it). However, you don't want Ruby to be polling the file system too often, as this is a relatively slow process which might cause a Ruby bottleneck - but it would fine if you can stand maybe a second-or-so's delay between updates.

If you need the transfer to be more responsive than that then, yes, you'll need to start delving into the Windows APIs - in particular, something called a "MappedFile". These use the "write behind" cache mentioned above to create a special file which behaves as a chunk of memory which can be shared between applications. Anything written to the file is instantly visible to any app which has the file open for reading. You would still have to poll the file, but this can be done using a simple flag inside the file data, which can be tested as quickly as reading a normal variable.

The spanner in the works is that Ruby's Win32API library doesn't work inside VST plugins, only EXEs (contrary to what the User Guide says). To use the MappedFile technique inside a plugin, you'd have to use the DLL primitive and write a custom DLL for it using e.g. C++ in Visual Studio. I did make a start on such a beastie a few years ago, but unfortunately I ran into problems that were a bit beyond my C++ coding chops!
All schematics/modules I post are free for all to use - but a credit is always polite!
Don't stagnate, mutate to create!
User avatar
trogluddite
 
Posts: 1706
Joined: Fri Oct 22, 2010 12:46 am
Location: Yorkshire, UK

Re: Transfer array between instances

Postby HughBanton » Sun Nov 01, 2020 2:44 pm

Most interesting, and Mapped File would seem to be the best route to take, even if I later opt for access from a remote pc. It does all appeal.

Unfortunately Trog, my C++ skills don't come anywhere near ++, more like C-- (or maybe C/99.9?) last time I tried anything serious :lol:

I did actually briefly employ DLL modules - pheww - when I first returned to FS in 2017, (I'd just flown in from Arduino Land), but I have't done much in C for ages and I'll be rustier than ever.

Still .. time on my hands for some strange reason, I might rise to the challenge!

Thanks

H
User avatar
HughBanton
 
Posts: 185
Joined: Sat Apr 12, 2008 3:10 pm
Location: Evesham, Worcestershire

Re: Transfer array between instances

Postby TheOm » Sun Nov 01, 2020 4:01 pm

trogluddite wrote:but unfortunately I ran into problems that were a bit beyond my C++ coding chops!

What problems did you encounter? I have experience with c++ and the windows api, so I might be able to help
TheOm
 
Posts: 103
Joined: Tue Jan 28, 2014 7:35 pm
Location: Germany

Re: Transfer array between instances

Postby trogluddite » Sun Nov 01, 2020 6:41 pm

TheOm wrote:
trogluddite wrote:but unfortunately I ran into problems that were a bit beyond my C++ coding chops!

What problems did you encounter? I have experience with c++ and the windows api, so I might be able to help

I think what stumped me most was concurrency. So maybe the problem was not so much C++ specifically, but lack of experience at working with multiple processes/threads accessing shared data. IIRC, the prototype I linked to above is not at all threadsafe - it assumes that execution of reader and writer cannot overlap, and even the simplest case of single-reader/single-writer could potentially suffer from data corruption. I know the basics of using mutexes when I need to sync threads spawned from the same process, but cross-process synchronisation (e.g. exe to plugin as Hugh suggested) is way beyond my experience.
All schematics/modules I post are free for all to use - but a credit is always polite!
Don't stagnate, mutate to create!
User avatar
trogluddite
 
Posts: 1706
Joined: Fri Oct 22, 2010 12:46 am
Location: Yorkshire, UK

Re: Transfer array between instances

Postby HughBanton » Sun Nov 01, 2020 7:50 pm

The more I think about it, it would be a fantastic FS asset if someone could pull this off!

At the moment It's mostly way over my head (never say never ...), but some of this MS reference already makes a bit of sense to me :

https://docs.microsoft.com/en-us/window ... and-memory

Hope it can be made to happen one day.

H
User avatar
HughBanton
 
Posts: 185
Joined: Sat Apr 12, 2008 3:10 pm
Location: Evesham, Worcestershire

Re: Transfer array between instances

Postby guyman » Sun Nov 01, 2020 9:04 pm

I almost put a bump on trog's original wireless tx/rx modules just a few days ago. +1 I'd love to see this happen.
User avatar
guyman
 
Posts: 187
Joined: Fri Mar 02, 2018 8:27 pm

Re: Transfer array between instances

Postby TheOm » Mon Nov 02, 2020 6:41 pm

I have reworked the code a bit and added some simple synchronization, just to guarantee that the whole shared memory region is updated atomically.
Attachments
Mapped File Source.zip
(277.65 KiB) Downloaded 181 times
TheOm
 
Posts: 103
Joined: Tue Jan 28, 2014 7:35 pm
Location: Germany

Re: Transfer array between instances

Postby trogluddite » Mon Nov 02, 2020 10:47 pm

TheOm wrote:I have reworked the code a bit and added some simple synchronization, just to guarantee that the whole shared memory region is updated atomically.

Many thanks - and so quickly too! :D
I have only had the chance for a quick glance so far, but the mutex handling doesn't look as frightening as I thought it might. I see now that I should have been using the Windows Mutex objects, where previously I was trying to figure it out using just std::mutex from the C++ libraries. Your example is a good starting point for some further research, too - so much appreciated!

PS) I also really like your version of the FlowStone header - it is such an improvement over DSPr's horrible mashup of compiler macros!
All schematics/modules I post are free for all to use - but a credit is always polite!
Don't stagnate, mutate to create!
User avatar
trogluddite
 
Posts: 1706
Joined: Fri Oct 22, 2010 12:46 am
Location: Yorkshire, UK

Next

Return to General

Who is online

Users browsing this forum: Google [Bot] and 23 guests