SAVE/LOAD multi-size arrays - must b better way

[RJHollins]

Thank-you TROG ... as usual, a lot of information to absorb.

I've assembled a modified schematic that centers around the Marshal routine you supplied.

I've kept your 'Save Load Application Data settings file' module as the reference model that handles the Directory Management.

I'm definately not comfortable playing RUBY when it comes to writing files or folders ... I've just not had any experience in those functions ... and won't want to tempt a possible disaster.

My lame-a** solution would probably look to a 'primitive' front-end. That would be a challenge enough

Posting the schematic ....


It would be nice if someday I could help someone else


**** EDIT ****

Updated schematic with 1st draft of Dir Management [Prims]

Save-Load Marshal Method [test] - 1.fsm

(30 KiB) Downloaded 955 times

Added Dir checking, FIND, Folder Create, confirmation LEDs, and routed the AUTO-LOAD after a SAVE.

This is still a 1st draft test of the idea.


oh ... wanted to comment on Trogs earlier post

Your logic seems sound enough - verify the contents of the save by re-loading; useful as you might otherwise not see it if the save fails for some reason. Windows 7/8 can be buggers for this as they'll sometimes not save a file as expected because of file permission, but without displaying any error!

With my earlier routine, I had wondered if 'file permissions' was a factor. At this point, I still think it was my shoddy programming

The reason I had gone the 're-load' route was actually to deal with my programs criteria. The User will have a special 'Patch list' file in XML format. I have them load this file into my VST, from which I parse, bend, fold, sort and organize into special groups. I wanted this to be a one time procedure. so I then have the app save off a configuration file [along with some other User settings]. Then, this new config file gets auto-loaded at run time. Most all of this config data is being routed around the schematic, filling up pull-down menus that now contain the patch names and PRGCHG numbers. These are used in the MIDI stream to control 'external' VST.

It needs to be a very dynamic file system, as I would never know the contents or amount of the Users XML. This Marshal routine provided the perfect solution, as the grouping and number of entries are easily maintain. This has greatly simplified the schematic. Secondly, the way You presented it allows even ME to customize it

The front-end piece is what I need to finalize. It obviously must work flawless. I'm testing the PRIM concept, and it does 'seem' to be working. But, as we are dealing with computers and various OS's ... 'seems to work' is not much of a confidence builder

Sure appreciate to check over the shoulder to correct where I might have gone wrong or overlooked!

Thanks !!!

[RJHollins]

Well ... have spent a good part of the nite troubleshooting via email.

I'm on WinXP ... bud is on Win 8.

I've completely gutted the old file save load routines and replaced them with the Marshal module.

On MY system ... it all works FLAWLESSLY. I have countless times alternated between both of our XML files used to generate the new configuration file that becomes the auto-load in.

Every time, it works on my system, both in FS AND as an exported VST. Not on my bud's system.

I had him send me a copy of the config file we are generating ... I replaced mine ... it doesn't load
I see an error message in the Marshal RUBY module saying:

AurgumentError:(in method'event'):dump format error (0xa)

So I then go back and Load in the copy of the XML file we are parsing ... Save off the Config version, Load in the Config file ... Works !

So ... I sent him a copy of the Config file that I generated here, for him to replace his copy and see if this loads in.

I don't know what is going on, or what else to do.

Not venting ... it just that the brain is total mush at this point.

What in the world could be going wrong to cause this


Note: I want to clarify ... I have a copy of his XML file that I'm using to generate the config.

either way ... I get NO errors on my system ...


** edit again ***

My tester just posted back to me before calling it a nite ...

He replaced his config file with the one I create ... no go

If I drank ... I would

[trogluddite]

What in the world could be going wrong to cause this

Hmm, compatibility between Windows version is going to be a must for any of us doing this kind of thing, so I've been doing some digging around - and this looks very similar to the problem that this Stack Overflow poster is having.

- The first part of the suggested solution can be implemented very easily - to force Ruby to treat the file as raw binary data. To do this, we just add the "b" flag to the File.open arguments...
File.open(filename, "rb")... for loading, and...
File.open(filename, "wb")... when saving.
So that's a very simple mod' to the code that we should try first - let me know if that helps, as I don't have easy access to a Win8 machine that I can try it on.

- The other part, where they mention "Encoding", is a little trickier.
There are various ways of turning a string character into binary numbers (e.g. ASCII, UTF-8, UTF-16 etc.) - and it's essential that the file is written and read using exactly the same encoding. IIRC, in some circumstances, Ruby reads the default encoding of the Operating System - and with the huge gap in time between XP and Win8, there's a fair chance that the default OS encoding might be different.
IIRC, it's also possible for the default read encoding and write encoding in Ruby to be set differently, as they don't share a single system setting. This might explain why your tester isn't even able to re-load files created on his own system.
I don't know this aspect of Ruby very well so far - but I'll do a bit more reading up and get back to you if I find any nuggets of advice on how to avoid encoding problems.

[RJHollins]

Thank you TROG, I'm following this closely.

I went back into the Ruby code and modified the parameters in the 'load' & 'save' to:

Code: Select all

File.open( @filename, "wb" ){ |file|  Marshal.dump( container, file) }

container = File.open( @filename, "rb"){ |file| Marshal.load(file) }

Without first re-saving the config file, I was able to load it in. Looked fine.
I did a new Save, and loaded that ... again, fine.

My 'tester' is on a remote session today, and will contact me later today. I'll report back after he's tested on his Win 8 system.

In order to simplify testing, I've saved my project as an EXE app. Since we are de-bugging this config file, why go through a host and VST. Even in the EXE mode, all continues to work as expected on my system.

... in holding pattern ....


oh ... just to be sure ... here is my Marshal Ruby module:

Code: Select all

# Assuming you have ruby inputs 'array1', 'array2', 'array3' for your arrays.
# + a string input 'filename' for the path
# + two trigger inputs 'save', and 'load'

def init
   @status = false
end

def save_arrays
  container = [ @array1, @array2, @array3, @array4,
   @array5, @array6, @array7, @array8,
    @array9, @array10, @array11, @array12,
     @array13, @array14, @array15, @array16, @array17 ]
  File.open( @filename, "wb" ){ |file|  Marshal.dump( container, file) }
end

def load_arrays
  container = File.open( @filename, "rb"){ |file| Marshal.load(file) }
  array1, array2, array3, array4, array5, array6, array7, array8,
  array9, array10, array11, array12, array13, array14, array15, array16,
  array17 = container
  output 0, array1
  output 1, array2
  output 2, array3
  output 3, array4
  output 4, array5
  output 5, array6
  output 6, array7
  output 7, array8
  output 8, array9
  output 9, array10
  output 10, array11
  output 11, array12
  output 12, array13
  output 13, array14
  output 14, array15
  output 15, array16
  output 16, array17
end

def event(in_id)
  case in_id
  when 'load' then load_arrays
  when 'save' then save_arrays
  output 17,  @status = true
  output 17, @status = false
  end
end

does this look OK ?

[RJHollins]

alrighty then ... had an opportunity to test the stand alone version on a Win 7 64 bit OS.

The app loaded, and I was able to load the XML file. It parsed as always.

Went to SAVE the config file in MyDocs ... did NOT see that written, LOADING the same config file [calling the same location] did NOT work ... looks like file not found.

Let me say, I know nothing of Win7 layout. Just finding the MyDocs folder was still guessing if that was indeed it

I saw no new files written [put into a self creating folder] ... So we have a fundamental issue here.

Now, on my testers system, he was able to send me the saved config file, [from Win 8], so either I'm not seeing it on 7, or 7 is different than 8. BTW ... never saw any error message from 7.

Things work perfectly on my XP system.

Apologies for my chaotic report ... just trying to get as much info out there ... and trying to figure out what the heck is wrong.

Well ... at least I got to see my app on a whole different system


*** EDIT ***

Hitting Google and searching Win7 'Documents and Settings' folder ....

Am I trying to use a bogus location ?

In my schematic ... the SYSFOLDERs prim is using ID 5 [according to FS discussion]. When I did a hard rive search on the Win7 unit I got to test ... the Folder and file that I was attempting to write is no where to be found.

I thought Doc-Setting "My Documents' would be a plain, easy, generic location to store this config file. Maybe this is NOT so ?!?

If not ... where ??? It no matter to me ... as long as I can write and read it on any of these OS's.

Sorry to bother everyone here, but I'm totally stuck and cannot easily test out any 'experiments'.

Honestly, I wouldn't mind storing this config file in the VST location ... but thought that IT might be protected. I just don't know what is wrong. If my code was wrong, I'd think I'd see that on my system But as of now, anyone that wamts to use my app has to come over to my house ... this is just not practical at the moment


alright ... maybe if we do it in an orderly manner, and you bring some good tunes to listen to.

[RJHollins]

OK ... I know its late ... and I'm a bit fried ... but we got some good news back from my tester !!

Although it still feels like a bit of a dream it appears that the modification to 'Raw Binary' within the Marshal routine has been SUCCESSFUL !


He tested it with both of our custom XML files ... both parsed out ... config file saved and loaded ... populated all the main controls as designed.

We will of course run the test again tomarrow ... but he sent the files ... and confirmed that he is seeing something we have not before ... his patches, and then mine !

He even has the file showing up in MyDocs location ... something that I didn't have when testing on the Win7 system.
I have to see what's up with that.

But hey ... some positive news. TROG [the Life Saver] once again ... and again .... and ...



If I drank ... I'd be drinking right now

[RJHollins]

... and he's back ...

GOOD NEWS ... confirm that the 'Raw Binary' setting in the Marshal routine works on WinXP-32, Win 7 - 64, and Win 8.1 -64 !

other stuff ... without incidence [so far], I have switched storing location of the config file to now reside with the VST location. [at least this is working with the test EXE model] ... soon to test the VST. This would be preferred to help keep some organization and tidiness for my apps. To facilitate this, I'm using the FOLDER primitive.

Now that we have success with the file write/read, it is time to go back and fix all the stuff I had to break during the troubleshoot. Basically, after the User manually locates to his special XML file ... once loaded and parsed ... I want the Save and Re-load to be automatic [behind the scenes]. When the Plugin launches, the config is Auto-loaded.

What an adventure ... Big THANKS to TROG !!! for sooo much I can't even list

And NOT to forget the excellent help from the Group here [and SM].

You Guys are making me look good ... and nothing easy about that

Back to work here!

[RJHollins]

for those following the tearful drama in my WinXP - Win7/8 thread ... I wanted to update this thread...
http://www.dsprobotics.com/support/viewtopic.php?f=2&t=1741

For reasons beyond my comprehension, the glimmer of hope that we had success writing and reading a simple file [whether using plain-jane text, or the super-wonderful Marshal routine in Raw Binary mode has failed to work in the VST format that all this resides in.

We have test going on in the other thread as we try to narrow in on a 'safe-haven' to store on.

Complicating things is that everything works on my WinXP -32 machine. My troubleshooting has been like a Hide&Seek game. I don't know what to rule in or out. My tester is an experience beta-man, with 2 machines [Win7 & Win8.1] both have failed the special test module that Drnkhbo has posted.

The initial success came using an EXE version of my plugin. The real VST version [identical code, different format] has failed every attempt. Looking at the config file that he was able to save does not look like the one I save on my system using his same file.

When I try to load his config copy, I get an error in the RUBY box like before.

I only have an XP machine to write on and test from. My tester says he's running as Admin, and has not had a permission issue with anything he has installed.

Bummed beyond belief ...