Re: Float scientific notation
Posted: Sun Jun 18, 2017 2:35 pm
martinvicanek wrote:You can do this but frankly I would recommend Ruby. The Format String module is very picky about the format input: FS will crash on invalid specifier strings!
Thanks martinvicanek for the DSP version. I did make one with the Ruby too. They both behave the same way when hitting the 7 digit limit. It looks like when you released your DSP math pack, I will quote what you said: "while accuracy is close to the limit imposed by single precision (7 significant digits)." So I stumbled upon a limiting factor with 7 digits. I was trying to work on a virtual capacitor and most of the time with low voltage DC circuits you work in uF. 1 Farad (F) is equal 1000000 Microfarads (µF) (0.000001). This number is automatically displayed as 1e-006. For now with it being 6 digits the conversion will work, but I am still surprised that there is not a bigger float or integer DSP module to handle the larger numbers.
Regards,
QuadBIT