Support

If you have a problem or need to report a bug please email : support@dsprobotics.com

There are 3 sections to this support area:

DOWNLOADS: access to product manuals, support files and drivers

HELP & INFORMATION: tutorials and example files for learning or finding pre-made modules for your projects

USER FORUMS: meet with other users and exchange ideas, you can also get help and assistance here

Float scientific notation

DSP related issues, mathematics, processing and techniques

Re: Float scientific notation

Postby pshannon » Sun Jun 18, 2017 2:35 pm

martinvicanek wrote:You can do this but frankly I would recommend Ruby. The Format String module is very picky about the format input: FS will crash on invalid specifier strings! :o :roll: :x


Thanks martinvicanek for the DSP version. I did make one with the Ruby too. They both behave the same way when hitting the 7 digit limit. It looks like when you released your DSP math pack, I will quote what you said: "while accuracy is close to the limit imposed by single precision (7 significant digits)." So I stumbled upon a limiting factor with 7 digits. I was trying to work on a virtual capacitor and most of the time with low voltage DC circuits you work in uF. 1 Farad (F) is equal 1000000 Microfarads (µF) (0.000001). This number is automatically displayed as 1e-006. For now with it being 6 digits the conversion will work, but I am still surprised that there is not a bigger float or integer DSP module to handle the larger numbers.

Regards,

QuadBIT
User avatar
pshannon
 
Posts: 34
Joined: Fri Jan 02, 2015 3:08 am

Previous

Return to DSP

Who is online

Users browsing this forum: No registered users and 3 guests