Subject: Floating point subtraction rounding error (NOT display error)
From: aahz@pythoncraft.com (Aahz)
Date: 12/13/2007 2:39:19 PM
In article <93a9ea9a-00ab-4022-a567-8ba268790681@s12g2000prg.googlegroups.com>,
Keflavich <keflavich@gmail.com> wrote:
>
>Hey, I have a bit of code that died on a domain error when doing an
>arcsin, and apparently it's because floating point subtraction is
>having problems. I know about the impossibility of storing floating
>point numbers precisely, but I was under the impression that the
>standard used for that last digit would prevent subtraction errors
>from compounding.
>
>Is there a simple solution to this problem, or do I need to run some
>sort of check at every subtraction to make sure that my float does not
>deviate? I'm not sure I know even how to do that.
Switch to Decimal module? Available in 2.4 and later.
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/
"Typing is cheap. Thinking is expensive." --Roy Smith
|