Welcome Guest. Please Login or Register. Apr 5th, 2018, 9:53pm
ATTENTION MEMBERS: Conforums will be closing it doors and discontinuing its service on April 15, 2018. Ad-Free has been deactivated. Outstanding Ad-Free credits will be reimbursed to respective payment methods.
If you require a dump of the post on your message board, please come to the support board and request it.
Subtraction
« Thread started on: Feb 16th, 2017, 4:53pm »
Hello,
Can somebody please tell me why the following IF statement does not display the text "Hello!"? I guess it's something to do with the numbers not being integers, but I'm not sure...
Code:
IF 0.3 - 0.2 = 0.1 THEN
PRINT "Hello!"
ENDIF
Thanks, Josh
Logged
Zaphod Guest
Re: Subtraction
« Reply #1 on: Feb 16th, 2017, 5:05pm »
Yes, the answer is that computers use floating point calculations so the result is not actually 0.1 but probably something like 0.0999999999998 and therefore fails the test. Try PRINT 0.3-0.2-0.1 It should be zero but isn't exactly.
Integer arithmetic works, decimal does not exactly but may differ in the last digits that you can't see most of the time. So you either need to bump up the numbers to be integers or look for the result to be between defined bands that allow for the error.
You can use something like ABS( result -target)< precision needed. IF ABS(X-0.1)< 0.001
Re: Subtraction
« Reply #2 on: Feb 16th, 2017, 7:39pm »
Thanks Zaphod; it makes sense to me now.
Something like this seems to fix the problem:
Code:
IF FNroundto1dp(0.3 - 0.2) = 0.1 THEN
PRINT "Hello!"
ENDIF
END
DEF FNroundto1dp(num)
num% = num*10
= num%/10
Logged
Zaphod Guest
Re: Subtraction
« Reply #3 on: Feb 16th, 2017, 8:53pm »
Now be very careful as when you assign a floating point number to an integer variable then it truncates the decimal part.
If the rounding error in the floating point variable is just over the next integer value it will give a different result to being just under. So your solution may sometimes be off by 0.1
Logged
