I think I have found a bug in MSVC 6's handling of floating-point code.
To demonstrate the problem, create a new workspace in MSVC 6 which is a Win32 console app and use it to compile the following code (as either C or C++) without tweaking any of the project-settings. When it asks you to enter three integers, enter 255 three times.
| 1 | /* Program to demonstrate some anomalous behavouor of floating point calculations in MSVC */ |
| 2 | |
| 3 | #include <stdio.h> |
| 4 | |
| 5 | // #include <math.h> /* It makes no difference if <math.h> is included or not */ |
| 6 | |
| 7 | |
| 8 | #define MIN(x,y) (((x) < (y)) ? (x) : (y)) |
| 9 | #define MAX(x,y) (((x) > (y)) ? (x) : (y)) |
| 10 | |
| 11 | |
| 12 | |
| 13 | int main(void) |
| 14 | { |
| 15 | int ia, ib, ic; |
| 16 | float fa, fb, fc; |
| 17 | float fx, fy; |
| 18 | float max, min, delta; |
| 19 | |
| 20 | |
| 21 | printf("Enter three integers (to demonstrate the problem, enter 255 three times)\n"); |
| 22 | scanf("%i %i %i", &ia, &ib, &ic); |
| 23 | |
| 24 | |
| 25 | fa = (float)ia / 255.0f; |
| 26 | fb = (float)ib / 255.0f; |
| 27 | fc = (float)ic / 255.0f; |
| 28 | |
| 29 | max = MAX(fa, MAX(fb, fc)); |
| 30 | min = MIN(fa, MIN(fb, fc)); |
| 31 | |
| 32 | |
| 33 | fx = max+min; |
| 34 | |
| 35 | delta = max - min; |
| 36 | |
| 37 | if(delta == 0.0f) |
| 38 | fy = 0.0f; |
| 39 | else |
| 40 | fy = delta/(2.0f-fx); |
| 41 | |
| 42 | |
| 43 | printf("\nfx == %f, fy == %f , fc == %f\n\n",fx, fy, fc); /* For some strange reason, the problem doesn't show up if fc is ommitted */ |
| 44 | printf("delta == %f\n\n", delta); |
| 45 | // printf("max, min == %f, %f\n\n", max, min); /* For some strange reason, if this line is un-commented, the problem doesn't show up */ |
| 46 | |
| 47 | |
| 48 | return 0; |
| 49 | } |
I am using MSVC 6 with service-pack 5 applied under Windows 2000. My machine is a 600MHZ AMD Athlon.
I also tried the code under Cygwin's GCC ( gcc version 3.2 20020927 (prerelease) ) and I've not been able to re-produce the problem. I am using the following command to compile:gcc -g testmsvcfloatbug.c -O3 -ffast-math -fomit-frame-pointer -o testmsvcfloatbug_gcc.exe. I have tried various combinations of this command involving replacing -O3 with -O2, using the -fno-strength-reduce flag, using -march=pentium2 and removing the -ffast-math flag. Despite this, the GCC version always outputs the correct results.
Anyway, here is the correct output which is what I would expect it to be.
Enter three integers (to demonstrate the problem, enter 255 three times) 255 255 255 fx == 2.000000, fy == 0.000000 , fc == 1.000000 delta == 0.000000
Here is what happens under MSVC
Enter three integers (to demonstrate the problem, enter 255 three times) 255 255 255 fx == 2.000000, fy == 1.#INF00 , fc == 1.000000 delta == 0.000000
The problem in MSVC appears regardless of whether the code is compiled as C or C++. However, in the debug-build, it behaves correctly. Also, if I replace the floats with doubles, it behaves correctly as well.
The code contains a line that compares a floating point variable 'delta' to 0.0f using '==' without any form of epsilon. Seeing that 255 was entered 3 times, then 'delta' whould be the result of subtracting two equal numbers which is 1.0f/255.0f - 1.0f/255.0f which should be 0.0f, right?
If I change the compare from "if(delta == 0.0f)" to "if(delta < 1.0f/1024.0f)", the problem disappears. One theory is that the FPU uses a system of odd-even rounding. That is, one "1.0f/255.0f" is rounded down to fit a float and the other is rounded up. The result of this is a very small difference that appears in 'delta' when the maximum and minimum are subtracted, which makes it past the "==0.0f" test and causes the divide to produce such a large number that it's as good as infinity. However, at the end, it always prints delta out to be "0.000000".
Also, by messing around with the number of floats used (eg. try un-commenting the last printf), the problem disappears as well. This would imply that there's a bug somewhere in MSVC's code for handling floats.
But nethertheless, the fact remains that GCC and MSVC (release-build) produce floating-point code with different behaviour which is demonstrated by the example program in this post.
Is this a know issure in MSVC 6? What other versions of MSVC exhibit this behavoiur?
AE.
Hmm, that's a weird bug. You could try looking at the actual fp assembly output. Microsoft probably would want to know about, if it isn't fixed in VC7 yet.
On the topic of weird bugs, here's a cool GCC bug I've tried up to GCC 3.2.2 (I've yet to compile 3.3). Any fool can crash their own program but it takes a lot of work to crash the compiler.
| 1 | template <class T> |
| 2 | class A { |
| 3 | class B; |
| 4 | }; |
| 5 | |
| 6 | template <class T> |
| 7 | class A<T> :: B { |
| 8 | B(T t); |
| 9 | }; |
| 10 | |
| 11 | template <class T> |
| 12 | void // error |
| 13 | A<T> :: B :: |
| 14 | B (T t) { |
| 15 | } |
| 16 | |
| 17 | template A; |
This may sound obscure but it actually had me scratching my head and trying to figure out what I had done wrong, as the compiler kept crashing instead of giving an error message.
Did you try it with %g or %E instead of %f?
Obviously it is some problem with the stack, but I'd like to think it's a problem with the format specifier and the size it's pulling from the stack.
Same results in MSVC 7.
I've noticed that ever since I've gone to MSVC7, I haven't been able to properly code any complicated project with floats in release mode.
In one project I had a vector class and I was spewing particles out in random directions that worked perfectly in release mode. For some reason in release mode the particles would only spew out with their x, y, and z values equal to 1, 0, or -1, rather than lots of interesting values inbetween, so my particles spewing out in all directions turned from a uniformish distribution to only comming out in an exact line in a few directions. It was if it was rounding all of the numbers as if I casted to int.
In my second project I wrote a skeletal animation system. I do all of my timing and quaternion/vector interpolations using seconds and using floats. In release mode with some models it misses some keyframes and doesn't find them (I play part of the animations by looking for keyframes within a certain time). With some other things it does weird random stuff. When I turned on "improve floating point consistency" the problems went away. I tried to debug in release mode but it was just way way too hard to do that, but it appears that a lot of key numbers were wrong or just off, sometimes by like 50% or more from value at the same time in the debug app, which agrees with values that I know are correct.
This make no sense that the option would help. The option forces the compiler to write the variable to memory (truncating it from 80-bit to 32-bit precision) inbetween every calculation rather than leaving the number in the register. To me that seems to mean that the option would REDUCE accuracy. It didn't seem to lower my framerate though so I just leave the option turned on but all of these events make me wonder if I'm coding something wrong.
I've noticed that ever since I've gone to MSVC7, I haven't been able to properly code any complicated project with floats in release mode.
What were you using before MSVC 7?
Have you considered installing MinGW32 (or if you're feeling masochistic, CygWin) so you can build working release-builds of your projects? That way, you can use all of MSVC's IDE's wonderful debugging-aids, and when it comes to building the release build, you can use GCC under MiGW32. I've noticed GCC 3.2's code is significantly faster then MSVC 6's code. I don't have code-speed comparisons for MSVC 7.
AE.
MSVC7 is much faster than 6. I used to use DJGPP, then I used MSVC6, and now I use MSVC.NET. I know that MSVC6 optimized a lot better than GCC 2.9x and that MSVC.NET is a much better optimizer than MSVC6. Most particularly that I like about MSVC.NET is that it can inline functions across object file boundaries, which is good because I can write clearer because I'm not required to write code in headers for optimal performance.
I do know that GCC 3 has much better optimization. I don't know how GCC 3.2+ compares to MSVC.NET and I don't know if GCC 3.2+ supports cross-objectfile optimizations/inlining.
As for mingw and whatnot, yes I do have it, and I use GCC 3.1 to compile in Linux. But I didn't use Allegro on these last 2 projects, and I had to write them using DirectX8.1 and Win32 applications, so I had no real reason to try to set up mingw32 when MSVC.NET works perfectly well.
First, I don't think you have an real floating-point error. Or, at least, it isn't something you should ever rely on out of any compiler/hardware. Try this:
Change your code to check to see if ((float)ia == 255.0f). Even if you type in the integer 255 and do the typecast, I don't think it is required for these to be equivalent.
Also, try using 256 instead of 255. As it can be exactly represented as a float, it may work.
Even if you type in the integer 255 and do the typecast, I don't think it is required for these to be equivalent.
The only time casting an int to a float causes loss of data is when the number of significant binary-digits (bits) in the int is greater than the mumber of bits used to represent the mantissa in the float (23, but thanks to the way floats work, it's effectively 24). This means that any int with 24 or less significant bits. can be safely converted to a float without loss of data. 255 only requires 8-bits to be represented (or 9 if it's a signed value), so it can be converted to a float without any loss of accuracy. However, 1/255 cannot be accurately converted into a float.
Also, try using 256 instead of 255. As it can be exactly represented as a float, it may work.
The whole idea of the program is that it demonstrates a bug in the assembler optimisations of the compiler. By examining the sourcecode that I posted earlier, both the GCC and the MSVC versions use exactly the same constant to represent 1/255.
As for dividing by 256 instead of 255, the reason I'm using 255 is because I want the value to be divided by 255 - not 256. I'm trying to convert a value from 0-255 to a value from 0-1, so dividing by 256 won't give me the result I want.
So far, I've not been able to re-produce the problem on GCC (although I've nowhere near exhausted it's plethora of optimisation-switches), but on MSVC, I can re-produce it on demand, which means there's a bug in the compiler and not in the CPU.
Same results in MSVC 7.
Just out of interest, did the machine you tried this on have an Intel or AMD CPU? Was it MSVC 7 (NET 2002) or MSVC 7.1 (NET 2003)?
AE.
AMD XP, and I believe it is 2002 edition of MSVC 7.
So far, I've not been able to re-produce the problem on GCC (although I've nowhere near exhausted it's plethora of optimisation-switches), but on MSVC, I can re-produce it on demand, which means there's a bug in the compiler and not in the CPU.
I still fail to see how this is a bug.
BTW, what is "delta" in the cases where it produces the "wrong" results? Is it +0.0f, or -0.0f (or something else)?
Andrei Ellman: Here's my guess as to what's happening: You calculate your three identical values, fa fb and fc. Then you compare them to find the biggest and smallest. However, while you're doing this, in optimizing mode, it's trying to keep those values in registers, it can ends up writting out one or two to memory instead. While they remain in registers, they have long double precision, because that's the precision of the FP registers, but because they're declared as floats, the ones that get written to memory are truncated to single precision. So when the comparisons are done, it finds that the full precision one is slightly different from the single-precision one, so max and min end up having a tiny difference.
A look at the asm seems to support this theory - notice how in the un-optimized version there are three fstp's for the three divisions, but in the optimized version there are only two - the third value is kept in a register.
Anyway, it's obnoxious behavior, but it's not exactly a bug. Try checking the "improve float consistency" checkbox under project settings -> C/C++ -> optimizations.
I've done yet more research. I tried checking all of MSVC's optimisation checkboxes that I expect it to use when using "Maximize Speed" (why oh why oh why does MSVC have to be awkward and hide which optimisation-checkboxes are checked when chosing one of the pre-defined optimisation-profiles?). Now, I was able to get the problem to appear depending on whether or not the "Improve Float Consistency" checkbox was checked (if it was checked, the code behaved as it should). So it appears Orz's theory to what's going on is what's likely to be causing the problem. I also changed the printf format specifier for floats from %f to %e and as a result, was able to see that there was in fact a very small value in delta that was too insignificant to notice if I printf'd floats with %f. When fy was 1.#INF00e+000, delta was always 5.913898e-008 and when the code was behaving as it should, both values were 0.000000e+000 . I also tried changing all 255s to 256s and the problem disappeared.
Meanwhile in GCC, I discovered the -ffloat-store flag and wondered if I could reproduce the problem by switching it on and off. I tried the four combinations of the flags -ffloat-store and -ffast-math (whaile using -O2) and still couldn't reproduce the problem in GCC. Perhaps GCC and MSVC initialise the state-word of the FPU to a slightly different value.
So the moral of this story is: Don't ever use '==' or '!=' with floats, even when it looks obvious that they will be equal. Instead, test to see if a range of values within the limits of an error-value (epsilon) overlaps with the other number's error-range. See this thread for suggestions on how to implement such comparisons. When using GCC, compile with the -Wfloat-equal warning-flag to get warnings when using == and != with floats. This is one of the flags not turned on by default when using -W and -Wall so you must always explicitly pass -Wfloat-equal .
AE.