Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » [MinGW] Stack Size

Credits go to Audric, bamccaig, GullRaDriel, ImLeftFooted, james_lohr, Oscar Giner, Tobias Dammers, and torhu for helping out!
This thread is locked; no one can reply to it. rss feed Print
[MinGW] Stack Size
Stas B.
Member #9,615
March 2008

Hi, guys... I'm writing a chess AI and my program is crashing after a certain function call, without ever executing any of its instructions, apparently. This observation leads me to conclude that it simply runs out of stack space.
Does anyone happen to know what the default stack size of programs compiled with MinGW is? Also, how do I increase the stack size to make my program stop crashing?

Thanks in advance.

bamccaig
Member #7,536
July 2006
avatar

Oscar Giner
Member #2,207
April 2002
avatar

pass --stack,size_in_bytes to the linker.

ld --stack,2097152
or
gcc -Wl,--stack,2097152

to set it to 2 MiB.

Note: this is Windows only. Linux executables don't have a stack size setting, it's an OS config setting.

bamccaig
Member #7,536
July 2006
avatar

It seems like the more correct solution is to refactor your program to not exceed its given stack, but I can imagine it would be nice to be sure before attempting to do that... :-/ Are you using recursion?

Tobias Dammers
Member #2,604
August 2002
avatar

I was going to say that a stack overflow under default settings probably means "ur doing it wrong", but with a chess program, I can imagine how you might end up with deeply nested function calls like that. Still, increasing the stack size feels a bit like solving the inefficient software problem by throwing more hardware at it.

It may be worth rewriting a few recursive algorithms iteratively; it's less elegant from a mathematical point of view, but computers (and compilers) like iterative solutions better than recursive ones most of the time. Example:

#SelectExpand
1// Recursive: 2// Needs to allocate n stack frames before returning a value. 3// For large values of n, this is very bad. 4int fibo(int n) { 5 if (n <= 1) 6 return 1; 7 return fibo(n-1) + fibo(n-2); 8} 9 10// Iterative: 11// Never allocates more than one stack frame plus 4 integers. 12int fibo(int n) { 13 if (n <= 1) 14 return 1; 15 16 int a = 1; 17 int b = 1; 18 int c = 2; 19 20 for (int i = 2; i < n; ++i) { 21 a = b; 22 b = c; 23 c = a + b; 24 } 25 return c; 26}

Try to write your recursive functions in a tail-recursion-optimizable way, that is, the recursive call should be the last statement in the function, or the argument to a return statement. This way, the compiler can often optimize the recursive call by popping the stack frame before doing the call.

If you need to pass around large amounts of data, consider heap allocation. It's a bit more work, and seems a bit less efficient (use a memory pool algorithm if you have to), but heap space is virtually unlimited.

And finally, there are some optimization settings you may want to fiddle with that are related to stack usage. Dive into the gcc documentation and see what it can do for you.

---
Me make music: Triofobie
---
"We need Tobias and his awesome trombone, too." - Johan Halmén

Stas B.
Member #9,615
March 2008

Thank you for your suggestions, guys.

For some reason, I just can't get it to work. Neither the linker commands suggested by Oscar nor setting the stack size using the EDITBIN utility that comes with VC++ did any good. It kept crashing while analyzing certain board positions, just like before. At first, I thought it may not be stack overflow after all. I tried reducing the stack size to a ridiculously low amount. Miraculously, it kept working for other board positions just fine. I can only conclude that the stack size remained the same. After running EDITBIN on my executable, I could see that it DID modify the file. It just didn't have any effect on the stack. ???

For now, I guess I will handle this situation by implementing a more efficient algorithm and by culling more game-tree nodes. However, it's only a question of time before I hit the stack limit again. I'm only searching to a depth of 5 plies, for now. Even if I can cull enough nodes to stop it from crashing, I will have the same problem again if I ever try to search to a greater depth. It's not really about the efficiency of the algorithm. I simply run out of stack space much faster than I run out of CPU power\RAM. :-/

@Tobias Dammers:
While possible in theory, I doubt that it's plausible to rewrite a minimax-type algorithm into something iterative... (But I might be wrong, of course!)
I can't think of any way to rewrite it in a "tail-recursion-optimizable way", either. If you know how these things can be done, I'll be glad to learn.
In any case, when you talk about heap allocation, you mean I should try mallocing things inside my function instead of using local variables? This should reduce stack usage considerably, but how do I do it without killing too much speed? :o

[EDIT]
Ohhh... I think I get it now. I'll pre-allocate a big chunk of memory and use it as a stack for local variables. This should help, since my local variables take about 130 bytes of stack space for each call. :P

Audric
Member #907
January 2001

I was about to propose to make some local variables static (checking that they don't surround a recursive call), but your last suggestion is the cleanest: malloc a structure for all the local variables, free it when you exit the function. The only local data will be a pointer.

Remember that function arguments are on the stack too, so you can move them in a malloc'ed structure as well.

ImLeftFooted
Member #3,935
October 2003
avatar

It's also possible you have a bug that is causing infinite recursion.

Stas B.
Member #9,615
March 2008

Errmmm... No. If that were the case, I would have found and fixed it, instead of trying to modify the stack size. How stupid do you think I am? :P
I'll try to allocate my own stack and see if it solves the problem. When I won't feel THIS lazy, that is.

torhu
Member #2,727
September 2002
avatar

I guess you know that you can use dumpbin /headers to see what stack size has been set. Or the free sysinternals vmmap tool to see the stack size in a running process.

But why not just build it with msvc and run it in the debugger? You'll probably get a message about stack overflow if that's what really happens.

ImLeftFooted
Member #3,935
October 2003
avatar

Stas B. said:

How stupid do you think I am?

No idea. How stupid are you?

Stas B.
Member #9,615
March 2008

Fairly stupid, yet smart enough to make sure that I'm not starting an infinite recursion. :)

GullRaDriel
Member #3,861
September 2003
avatar

"Prove it !" (c) Dustin Dettmer ;D

"Code is like shit - it only smells if it is not yours"
Allegro Wiki, full of examples and articles !!

james_lohr
Member #1,947
February 2002

Stas B. said:

This observation leads me to conclude that it simply runs out of stack space.

That's a ridiculous conclusion. You've quite blatantly got some sort of pointer error. Always assume that it is you doing something wrong long before you start blaming the compiler.

Stas B.
Member #9,615
March 2008

James Lohr, you're right. ;D
It turns out the stack has nothing to do with it.
I have absolutely no idea why it crashes, though.
It crashes during a function call, before actually executing any of its instructions. (Before executing any of the instructions I have explicitly written, in any case.) Even if I pass it a bad pointer, how can it have any effect before I actually try to do something with it? ?????????

[EDIT]

Basically, it's like this:

#SelectExpand
1 int recursive_call(...) 2 { 3 ... 4 DEBUG1 = 123; 5 foo(...); 6 DEBUG1 = 321; 7 ... 8 } 9 10 void foo(...) 11 { 12 DEBUG2 = 456; 13 ... 14 DEBUG2 = 654; 15 return 16 }

If I check the values of DEBUG1 and DEBUG2 after I get a crash message but before I terminate the program, I get DEBUG1 = 123, DEBUG2 = 654.
Either it crashed exactly after "DEBUG2 = 654", or the value stayed after the last successful call and the first line of "foo" was not executed during the next call.

Audric
Member #907
January 2001

What does GDB say about the size of the backtrace, the place where it crashes, and the actual error (sigsegv, or whatever)

Stas B.
Member #9,615
March 2008

The only debugger that I can use is the one that comes with Visual Studio... Which I don't have access to, right now. Anyway, I have determined where exactly it crashes (see above) and that it's not a stack overflow. (I'm using my own stack implementation now. It pops a message box in case of stack overflow.) I don't get any useful error message. Just a generic "Chess.exe has encountered a problem and needs to be closed... blah blah blah..." :-/

bamccaig
Member #7,536
July 2006
avatar

Stas B.
Member #9,615
March 2008

I'm an idiot. :-[
James Lohr was right. It was a pointer error. I found the real line that caused the crash after turning off compiler optimizations. I guess I really should get a normal debugger and learn how to use it. Oh, well. Thanks, everybody. ;D

Oscar Giner
Member #2,207
April 2002
avatar

[edit]
oh, ok you solved it :P
[/edit]

Considering that changing the stack size doesn't solve anything (have you tried to set it to a big number, something like 16MB or maybe more?), and that local variables take only 130 bytes, if it's really a stack overflow exception, then you certainly have an infinite recursion bug. The default stack size is certainly more than 1MB, so it'd take more than 8000 recursive calls to produce a stack overflow.

It's always posible to convert a recursive algorithm into an iterative one. It's just that's only trivial to do when there's only one recursive call, and this one is the last in the function. If not, there are methods for doing it, but they always include having a stack structure and usually not worth to implement.

Tobias Dammers
Member #2,604
August 2002
avatar

Stas B. said:

Fairly stupid, yet smart enough to make sure that I'm not starting an infinite recursion.

It doesn't take much stupidity to cause infinite recursions by accident. In fact, it is ridiculously easy. Also:

Stas B. said:

I'm an idiot. :-[
James Lohr was right. It was a pointer error. I found the real line that caused the crash after turning off compiler optimizations.

It doesn't take a lot of brains to figure out that compiler optimizations may reorder statements, omit unnecessary parts of your code, and otherwise break the nice 1:1 mapping of source lines to execution steps, making debugging an exercise in futility. This is exactly why most people have a setting somewhere in their build script (or IDE settings or whatever you fancy) that says either "debug" or "release".

---
Me make music: Triofobie
---
"We need Tobias and his awesome trombone, too." - Johan Halmén

Stas B.
Member #9,615
March 2008

Well, I figured it out... eventually... after checking an option that seemed more likely to me, given the particular task I was doing. It's not like I don't realize that code can't be optimized and remain unchanged. The optimizer is off by default and I can't recall turning it on. :-[

Audric
Member #907
January 2001

Generally, if I only want to look around (count how many times I pass in a function, or peek at a variable), I won't bother recompile everything with -O0. But if I need to step-by-step, for example, I know that the sequence of source code lines is going to seem very illogical, unrelated to the expected sequence of execution. I then know I need to stop wasting time and just do a make clean + make not_optimized.

The first introduction to GDB is very easy, when you have a program that crashes on a reproduceable situation :
(Re)compile with -O0 and -g
"gdb <yourprogram>"
"run"
Use your program until it crashes. Instead of a bland generic message (or silent shutdown when using SDL) you return to interactive GDB, it shows the precise error reason and the line of code.
Type "bt" to get the complete backtrace, it's often very instructive.
If you need the value of a variable, type "print <variable>".
"quit" to exit.

Go to: