I'm having some difficulties debugging a crash in my ManyMouse project. For some reason it is crashing in al_set_target_bitmap. old_shader is usually 0xff0000ff or some other trash value, which would seem to indicate I'm dealing with a display (or at least a shader) that has already been destroyed.
gdb doesn't work very well on Windows 10 (it hangs when the program crashes) so I'll debug it more when I get home and I can use my Vista machine.
0x6773616e in al_set_target_bitmap (bitmap=0x2a5fa78) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 443 old_shader->vt->unuse_shader(old_shader, old_display);
If gdb does not work well, maybe it's because you made a total mess of the stack ^^
I would happily give it a go on linux. Wait: where is the code to debug ?
I removed a couple lines where I used al_hold_bitmap_drawing(false), and now once in a while I get a crash where an assertion fails in tls.c on line 403 :
What does that mean? I never hold bitmap drawing, so I don't know why it would be true.
I originally added those lines in because of this same crash, and it seemed to work.
@Gully
It's a windows only project at the moment, because it uses the RawInput API from the win32 code library.
The font addon will change the target bitmap and also hold bitmap drawing (for example here [1])... but as long as you only call font functions from the thread which does drawing operations there should be no problem. If you have several threads make sure only one can do font functions (this includes things like measuring the width of a string).
I draw each window in its own window callback registered with allegro, so each window's drawing operation is running in its own thread, at least for my mouse 'windows'. It's because I'm using a DIB to draw the window transparently, something which I haven't figured out how to do with allegro yet. A mouse cursor can't just be a solid rectangle.
Hm, make sure each window has its own, separate set of bitmaps and fonts in that case. In theory Allegro can support shared contexts but there's probably a few places (like in the font addon) which can't deal with it.
Okay, I fixed one crash where I was calling al_set_target_backbuffer(NULL);. I mixed that up with al_set_target_bitmap(NULL).
Edit
-------------------
Shouldn't al_set_target_backbuffer(NULL) fail an assert or something? Or al_get_backbuffer(NULL)?
-------------------
But I'm still getting the original crash in al_set_target_bitmap. It is crashing on line 443 of tls.c :
old_shader->vt->unuse_shader(old_shader, old_display);
One time old_shader was 0xabababab, one time it was 0x1, and now I can't reproduce it at the moment. I think there is some uninitialized memory on the loose.
Okay, here's a log :
Here's another log (it shows the bitmap is valid I believe)
Are you using al_use_shader anywhere? Are you ever destroying any shaders?
No, not directly anyway.
Edit
-----------------
Elias - I think that _al_d3d_destroy_bitmap is not resetting the tls->target_bitmap. It's setting d3d_display->target_bitmap to NULL, but it never touches the tls target bitmap. In my log I see that the bitmap that tls->target_bitmap refers to has been destroyed already in my code.
https://github.com/liballeg/allegro5/blob/5.1/src/tls.c#L420
https://github.com/liballeg/allegro5/blob/5.1/src/win/d3d_disp.cpp#L2417
I see - but shouldn't this catch it?
https://github.com/liballeg/allegro5/blob/5.1/src/bitmap.c#L207
I think the only place which would normally call _al_d3d_destroy_bitmap would be al_destroy_bitmap. (And then probably from special occasions like destroying a display.) In the al_destroy_bitmap case it looks like the target bitmap is reset to the display backbuffer or NULL if there is no current display on the thread.
Sorry for not actually looking into making a patch - can't easily test here so just looking at the code.
Do you know if the wrong target bitmap is/was the display backbuffer of a deleted display and not a normal bitmap? Are you deleting any displays or bitmaps at all?
Elias - if tls->current_display isn't reset, then running al_destroy_bitmap could reset the tls->target_bitmap to an already destroyed bitmap (namely the backbuffer in this case), by calling al_set_target_bitmap(al_get_backbuffer(display));. But looking closer, it appears that case is handled in al_destroy_display, on lines 159-160 :
if (display == al_get_current_display()) _al_set_current_display_only(NULL);