Crash in al_set_target_bitmap
Edgar Reynaldo

I'm having some difficulties debugging a crash in my ManyMouse project. For some reason it is crashing in al_set_target_bitmap. old_shader is usually 0xff0000ff or some other trash value, which would seem to indicate I'm dealing with a display (or at least a shader) that has already been destroyed.

gdb doesn't work very well on Windows 10 (it hangs when the program crashes) so I'll debug it more when I get home and I can use my Vista machine.

0x6773616e in al_set_target_bitmap (bitmap=0x2a5fa78) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
443           old_shader->vt->unuse_shader(old_shader, old_display);

GullRaDriel

If gdb does not work well, maybe it's because you made a total mess of the stack ^^
I would happily give it a go on linux. Wait: where is the code to debug ?

;D

Edgar Reynaldo

I removed a couple lines where I used al_hold_bitmap_drawing(false), and now once in a while I get a crash where an assertion fails in tls.c on line 403 : 

   ASSERT(!al_is_bitmap_drawing_held());

What does that mean? I never hold bitmap drawing, so I don't know why it would be true.

I originally added those lines in because of this same crash, and it seemed to work.

@Gully
It's a windows only project at the moment, because it uses the RawInput API from the win32 code library.

Elias

The font addon will change the target bitmap and also hold bitmap drawing (for example here [1])... but as long as you only call font functions from the thread which does drawing operations there should be no problem. If you have several threads make sure only one can do font functions (this includes things like measuring the width of a string).

References

  1. https://github.com/liballeg/allegro5/blob/5.1/addons/ttf/ttf.c#L582
Edgar Reynaldo

I draw each window in its own window callback registered with allegro, so each window's drawing operation is running in its own thread, at least for my mouse 'windows'. It's because I'm using a DIB to draw the window transparently, something which I haven't figured out how to do with allegro yet. A mouse cursor can't just be a solid rectangle.

Elias

Hm, make sure each window has its own, separate set of bitmaps and fonts in that case. In theory Allegro can support shared contexts but there's probably a few places (like in the font addon) which can't deal with it.

Edgar Reynaldo

Okay, I fixed one crash where I was calling al_set_target_backbuffer(NULL);. I mixed that up with al_set_target_bitmap(NULL).

Edit
-------------------
Shouldn't al_set_target_backbuffer(NULL) fail an assert or something? Or al_get_backbuffer(NULL)?
-------------------

But I'm still getting the original crash in al_set_target_bitmap. It is crashing on line 443 of tls.c : 

      old_shader->vt->unuse_shader(old_shader, old_display);

One time old_shader was 0xabababab, one time it was 0x1, and now I can't reproduce it at the moment. I think there is some uninitialized memory on the loose.

Okay, here's a log :

#SelectExpand
1Program received signal SIGSEGV, Segmentation fault. 2[Switching to Thread 5496.0x11e0] 30x6773616e in al_set_target_bitmap (bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 4443 old_shader->vt->unuse_shader(old_shader, old_display); 5(gdb) bt 6#0 0x6773616e in al_set_target_bitmap (bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 7#1 0x64884e7b in FCFSMouseStrategy::DrawPointers (this=0x22fa98, strategy_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:237 8#2 0x648851c7 in FCFSMouseStrategy::SetFCFSStrategyState (this=0x22fa98, new_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:293 9#3 0x64884907 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=..., process_input=true) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:158 10#4 0x648847b4 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:124 11#5 0x64883d75 in MouseController::HandleRawInput (this=0x22fa78, rawinput=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseController.cpp:155 12#6 0x0040476d in _fu68___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1403 13#7 0x00404380 in _fu60___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1205 14#8 0x0040177a in _fu0___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:44 15#9 0x677a45ba in window_callback (hWnd=0xa124a, message=255, wParam=0, lParam=-1984025671) at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:428 16#10 0x7611f8d2 in USER32!GetMessageW () from C:\Windows\system32\user32.dll 17#11 0x7611f794 in USER32!GetWindowLongW () from C:\Windows\system32\user32.dll 18#12 0x677a43b3 in accept_mouse_event () at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:372 19#13 0x000a124a in ?? () 20#14 0x000000ff in ?? () 21#15 0x00000000 in ?? () 22(gdb) bt 1 full 23#0 0x6773616e in al_set_target_bitmap (bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 24 tls = 0x29e4440 25 old_display = 0x29e0fe8 26 new_display = 0x29e0fe8 27 old_shader = 0x79d69c77 28 new_shader = 0x0 29 same_shader = false 30 bitmap_flags = 4097 31 __func__ = "al_set_target_bitmap" 32(More stack frames follow...) 33(gdb) p *old_shader
34Cannot access memory at address 0x79d69c77
35(gdb) quit 36A debugging session is active. 37 38 Inferior 1 [process 5496] will be killed.

Here's another log (it shows the bitmap is valid I believe)

#SelectExpand
1Program received signal SIGSEGV, Segmentation fault. 2[Switching to Thread 5752.0x123c] 30x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 4443 old_shader->vt->unuse_shader(old_shader, old_display); 5(gdb) p *bitmap 6$1 = {vt = 0x0, _format = 9, _flags = 4097, _display = 0x0, _memory_format = 0, w = 16, h = 24, pitch = 64, cl = 0, cr_excl = 16, ct = 0, cb_excl = 24, locked = false, lock_x = 0, lock_y = 0, lock_w = 16, lock_h = 24, lock_data = 0x2298358, lock_flags = 1, locked_region = {data = 0x2298358, format = 9, pitch = 64, pixel_size = 4}, transform = {m = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 7 0, 0, 1}}}, inverse_transform = {m = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}}}, inverse_transform_dirty = true, proj_transform = {m = {{0.125, 0, 0, 0}, {0, -0.0833333358, 0, 0}, {0, 0, 1, 0}, {-1, 1, 0, 1}}}, shader = 0x0, parent = 0x0, xofs = 0, yofs = 0, memory = 0x2298358 "", extra = 0x0, dirty = true} 8(gdb) p *old_shader 9Cannot access memory at address 0xff0000ff 10(gdb) bt 1 full 11#0 0x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 12 tls = 0x2214440 13 old_display = 0x2211b30 14 new_display = 0x2211b30 15 old_shader = 0xff0000ff 16 new_shader = 0x0 17 same_shader = false 18 bitmap_flags = 4097 19 __func__ = "al_set_target_bitmap" 20(More stack frames follow...) 21(gdb) bt 22#0 0x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 23#1 0x64884e7b in FCFSMouseStrategy::DrawPointers (this=0x22fa98, strategy_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:237 24#2 0x648851c7 in FCFSMouseStrategy::SetFCFSStrategyState (this=0x22fa98, new_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:293 25#3 0x64884907 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=..., process_input=true) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:158 26#4 0x648847b4 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:124 27#5 0x64883d75 in MouseController::HandleRawInput (this=0x22fa78, rawinput=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseController.cpp:155 28#6 0x0040476d in _fu68___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1403 29#7 0x00404380 in _fu60___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1205 30#8 0x0040177a in _fu0___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:44 31#9 0x677a45ba in window_callback (hWnd=0x1a156c, message=255, wParam=0, lParam=10627997) at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:428 32#10 0x7611f8d2 in USER32!GetMessageW () from C:\Windows\system32\user32.dll 33#11 0x7611f794 in USER32!GetWindowLongW () from C:\Windows\system32\user32.dll 34#12 0x677a43b3 in accept_mouse_event () at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:372 35#13 0x001a156c in ?? () 36#14 0x000000ff in ?? () 37#15 0x00000000 in ?? () 38(gdb)

Elias

Are you using al_use_shader anywhere? Are you ever destroying any shaders?

Edgar Reynaldo

No, not directly anyway.

Edit
-----------------
Elias - I think that _al_d3d_destroy_bitmap is not resetting the tls->target_bitmap. It's setting d3d_display->target_bitmap to NULL, but it never touches the tls target bitmap. In my log I see that the bitmap that tls->target_bitmap refers to has been destroyed already in my code.

https://github.com/liballeg/allegro5/blob/5.1/src/tls.c#L420

tls.c#SelectExpand
419 if (tls->target_bitmap) 420 old_shader = tls->target_bitmap->shader; 421 else 422 old_shader = NULL;

https://github.com/liballeg/allegro5/blob/5.1/src/win/d3d_disp.cpp#L2417

d3d_disp.cpp#SelectExpand
2411void _al_d3d_destroy_bitmap(ALLEGRO_BITMAP *bitmap) 2412{ 2413 ASSERT(!al_is_sub_bitmap(bitmap)); 2414 ALLEGRO_DISPLAY_D3D *d3d_display = (ALLEGRO_DISPLAY_D3D*)_al_get_bitmap_display(bitmap); 2415 2416 if (bitmap == d3d_display->target_bitmap) {
2417 d3d_display->target_bitmap = NULL;
2418 } 2419 2420 ALLEGRO_BITMAP_EXTRA_D3D *d3d_bmp = get_extra(bitmap); 2421 2422 if (d3d_bmp->video_texture) { 2423 if (d3d_bmp->video_texture->Release() != 0) { 2424 ALLEGRO_WARN("d3d_destroy_bitmap: Release video texture failed.\n"); 2425 } 2426 } 2427 if (d3d_bmp->system_texture) { 2428 if (d3d_bmp->system_texture->Release() != 0) { 2429 ALLEGRO_WARN("d3d_destroy_bitmap: Release system texture failed.\n"); 2430 } 2431 } 2432 2433 if (d3d_bmp->render_target) { 2434 if (d3d_bmp->render_target->Release() != 0) { 2435 ALLEGRO_WARN("d3d_destroy_bitmap: Release render target failed.\n"); 2436 } 2437 } 2438 2439 al_free(bitmap->extra); 2440}

Elias

I see - but shouldn't this catch it?

https://github.com/liballeg/allegro5/blob/5.1/src/bitmap.c#L207

#SelectExpand
1void al_destroy_bitmap(ALLEGRO_BITMAP *bitmap) 2{ 3 if (!bitmap) { 4 return; 5 } 6 7 /* As a convenience, implicitly untarget the bitmap on the calling thread 8 * before it is destroyed, but maintain the current display. 9 */ 10 if (bitmap == al_get_target_bitmap()) { 11 ALLEGRO_DISPLAY *display = al_get_current_display(); 12 if (display) 13 al_set_target_bitmap(al_get_backbuffer(display)); 14 else 15 al_set_target_bitmap(NULL); 16 }

I think the only place which would normally call _al_d3d_destroy_bitmap would be al_destroy_bitmap. (And then probably from special occasions like destroying a display.) In the al_destroy_bitmap case it looks like the target bitmap is reset to the display backbuffer or NULL if there is no current display on the thread.

Sorry for not actually looking into making a patch - can't easily test here so just looking at the code.

Do you know if the wrong target bitmap is/was the display backbuffer of a deleted display and not a normal bitmap? Are you deleting any displays or bitmaps at all?

Edgar Reynaldo

Elias - if tls->current_display isn't reset, then running al_destroy_bitmap could reset the tls->target_bitmap to an already destroyed bitmap (namely the backbuffer in this case), by calling al_set_target_bitmap(al_get_backbuffer(display));. But looking closer, it appears that case is handled in al_destroy_display, on lines 159-160 : 

   if (display == al_get_current_display()) 
      _al_set_current_display_only(NULL);

Thread #615871. Printed from Allegro.cc