Crash in al_set_target

Crash in al_set_target_bitmap

Edgar Reynaldo

Major Reynaldo

May 2007

I'm having some difficulties debugging a crash in my ManyMouse project. For some reason it is crashing in al_set_target_bitmap. old_shader is usually 0xff0000ff or some other trash value, which would seem to indicate I'm dealing with a display (or at least a shader) that has already been destroyed.

gdb doesn't work very well on Windows 10 (it hangs when the program crashes) so I'll debug it more when I get home and I can use my Vista machine.

0x6773616e in al_set_target_bitmap (bitmap=0x2a5fa78) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
443           old_shader->vt->unuse_shader(old_shader, old_display);

Eagle and Allegro 5 binaries | Older Allegro 4 and 5 binaries | Allegro 5 compile guide
King Piccolo will make it so Then I could just flip it with some sweet, sweet matrix math. In either case, I’ll ride this wagon until the wheels fall off. Thanks to those that keep it running

GullRaDriel

Member #3,861

September 2003

If gdb does not work well, maybe it's because you made a total mess of the stack ^^
I would happily give it a go on linux. Wait: where is the code to debug ?

"Code is like shit - it only smells if it is not yours"
Allegro Wiki, full of examples and articles !!

Edgar Reynaldo

Major Reynaldo

May 2007

I removed a couple lines where I used al_hold_bitmap_drawing(false), and now once in a while I get a crash where an assertion fails in tls.c on line 403 :

   ASSERT(!al_is_bitmap_drawing_held());

What does that mean? I never hold bitmap drawing, so I don't know why it would be true.

I originally added those lines in because of this same crash, and it seemed to work.

@Gully
It's a windows only project at the moment, because it uses the RawInput API from the win32 code library.

Elias

Member #358

May 2000

The font addon will change the target bitmap and also hold bitmap drawing (for example here ^[1])... but as long as you only call font functions from the thread which does drawing operations there should be no problem. If you have several threads make sure only one can do font functions (this includes things like measuring the width of a string).

References

https://github.com/liballeg/allegro5/blob/5.1/addons/ttf/ttf.c#L582

--
"Either help out or stop whining" - Evert

Edgar Reynaldo

Major Reynaldo

May 2007

I draw each window in its own window callback registered with allegro, so each window's drawing operation is running in its own thread, at least for my mouse 'windows'. It's because I'm using a DIB to draw the window transparently, something which I haven't figured out how to do with allegro yet. A mouse cursor can't just be a solid rectangle.

Elias

Member #358

May 2000

Hm, make sure each window has its own, separate set of bitmaps and fonts in that case. In theory Allegro can support shared contexts but there's probably a few places (like in the font addon) which can't deal with it.

--
"Either help out or stop whining" - Evert

Edgar Reynaldo

Major Reynaldo

May 2007

Okay, I fixed one crash where I was calling al_set_target_backbuffer(NULL);. I mixed that up with al_set_target_bitmap(NULL).

Edit
-------------------
Shouldn't al_set_target_backbuffer(NULL) fail an assert or something? Or al_get_backbuffer(NULL)?
-------------------

But I'm still getting the original crash in al_set_target_bitmap. It is crashing on line 443 of tls.c :

      old_shader->vt->unuse_shader(old_shader, old_display);

One time old_shader was 0xabababab, one time it was 0x1, and now I can't reproduce it at the moment. I think there is some uninitialized memory on the loose.

Okay, here's a log :

#SelectExpand
  1Program received signal SIGSEGV, Segmentation fault.
  2[Switching to Thread 5496.0x11e0]  30x6773616e in al_set_target_bitmap(bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
  4443           old_shader->vt->unuse_shader(old_shader, old_display);  5(gdb) bt
  6#0  0x6773616e in al_set_target_bitmap (bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443  7#1  0x64884e7b in FCFSMouseStrategy::DrawPointers (this=0x22fa98, strategy_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:237  8#2  0x648851c7 in FCFSMouseStrategy::SetFCFSStrategyState (this=0x22fa98, new_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:293  9#3  0x64884907 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=..., process_input=true) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:158 10#4  0x648847b4 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:124 11#5  0x64883d75 in MouseController::HandleRawInput (this=0x22fa78, rawinput=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseController.cpp:155 12#6  0x0040476d in _fu68___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1403 13#7  0x00404380 in _fu60___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1205 14#8  0x0040177a in _fu0___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:44 15#9  0x677a45ba in window_callback (hWnd=0xa124a, message=255, wParam=0, lParam=-1984025671) at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:428 16#10 0x7611f8d2 in USER32!GetMessageW () from C:\Windows\system32\user32.dll 17#11 0x7611f794 in USER32!GetWindowLongW () from C:\Windows\system32\user32.dll 18#12 0x677a43b3 in accept_mouse_event () at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:372 19#13 0x000a124a in ?? () 20#14 0x000000ff in ?? () 21#15 0x00000000 in ?? () 22(gdb) bt 1 full
 23#0  0x6773616e in al_set_target_bitmap (bitmap=0x29db708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443 24        tls =0x29e4440 25        old_display =0x29e0fe8 26        new_display =0x29e0fe8 27        old_shader =0x79d69c77 28        new_shader =0x0 29        same_shader =false 30        bitmap_flags =4097 31        __func__ ="al_set_target_bitmap" 32(More stack frames follow...) 33(gdb) p *old_shader
 34Cannot access memory at address 0x79d69c77
 35(gdb) quit
 36A debugging session is active.
 37 38        Inferior 1[process 5496] will be killed.

Here's another log (it shows the bitmap is valid I believe)

#SelectExpand
  1Program received signal SIGSEGV, Segmentation fault.
  2[Switching to Thread 5752.0x123c]
  30x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
  4443           old_shader->vt->unuse_shader(old_shader, old_display);
  5(gdb) p *bitmap
  6$1 = {vt = 0x0, _format = 9, _flags = 4097, _display = 0x0, _memory_format = 0, w = 16, h = 24, pitch = 64, cl = 0, cr_excl = 16, ct = 0, cb_excl = 24, locked = false, lock_x = 0, lock_y = 0, lock_w = 16, lock_h = 24, lock_data = 0x2298358, lock_flags = 1, locked_region = {data = 0x2298358, format = 9, pitch = 64, pixel_size = 4}, transform = {m = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0,
  7        0, 0, 1}}}, inverse_transform = {m = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}}}, inverse_transform_dirty = true, proj_transform = {m = {{0.125, 0, 0, 0}, {0, -0.0833333358, 0, 0}, {0, 0, 1, 0}, {-1, 1, 0, 1}}}, shader = 0x0, parent = 0x0, xofs = 0, yofs = 0, memory = 0x2298358 "", extra = 0x0, dirty = true}
  8(gdb) p *old_shader
  9Cannot access memory at address 0xff0000ff
 10(gdb) bt 1 full
 11#0  0x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
 12        tls = 0x2214440
 13        old_display = 0x2211b30
 14        new_display = 0x2211b30
 15        old_shader = 0xff0000ff
 16        new_shader = 0x0
 17        same_shader = false
 18        bitmap_flags = 4097
 19        __func__ = "al_set_target_bitmap"
 20(More stack frames follow...)
 21(gdb) bt
 22#0  0x6773616e in al_set_target_bitmap (bitmap=0x220b708) at C:\mingw\LIBS\A5112release\allegro\src\tls.c:443
 23#1  0x64884e7b in FCFSMouseStrategy::DrawPointers (this=0x22fa98, strategy_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:237
 24#2  0x648851c7 in FCFSMouseStrategy::SetFCFSStrategyState (this=0x22fa98, new_state=FCFS_STATE_PROCESSING_CLICK) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:293
 25#3  0x64884907 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=..., process_input=true) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:158
 26#4  0x648847b4 in FCFSMouseStrategy::HandleInput (this=0x22fa98, input=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseStrategy.cpp:124
 27#5  0x64883d75 in MouseController::HandleRawInput (this=0x22fa78, rawinput=...) at C:\Research\RawInput\RainDropMouse\ManyMouse\src\MouseController.cpp:155
 28#6  0x0040476d in _fu68___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1403
 29#7  0x00404380 in _fu60___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:1205
 30#8  0x0040177a in _fu0___ZN9ManyMouse3logE () at C:\Research\RawInput\RainDropMouse\ManyMouse\src\RawInputHandler.cpp:44
 31#9  0x677a45ba in window_callback (hWnd=0x1a156c, message=255, wParam=0, lParam=10627997) at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:428
 32#10 0x7611f8d2 in USER32!GetMessageW () from C:\Windows\system32\user32.dll
 33#11 0x7611f794 in USER32!GetWindowLongW () from C:\Windows\system32\user32.dll
 34#12 0x677a43b3 in accept_mouse_event () at C:\mingw\LIBS\A5112release\allegro\src\win\wwindow.c:372
 35#13 0x001a156c in ?? ()
 36#14 0x000000ff in ?? ()
 37#15 0x00000000 in ?? ()
 38(gdb)

Elias

Member #358

May 2000

Are you using al_use_shader anywhere? Are you ever destroying any shaders?

--
"Either help out or stop whining" - Evert

Edgar Reynaldo

Major Reynaldo

May 2007

No, not directly anyway.

Edit
-----------------
Elias - I think that _al_d3d_destroy_bitmap is not resetting the tls->target_bitmap. It's setting d3d_display->target_bitmap to NULL, but it never touches the tls target bitmap. In my log I see that the bitmap that tls->target_bitmap refers to has been destroyed already in my code.

https://github.com/liballeg/allegro5/blob/5.1/src/tls.c#L420

tls.c#SelectExpand
 if (tls->target_bitmap)
    old_shader = tls->target_bitmap->shader;
 else
    old_shader = NULL;

https://github.com/liballeg/allegro5/blob/5.1/src/win/d3d_disp.cpp#L2417

d3d_disp.cpp#SelectExpand
 2411void _al_d3d_destroy_bitmap(ALLEGRO_BITMAP*bitmap) 2412{ 2413ASSERT(!al_is_sub_bitmap(bitmap)); 2414   ALLEGRO_DISPLAY_D3D *d3d_display =(ALLEGRO_DISPLAY_D3D*)_al_get_bitmap_display(bitmap); 2415 2416if(bitmap == d3d_display->target_bitmap){ 2417      d3d_display->target_bitmap = NULL;
 2418} 2419 2420   ALLEGRO_BITMAP_EXTRA_D3D *d3d_bmp = get_extra(bitmap); 2421 2422if(d3d_bmp->video_texture){ 2423if(d3d_bmp->video_texture->Release()!=0){ 2424         ALLEGRO_WARN("d3d_destroy_bitmap: Release video texture failed.\n"); 2425} 2426} 2427if(d3d_bmp->system_texture){ 2428if(d3d_bmp->system_texture->Release()!=0){ 2429         ALLEGRO_WARN("d3d_destroy_bitmap: Release system texture failed.\n"); 2430} 2431} 2432 2433if(d3d_bmp->render_target){ 2434if(d3d_bmp->render_target->Release()!=0){ 2435         ALLEGRO_WARN("d3d_destroy_bitmap: Release render target failed.\n"); 2436} 2437} 2438 2439al_free(bitmap->extra); 2440}

Elias

Member #358

May 2000

I see - but shouldn't this catch it?

https://github.com/liballeg/allegro5/blob/5.1/src/bitmap.c#L207

#SelectExpand
  1void al_destroy_bitmap(ALLEGRO_BITMAP *bitmap)
  2{
 if (!bitmap) {
    return;
 }
  6
 /* As a convenience, implicitly untarget the bitmap on the calling thread
  * before it is destroyed, but maintain the current display.
  */
 if (bitmap == al_get_target_bitmap()) {
    ALLEGRO_DISPLAY *display = al_get_current_display();
    if (display)
       al_set_target_bitmap(al_get_backbuffer(display));
    else
       al_set_target_bitmap(NULL);
 }

I think the only place which would normally call _al_d3d_destroy_bitmap would be al_destroy_bitmap. (And then probably from special occasions like destroying a display.) In the al_destroy_bitmap case it looks like the target bitmap is reset to the display backbuffer or NULL if there is no current display on the thread.

Sorry for not actually looking into making a patch - can't easily test here so just looking at the code.

Do you know if the wrong target bitmap is/was the display backbuffer of a deleted display and not a normal bitmap? Are you deleting any displays or bitmaps at all?

--
"Either help out or stop whining" - Evert

Edgar Reynaldo

Major Reynaldo

May 2007

Elias - if tls->current_display isn't reset, then running al_destroy_bitmap could reset the tls->target_bitmap to an already destroyed bitmap (namely the backbuffer in this case), by calling al_set_target_bitmap(al_get_backbuffer(display));. But looking closer, it appears that case is handled in al_destroy_display, on lines 159-160 :

   if (display == al_get_current_display()) 
      _al_set_current_display_only(NULL);