I recently upgraded Allegro on my Mac to v4.2.0 Beta 3. The compile went fine, with occasional warnings here and there. I'm able to compile and run my game fine, with no real issues. My problem is with Grabber. While trying to update one of my datafiles, Grabber put me into the "spinning pinwheel of death" mode while saving. My datafile is now destroyed (nothing to cry over, I have backups). What I find interesting, though, is how the grabber dies.

When I run grabber in windowed mode, it can die on loading and/or saving a prexisting datafile. It can also die when simply starting up. The program loads, I get a window with a black screen, and then the allegro gui never displays.

Doing the same operations while running in full screen mode, however, I don't see any of the same issues.

At first I thought it might be because it's trying to run in some color depth that OSX might not like in windowed mode (my PowerBook is picky and prefers 15bit to 16bit when running in a window). That wasn't it either. It just appears that Grabber locks up and "pinwheels" for eternity when running in windowed mode.

Another thing I noticed is that the mouse cursor is all screwed up. Most of the time it's fine, but sometimes it decides that the actual position of the mouse pointer and the mouse cursor should no longer point to the same place. For instance, I will have my mouse cursor clearly over the file menu, but the object menu is blackened (as if I'm hovering over it).

Can anyone else running the latest OSX/Allegro verify this for me?
I'm running it from the terminal, issuing the -windowed switch.

[Edit:] By the way, if it offers any help: Previous to this I was running Allegro 4.1.15. I would run grabber in windowed mode for all of my datafile editing and never once encountered this pseudo-hanging issue.

[Edit 2]: Some more additional info. I ran it (non debug mode version) through GDB and got some information for a session where it was stuck trying to load (it shows a black screen and the Allegro gui has yet to appear)

[Edit 3]: I compiled the debugging version. Using GDB I was able to gather some nice information from the thread that all of the good Allegro stuff runs in:

(gdb) thread 2
[Switching to thread 2 (process 5487 thread 0x3d03)]
#0  0x9002ca98 in semaphore_wait_signal_trap ()
(gdb) bt
#0  0x9002ca98 in semaphore_wait_signal_trap ()
#1  0x90001c2c in pthread_mutex_lock ()
#2  0x300dad20 in osx_mouse_set_sprite (sprite=0x380590, x=1, y=1) at src/macosx/qzmouse.m:347
#3  0x30074f50 in show_mouse (bmp=0x181aa00) at src/mouse.c:633
#4  0x30047e18 in do_dialog (dialog=0x2cc28, focus_obj=48) at src/gui.c:797
#5  0x00010f68 in _mangled_main (argc=2, argv=0xbffffcac) at tools/grabber.c:3868
#6  0x000260e4 in +[AllegroAppDelegate app_main:] (self=0x390c0, _cmd=0x2a924, arg=0x0) at src/macosx/main.m:168
#7  0x9287c2b4 in forkThreadForFunction ()
#8  0x9002c3d4 in _pthread_body ()
(gdb)

I guess it depends on what operation you're doing. I never was able to get it to not crash when saving a slightly large datafile. With loading a datafile, and loading up the program initially, I'd say it's reproducible to ~ < 10 runs. As for binary searching for patches with CVS: I have no idea how to do that. I'm not very familiar with CVS at all.

Allegro 4.2.0 Beta 2 is the one that doesn't compile correctly for OSX without modification:

gcc -DALLEGRO_SRC -DALLEGRO_LIB_BUILD -Wall -Wno-unused -Wno-long-double  -O2 -funroll-loops -ffast-math -fomit-frame-pointer -fno-common -pipe -dynamic -DALLEGRO_USE_C -I. -I./include -o obj/macosx/alleg/gui.o -c src/gui.c
src/gui.c: In function 'update_menu':
src/gui.c:1816: error: invalid storage class for function 'shutdown_single_menu'

I'll have to look up some instructions on how to fix this problem. If I remember correctly, Thomas Harte has a fix lingering around on the forums somewhere. [Edit]: Found the thread in case anyone else might want to test this out, as well: [url http://www.allegro.cc/forums/view_thread.php?_id=486790]

One thing I noticed while building was that it was complaining that plugin(s).h did not exist in obj/macosx. I did a diff of 4.1.15 and 4.2.0 Beta 3 and noticed there were no differences, so I just made a copy of the file and the compile completed fine.

[Edit 2]: I have confirmed that this issue exists in Allegro 4.2.0 Beta 2 as well. In my initial bunch of runs I was able to get it to hang on load on the third try. I then proceeded to double test 4.1.15. It had no problems. I then decided that maybe I had read the version incorrectly in the Grabber's help tool, so I went to load it (4.2.0 Beta 2) once again, through gdb. On the first try I got the hanging issue. The mouse focus issue still exists as well, but I think this is a problem with scrolling the mouse off of the window's focus and then back into the window's focus.

The issue was apparently introduced between versions 4.1.15 and 4.2.0 Beta 2.

If you can offer me some guidance on how to do binary searching via CVS I'd be glad to help debug this problem further.

I'm afraid, now, that I might have been building Grabber, but it was not using the correct dynamic libraries. For instance, I was building grabber, but not installing the Allegro library for that particular build. Is this the correct way of going about this process? I'm not sure. In the help menu for the Grabber it reported the correct version of Allegro. I'm not sure if this is something that is determined dynamically, or is stuck in there at compile time. The reason I bring up this issue is the following: after building 4.1.18 I got some errors when loading into GDB; mainly:

Reading symbols for shared libraries . done
dyld: lazy symbol binding failed: Symbol not found: __get_allegro_version
  Referenced from: /Current Project/allegro_development/allegro-4-1-18/tools/grabber
  Expected in: /usr/local/lib/liballeg-4.1.dylib

dyld: Symbol not found: __get_allegro_version
  Referenced from: /Current Project/allegro_development/allegro-4-1-18/tools/grabber
  Expected in: /usr/local/lib/liballeg-4.1.dylib

liballeg-4.1.dylib is just a symlink to my (working) 4.1.15 version of the library.

[Edit]: I played around with the symlink and made it point to my local version of the 4.1.18 library. Bingo, the error still occurs. When I built 4.1.18 I had to, again, make a copy of the plugins.h file (from 4.1.15) and put it into the ./obj/macosx directory. I had to make the other simple changes to src/gui.c and src/sound.c (the static issue).

It appears, now, that this bug was introduced between 4.1.15 and 4.1.18. I did even more sanity checking to make sure that 4.1.15 doesn't create this problem (multiple loads into gdb). I still can't get it to happen under 4.1.15. (By the way, all of my Allegro development under OSX has mostly been done with 4.1.15--which includes using the grabber tool on a daily basis. In all of that time, I've never had it lock up on me).

[Edit 2]: Ok. So, just to ensure I haven't been doing things wrong, I re-set up my symlinks (in /usr/local/lib) to explicitly point to the libraries I wanted them to point to (the working versions that I had just built) I was still able to get Beta 2 to crash, as well as 4.1.18. Talking with a fellow developer in #allegro, they were unable to get grabber to crash running 4.1.18 [Their system: G4 running OSX v10.3]. [Sub edit: upon further inspection I noticed the symlink: liballeg.dylib, which basically points to the latest version of the allegro library (beta 3 in my case). I'm not sure if this symlink really affects anything, as I'm able to run my 4.1.15 version of the grabber with no sideaffects. Oh, the library madness)].

I'm fairly new to helping out with Allegro development, so any general pointers will be much appreciated. For instance, is there some preferred method to ensure that your executable is running the proper library, etc, etc? I'd really be disappointed if this issue was something between beta3 and beta2 and my lack of knowledge in this kind of thing led me on a wild goose chase.

I'm done debugging this issue for tonight. Hopefully over the weekend I'll be able to provide some more information if you need any additional testing done. Also, by that time, someone else with a Mac (possibly running Tiger) will have been able to test this issue out.

Yes, this definitely qualifies as a show-stopper.

Depends. Generally not, but it should be fine if you staticlink. Note that something compared against Allegro 4.2 should not even start if you only have a 4.1.x dynamic library lying around.

This is actually not the proper way to test this. Allegro 4.1 libraries are not ABI compatible, so you can't just substitute 4.1.18 for 4.1.15 and expect it to work. You need to recompile and relink your programme.
In cases like this, I would (in Linux) build a static version of the library without modules to make sure I'm using what I think I'm using. I don't know how to do the same under MacOS X, or even if it's possible at all (see the readme file though, it should be documented there).

Eek! My guess is that that should include a version number in the name somewhere (because different versions of the library are incompatible). This is a problem with the MacOS X build/install system we need to fix, I think.

You're doing fine so far.

As I said, I would staticlink under Linux, probably under Windows too. I'm not sure how to do that under MacOS X though.
The best alternative I can think of is to run `make uninstall' for the version you currently have installed (check that it's gone) and then run `make install' for the new version.

Just keep the source tree in tact, you'll be able to rerun make install from there.

If it is between 4.1.15 and 4.2 beta2, then please try to narrow it down further, say between 4.1.16 and 4.1.17.

Take what time you need. I'll postpone the 4.2.0 RC1 until this is fixed, and it's more important that we get it right than that we rush the release out of the door.

otool -L is the equivalent.

Hmm. Should we not add a Function to grab the version of the running allegro? So we can compare the compiled version vs. the running version?

I'm waiting for Ultio to report back about the static-link and also if exmouse shows the same problem. Otherwise there's not enough info to go on. I tried starting grabber via gdb about 20 times and did not see the problem, but that doesn't necessarily mean anything.

Pete

So 4.1.16 is OK and 4.1.17 is not OK?

There were quite a few differences to the OSX code in that time-period.

Attached is the output of cvs diff -u -r v4-1-16 -r v4-1-17 src/macosx/

I'll look into it!

Pete

[edit]
OK, some progress but I'm really tired so I won't finish it today. Brain dump follows:
1. I have seen the problem that you describe, Ultio I got it 2 times out of 3 by starting gdb then 'wobbling' the terminal window around with the mouse, until the grabber window came up.
2. It looks like the problem might be with the mutexes osx_window_mutex and osx_event_mutex
From what I can see, show_mouse calls acquire_screen, which locks the window mutex. Then it calls gfx_driver->set_mouse_sprite.
osx_window_mutex arbitrates between the main thread which handles events and screen updates, and a secondary thread which is the actual Allegro app code (the grabber in this case). osx_event_mutex is something to do with events ( ) I think that set_mouse_sprite (in thread #2) is waiting on the event mutex, which won't get released because the main thread is waiting for the window mutex because it wants to update dirty lines.

What I am thinking is that the update dirty lines is not so critical - if it can't lock the mutex it should just continue, which would result in one frame dropped, but no deadlocks. I'll try this as an option - in the meantime I'd love to hear from any thread experts.

Night night!

Pete

OK, have a look at this code; it's the main event loop (thread #1)

while (1) {
      pthread_mutex_lock(&osx_event_mutex);
      osx_event_handler();
      if (osx_gfx_mode == OSX_GFX_WINDOW)
         osx_update_dirty_lines();
      else if (osx_gfx_mode == OSX_GFX_FULL) {
         if ((osx_palette) && (osx_palette_dirty)) {
            CGDisplaySetPalette(kCGDirectMainDisplay, osx_palette);
      osx_palette_dirty = FALSE;
   }
      }
      pthread_mutex_unlock(&osx_event_mutex);
      usleep(1000000 / refresh_rate);
   }

You can that, when osx_update_dirty_lines() is called, the osx_event_mutex is held. osx_update_dirty_lines locks the window mutex, does the update and unlocks it.
In the app itself (thread #2), the grabber calls acquire_screen, which holds the window mutex. It then does a set_mouse_sprite, which ends up in osx_set_mouse_sprite:

int osx_mouse_set_sprite(BITMAP *sprite, int x, int y)
{
...snip...
   pthread_mutex_lock(&osx_event_mutex);
   osx_cursor = requested_cursor = cursor;
   pthread_mutex_unlock(&osx_event_mutex);
   if (temp)
      destroy_bitmap(temp);

#undef GET_PIXEL_DATA
   return 0;
}

So, what can happen is that thread #1 holds the event mutex, thread #2 holds the window mutex, and neither can proceed without the other releasing the other mutex. Note also that the fullscreen code is different - that's why you don't see it in fullscreen.

Solutions: osx_update_dirty_lines can use pthread_mutex_trylock instead of pthread_mutex_lock. The problem is if the app is spending almost all of its time with the screen acquired, then osx_update_dirty lines will be
unlikely to obtain the window mutex, and, although the app will run, the screen won't update. A time-out on the lock might help - e.g try for 100ms and then give up.

Alternatively, I'm not sure why the event mutex needs to be held when osx_update_dirty_lines is called. Maybe these two can be separated, so thread #1 never needs to hold both at once.

I wonder if someone can check my logic on this, and I will prepare some patches for testing.

Pete

Is the code for both thread 1 and thread 2 Allegro code? For one reason or another I was under the impression that the code for thread 1 was some OSX specific stuff that we didn't really have access to. Now that I look at it, though, it seems as if it's Allegro source that makes some calls to OSX native functions.

Care to share what files these are that you're looking at?

I changed the code to this and the grabber hang problem went away, so it's definately the culprit:

ZD was built with 4.1.13 I believe, and I haven't compiled it against anything new since then.

If nothing has changed in these files between 4.1.15 and 4.2.0 Beta 3 I can build a temporary version of allegro and link my current project against it. Currently it doesn't use any mouse related stuff but I can always toss some in just for the heck of it.

I'd suggest PMing lillo. I know he still pokes around from time to time. We could always try simply emailing him, as well.