Setup: Allegro 5.2.2.1 installed into Visual Studio 2015 via NuGet: https://www.nuget.org/packages/Allegro/ an a 64-bit Windows 7.
TL;DR I believe that ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY is being incorrectly fired while the mouse is outside the window but is moving over its edge.
I extended the tutorial code from https://wiki.allegro.cc/index.php?title=Allegro_5_Tutorial/Input to include both mouse-enter and mouse-leave functionality. Essentially, the bouncer will bounce from wall to wall while no mouse is over the window:
{"name":"610949","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/7\/e\/7eac12146bffadc87b1e2153871c9572.png","w":816,"h":638,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/7\/e\/7eac12146bffadc87b1e2153871c9572"}
When the mouse is inside the window, the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY event is called, and the bouncer follows the mouse and its position is displayed on the 2nd line:
{"name":"610952","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/4\/f\/4ff62701ca38ec2fa14e2b1be8d5c7b7.png","w":816,"h":638,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/4\/f\/4ff62701ca38ec2fa14e2b1be8d5c7b7"}
Moving the mouse back off the window fires the ALLEGRO_EVENT_MOUSE_LEAVE_DISPLAY event, which allows the bouncer to wall-bounce again.
Around the edge of the window is a 5-pixel "resize" edge. My screen is 800x600, and if my mouse is exactly x=800, ALLEGRO_EVENT_MOUSE_LEAVE_DISPLAY is fired. If my mouse moves anywhere between x=801 to x=805, the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY event is fired a second time. But moving my mouse totally off the window (x=806 and further), no ALLEGRO_EVENT_MOUSE_LEAVE_DISPLAY is called a second time, meaning that my mouse is still "holding" the bouncer in place.
{"name":"610953","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/3\/0\/305a0d92a3b8480d840274f7a694e4f2.png","w":816,"h":638,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/3\/0\/305a0d92a3b8480d840274f7a694e4f2"}
Is this an engine issue, or should I deal with this in code and "simulate" a second ALLEGRO_EVENT_MOUSE_LEAVE_DISPLAY event by detecting the actual mouse position in relation to the screen's dimensions?
EDIT:
A fix is to just programmatically ignore invalid mouse positions:
Bumping so it doesn't get locked. Haven't had time to look at this yet.
Part of the problem I see is this code current_pos.x > SCREEN_W. The right edge of the screen is SCREEN_W-1, not SCREEN_W. You should have something like current_pos.x >= SCREEN_W.
I haven't examined your code too thuroughly, but that stood out for me. The screen goes from zero to SCREEN_W-1 in the X direction and zero to SCREEN_H-1 in the Y. In this case your screen goes from zero to 799 in the X, and zero to 599 in the Y. If you're testing to see if it is greater than 800, than what happens if it is on 800 in the X, or on 600 in the Y?
Changing > SCREEN_W to >= SCREEN_W changes nothing about the issue with the allegro event being fired: all it does is make the cube bounce one pixel sooner.
The mouse still fires off the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY event when it moves over the window edge.
Under the Windows 7 Control Panel: Control Panel\All Control Panel Items\Personalization I changed to another theme which had about a 2-pixel-thick window border. This event is NOT fired when I move around that border.
The default Windows 7 theme (Aero), however, does have this 5-pixel-thick border.
I tried to write this sample code above as platform-agnostic as possible so it could be tested on other systems (Linux, OS X, ...) to rule out whether this was platform and/or theme specific.
The mouse still fires off the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY event when it moves over the window edge.
Have you tried the example programs to see if they do the same thing? ex_mouse_focus.exe is a good test, it reports when the mouse goes outside the display and seems to work no problem for me. It only reports when the mouse is over the actual display and not the border.
Also ex_mouse_events.exe also accurately reports the mouse outside the display.
Edit: I noticed in this section of code a potential problem...
else if (event.type == ALLEGRO_EVENT_MOUSE_AXES || event.type == ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY) {
...the ball will be mouse controlled under two conditions, entering the display OR ALLEGRO_EVENT_MOUSE_AXES triggers. So, is the mouse axes still firing while the mouse is outside the display? I would think so.
ex_mouse_focus.exe turns from black to red when the mouse is over it. When the mouse is over that 5-pixel window edge, it displays "Outside either display", which is repeated as I drag my mouse around it. When I move completely off the window, all text output stops; when I move into the window, it becomes red and the console shows the "In display 1, x = ..., y = ..." properly.
ex_mouse_events.exe only updates the mouse coordinates while over the window: it doesn't update anything while I drag over the window edges.
I can separate the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY from the ALLEGRO_EVENT_MOUSE_AXES events, although I probably need to detect on launch whether the mouse is already over the window, else the ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY event will not get initially fired.
EDIT:
I wasn't able to properly detect whether the mouse was over the window when the display is created, so a work-around had to be used instead. I introduced a b_mouse_in_window and check it, as well as whether the bouncer is controlled, when event.type == ALLEGRO_EVENT_MOUSE_AXES. I update that value when either event.type == ALLEGRO_EVENT_MOUSE_ENTER_DISPLAY or event.type == ALLEGRO_EVENT_MOUSE_LEAVE_DISPLAY.
That seems to get rid of the problem with the window borders. Code below if anyone wanted to see the changes to the events.
I just wish that I could get rid of that nasty b_mouse_in_window check within the loop. Like just check the mouse position before I enter `while (1) { ... }` and set the bouncer->SetControlled() value there.