Allegro5, OpenGL contexts, and transparent windows
Edgar Reynaldo

Okay, here's the basic idea. I'm creating layered windows with the intent to make them shaped and either overlaid or blended using UpdateLayeredWindow or SetLayeredWindowAttributes for the purposes of drawing multiple mice in one or more overlay windows, an ability which is currently unsupported by the Windows OS.

First you have to create a layered window. You can set this after window creation (so you can use an allegro window handle from al_get_win_window_handle) using SetWindowLong (if (0 == SetWindowLong(hwnd , GWL_EXSTYLE , WS_EX_LAYERED) {Fail();}).

However, I have had difficulty with this approach.

I have figured out how to draw transparent areas using FillRect and GetDC.

#SelectExpand
1 handle = al_get_win_window_handle(display); 2 3 if (0 == SetWindowLong(handle , GWL_EXSTYLE , WS_EX_LAYERED)) { 4 log.Log("Couldn't set WS_EX_LAYERED style attribute\n"); 5 } 6 7 COLORREF trans_key = RGB(0,0,0); 8 9 HDC hdc = GetDC(handle); 10/* 11typedef struct _RECT { 12 LONG left; 13 LONG top; 14 LONG right; 15 LONG bottom; 16} RECT, *PRECT; 17*/ 18 RECT r; 19 r.left = sw/3; 20 r.right = (2*sw)/3; 21 r.top = 0; 22 r.bottom = sh; 23 24 RECT r2; 25 r2.left = 0; 26 r2.right = sw; 27 r2.top = sh/3; 28 r2.bottom = (2*sh)/3; 29 30 HBRUSH hbrush_trans = CreateSolidBrush(trans_key); 31 if (!hbrush_trans) { 32 log.Log("Failed to create HBRUSH.\n"); 33 ReleaseDC(handle , hdc); 34 return 12; 35 } 36 37 FillRect(hdc , &r , hbrush_trans); 38 FillRect(hdc , &r2 , hbrush_trans); 39/* 40 for (int i = 0 ; i < sw*sh/10 ; ++i) { 41 int x = (rand()/(float)RAND_MAX)*sw; 42 int y = (rand()/(float)RAND_MAX)*sh; 43 SetPixel(hdc, x , y , trans_key); 44 } 45*/ 46 DeleteObject(hbrush_trans); 47 hbrush_trans = 0; 48 49 if (!SetLayeredWindowAttributes(handle , trans_key , alpha , LWA_COLORKEY | LWA_ALPHA)) { 50 log.Log("Couldn't set color key!\n"); 51 }

What I would like is to be able to draw with allegro instead, as it has image loading and drawing functions so I don't have to write them all over again myself.

So this brings up a few questions. How does allegro draw to its window's HDC? Ie. is it possible for me to use al_map_rgb or al_map_rgba to match the color key I created and set for the window so I can draw transparent areas using allegro functions? (COLORREF trans_key = RGB(0,0,0);).

I've seen a way to create a HGLRC (Handle to GL Render Context) using wglCreateContext and a PIXELFORMATDESCRIPTOR that draws into a DIB but I don't know how to hack allegro to draw to this context instead of the one they create (alternative #2).

If I can either draw directly to allegro's window transparently or set allegro's opengl context to draw into a DIB then I could accomplish what I have set out to do. Any help in any area would be appreciated.

I will link several programs that demonstrate drawing an overlay window onto the desktop, that updates and animates in real time. What I want is to be able to adapt them to work with allegro if possible.

The set of layered window demos here all compile with MinGW 4.8.1 using only minor modifications.

http://www.dhpoware.com/demos/index.html

Their Layered window demo shows a shaped window.

Their GL Layered demo shows a rotating cube.

And their D3D demo shows a rotating teapot.

If anybody can help me figure out how to adapt these to work with Allegro I would appreciate it.

beoran

Sorry to be not helpful, but I think both transparent and shaped windows would both make great features for Allegro itself. Did anyone start work on this before?

I notice SDL2 does have shaped windows support, (https://fossies.org/dox/SDL2-2.0.3/SDL__shape_8c_source.html#l00033) So we have some code to help us get started.

Edgar Reynaldo

I basically told you how to do it - what I don't know is how to make allegro drawing compatible with the layered window. I've got a working transparent window with blank areas that fades in and out over time. The stuff works. I just don't know how to make it compatible with allegro drawing functions.

SiegeLord, Elias and I were discussing this on IRC and it seems the way forward is to hack allegro to draw to a memory DC with a DIB selected into it, and create the rendering context with that. That way allegro will be drawing to the DIB instead of the window or its own backbuffer. Then the DIB is drawn to the HWND's HDC using BitBlt. So either replace the HWND, HDC, or HGLRC that allegro uses. This may use software rendering though, I haven't had a chance to get anything working yet. There is another option with OpenGL, to draw to an offscreen pbuffer and then draw that to the window.

The other possiblity is that there is some kind of format incompatiblity. I tried setting the colorkey to RGB(0,0,0) and drawing with al_map_rgb(0,0,0,0) and al_map_rgba(0,0,0,255), and neither matched the colorkey after being drawn to allegro's context, which is weird. Hence my idea that it is a format incompatiblity or else there is some pixel format conversion going on.

beoran

Hmm, this sounds complex compared to what is needed under X Windows. Definitely this definitely needs to be hardware accellerated somehow, so OpenGL seems the way forward. Also how could this work with DirectX?

Edgar Reynaldo

If you look at the third demo on the page I linked above you can see how to get it to work with D3D. I think all it takes is hacking allegro to draw to a custom context instead of the window, and then let the user blit that to the window.

beoran

Hmm, I looked at that example and the The DirectX code looks reasonable. The main complexity is that, as you say we have to draw to a secondary hardware buffer. At the moment I don't quite see how we can implement this elegantly in Zllegro, but the complexity seems low enough to make this a worth while feature.

Edgar Reynaldo

What would really be great is if I could figure out why drawing al_map_rgb(0,0,0) with an overwrite blender doesn't map to RGB(0,0,0) for the colorkey. I need to figure out how allegro is drawing to the HDC. Hmm maybe I could use GetPixel to check the values... Uhh maybe not - I'm getting back old values for some reason. Gotta investigate.

Edit
I was doing all kinds of stuff wrong. Heh.

I got multiple mouse overlays working and took a screenie for you :

{"name":"609462","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/1\/5\/155e1671c2dfd1ee5600bbda64a2a136.png","w":1280,"h":800,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/1\/5\/155e1671c2dfd1ee5600bbda64a2a136"}609462

Link to image for full view ( https://www.allegro.cc/files/attachment/609462 )

Here's a little demo package you can try (win32 only, no src for now, sorry)
Demo1.7z
Demo1.tar.gz

Add as many mice as you can find and run the program. As soon as one activates in any way it will show up on the desktop. Your system mouse is still there and fully functional, don't worry. Each one corresponds to its own attached mouse device. They all still have control over the system cursor as a group as well though. They don't click through to windows beneath them yet, I'm working on that now. ESC to quit.

Note: I'm only using a primitive ARGB_8888 Lock on allegro bitmaps to use SetPixel to write to a DIB, which I BitBlt to the window HDC during WM_PAINT. I haven't got the nice opengl direct rendering setup yet.

beoran

Well it's quite cool already. I hope you get HW accelleration to work, then we can start looking at how to integrate this with Allegro.

Thread #615412. Printed from Allegro.cc