Looking at my program in task manager (on Windows 7), I noticed that every time I toggle windowed fullscreen, the memory of the program jumps up significantly. It jumps up by around 27 mb each time I toggle to and from windowed fullscreen, and never goes back down.

If I switch in and out until the program uses more than 512 mb, it crashes! It only takes about 16 toggles before crashing

The only thing I do when toggling is call this:
al_toggle_display_flag(display, ALLEGRO_FULLSCREEN_WINDOW, fullscreen);

Could it be creating a new backbuffer bitmap and not destroying the old one?

EDIT: This happens in openGL mode, it doesn't seem to occur in direct3d. That being said I'm using openGL, so switching is not a solution.

I'm not able to solve this problem myself, but I know what kind of information the people who do work on Allegro will want to know:

1. What version of Allegro are you using?

2. Does the problem still occur in other versions of Allegro, including the latest 5.1.x version?

3. If you close out the video mode entirely using al_destroy_display() and create a new one, does the memory leak still happen?

OpenGL has something special called Framebuffer Objects (FBOs) that Allegro manages internally. It's possible something about calling al_toggle_display_flag() isn't properly handling these. Again though, I don't work on Allegro so I couldn't say for certain if it's related to this, but I know FBOs have been a source of problems in the past with the OpenGL side of things.

1. I'm using 5.0.10

2. I tried it in 5.1.7 as well, the problem persists. It also happens if ALLEGRO_OPENGL_3_0 is turned on.

3. Destroying and creating the display causes the problem to go away! However it makes the application run very slowly. I've heard I have to reload all of the graphics to put them into video memory, however before doing this I'd like to know if there's a better way.

You probably already know this, but just in case you don't, ALLEGRO_OPENGL_3_0 is just a modifier for ALLEGRO_OPENGL. You need ALLEGRO_OPENGL to actually use Opengl. without it you get D3D.

Yes, I meant opengl and opengl3 set on.

So is this a bug in allegro? Should I actually be destroying the display, or is there something else I can do?

It may be a bug in allegro. It may be keeping the temporary memory bitmaps around after attempting to reupload them on the resize (essentially a resize completely re-creates the context, which needs to reupload everything).

Definitely sounds like a bug.