After reading the Wolfire Blog, I was inspired to incorporate WebKit into my Elite-like 3D game, for use as a UI renderer. WebKit is the HTML rendering engine behind Safari and Google Chrome, and is touted as being fast and bleeding edge. Wolfire displayed beautiful results of its usage as a UI platform in their game Overgrowth.
So, I downloaded Awesomium (WebKit wrapper) and AwesomiumDotNet, and had Allegro.cc rendering in my XNA game in no time:
{"name":"601991","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/1\/1\/11e672c43eedd923beb02d494760e269.png","w":1040,"h":806,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/1\/1\/11e672c43eedd923beb02d494760e269"}
It was really easy to use. Literally took minutes to get going once I found what I needed. It still won't load a local HTML file, but I'll work on that.
Why did I do this? WebKit Primer Part 2: CSS transitions. That's why. I doubt I have the graphical skill to make my UI look half as good as that, but it's worth a try.
The library is C++ based, and spits out raw BGRA32 data, so it should be a breeze to get going in an Allegro game.
EDIT: Fixed loading local files. Had to append an extra backslash:
string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().GetModules()[0].FullyQualifiedName); myWebCore.BaseDirectory = path + "\\";
EDIT: And thanks to my lovely DevConsole powered by IronPython, I'm able to fiddle with the HTML in-game. I'll likely replace it with the WebKit console or something, but still awesome
EDIT: I've attached all the keyboard and mouse events to WebKit. Took a little work, because I wanted to grab them right from the Window's events, instead of XNA, so I had to go find a hack. But, everything seems to be working just fine. In fact, I'm writing this update from within my game Only problem is the cursor doesn't change. Hmmm ....
Interesting kit, indeed !
I certainly find this to be an interesting use for WebKit. Are you planning on using it for the entire GUI, or just parts?
This is very interesting. I'm definitely going to check this out for my game next year. It seems a wonderful alternative to cleaning up my four year old GUI library.
How is the speed? Any particular quirks you've noticed about setup or rendering?
Are you planning on using it for the entire GUI, or just parts?
Hopefully the entire GUI. I am concerned that it won't be able to handle 3D labels, though. For example in my game I'd like to have planets within the view labeled by projecting their 3D coordinates to 2D screen coordinates. I don't know if WebKit can update fast enough for that.
Any particular quirks you've noticed about setup or rendering?
My biggest issue right now is it doesn't seem to provide any way to return results to JavaScript in a callback. When JavaScript calls into my engine, Awesomium raises an event. My engine can pull in all the function arguments and such, but there's no way to return a value! I can't find Awesomium's source code, so I'm poking around to see how difficult it would be to hook up to WebKit directly ...
EDIT:
Looks like the reason you can't return results is because the callbacks are queued up as events. From JavaScript's perspective they are asynchronous. I guess that solves any threading issues, but it also means callbacks can't return values. Bah-hum-bug. The only seems to be to not use such a design pattern, and instead have the engine update properties appropriately.
I've been digging into this since I last posted. Awesomium went closed source just before the release of 1.5, which is why you can't find it. The source from when it was still lgpl is at git://github.com/pathorn/awesomium.git.
Just after it went closed source. Sirikata started their own version (they had been using Awesomium up until then) called Berkelium. It is still in beta and I have ruled it out because it is a strictly multiprocess design (As in, creating a window in game would literally start up a chromium browser as a sub process. IPC, from what I've seen of it, is too slow/bulky). Berkelium also has no way of getting javascript values.
Anyway, the closed source version of Awesomium doesn't support Linux (yet). Its creator said he'd have Linux support in around July, but that has clearly passed. People have been asking but no word, so my faith is shaky (and my game engine starts getting officially built next month (by which I mean should be mostly feature complete by), so no use waiting).
As a result of all this, I grabbed the most recent version of Awesomium source (from around December of 09. It is not and will not be maintained, according to the Awesomium author) and have been working with that. Looking at the V8 source in Chromium, it seems the V8 engine SHOULD be very easy to bind to C++. The Awesomium source I have already supports getting some values out of JavaScript (just not objects, like the most recent version does.).
So, anyway, I'll be working on getting much better binding into the library and bringing it up to speed (I figure there is no reason to have Lua in an engine if I'm going to be including JavaScript as a dependency anyway).
Let me know of anything else you find code wise. I'm probably going to try hooking directly into V8 and thus subvert the queue where I can, but all information is still helpful when working with REALLY BIG codebases I've never seen before.
Awesomium went closed source just before the release of 1.5, which is why you can't find it. The source from when it was still lgpl is at git://github.com/pathorn/awesomium.git.
That explains it! I had indeed found the repo, but wasn't sure why it was the only one to be found.
The relevant piece of code I found was in ClientObject.cpp:
void NamedCallback::handleCallback(const CppArgumentList& args, CppVariant* result) { Awesomium::JSArguments jsArgs; initFromCppArgumentList(args, jsArgs); Awesomium::WebCore::Get().queueEvent(new WebViewEvents::InvokeCallback(view, name, jsArgs)); result->SetNull(); }
WebKit's JS engine calls that when JavaScript code tries to call one of the registered callbacks. i.e. x = Client.GetSomethingImportant();. As you can see, it does not immediately call the Listener's onCallback. Instead, it queues an event. So, from the perspective of JavaScript, the callback returns immediately and doesn't actually execute.
The solution is simple. Rewrite the above function to immediately call the desired engine function, and place the return value into result. The danger is, though, that your engine function will likely be called from a different thread, so now you have to handle multi-threaded locking and such.
I'm a bit torn. Part of me wants to work on this, because I think it would be useful to me in the long-term, and perhaps useful for Allegro 5 UI's. On the other hand, this is a lot of work, and completely halts game-play related work in my game. Hmmm ...
EDIT: Looks like the Awesomium code from that repo is LPGL'd, so we are free to modify it. Might be worth trying to update it a bit to work like the closed-source version, and to support synchronous callbacks.
From my perspective, making GUI libraries is incredibly boring and horribly unsatisfying (there is never time to get all the features you may need) and, in general, GUI's sort of get pushed off to the side. Making a pre-existing HTML engine work in game, on the other hand, is fun, challenging, feature complete (particularly with those css3 gems), and opens up the possibility of JavaScript in game (which is nice if only because it has a debugger, profiler, can be tested externally, and has a very wide user base).
Not to mention I've got nothing to do but Engine work until school starts and whatever ideas the rest of the team have been cooking up need to be fleshed out.
Although, anything that is a lot of work is extremely risky to a development cycle. Far too many projects have fallen off the back burner while some big new feature was being worked on. Just my $0.2
Anyway, looking a bit more through the Chromium repo and the Awesomium source, it seems the ability is not only already there to bind a class to JavaScript, it's documented with examples!
http://src.chromium.org/viewvc/chrome/trunk/src/webkit/glue/cpp_binding_example.h?view=markup
The entire glue parent directory is mostly the focus of interest. This may not be as overbearing as originally foreseen.
opens up the possibility of JavaScript in game (which is nice if only because it has a debugger, profiler, can be tested externally, and has a very wide user base).
That's basically why I wanted to use WebKit as my UI. CodeMirror is a great example. It's an in-browser code editor. A game GUI based on WebKit can literally drop CodeMirror right in and have an in-game code editor. A few more hours of work and you could easily have live scripting updates for your engine.
You should also be able to use the many DOM inspectors out there to do live inspection on the UI, and live updates there as well. Plus, as you mentioned, you can just test all the UI in a regular browser (minus engine-specific calls).
Although, anything that is a lot of work is extremely risky to a development cycle.
To really take advantage of WebKit as a UI, you will need a windowing system built around the WebViews. That's probably the hardest part, in my opinion. Besides the JavaScript issues outlined in the previous posts, the need to design a Windowing system is the other complication keeping me away from using WebKit immediately.
Meh, I'll probably get back to it once I've made more gameplay progress.
EDIT: Here's CodeMirror running in-game:
{"name":"602031","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/9\/3\/938cd1e920071258878a7af5e474b488.png","w":1040,"h":806,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/9\/3\/938cd1e920071258878a7af5e474b488"}
Pretty neat stuff. Slap a file browser on it, and a button to do a "live-update" and I've got myself a handy development platform inside the game.
Pretty cool stuff, for sure. I've been working at it, but so far the whole day has gone toward just getting a working chromium build. (I had one, but I want a beta so it's a bit more stable). I keep running out of diskspace during checkout and build, or swapspace during linking. A bunch of minor problems, but when each little step takes hours, the day just disappears.
Everything should be worked out though so I should have a nice build ready tomorrow morning.
Personally, I was looking for using WebKit more for only UI related things. I'm not sure WebViews would be an entirely useful feature from my standpoint.
Personally, I was looking for using WebKit more for only UI related things. I'm not sure WebViews would be an entirely useful feature from my standpoint.
From the perspective of Awesomium, you create two things, a WebCore object, and a WebView. The WebCore is just a manager of sorts. A WebView object is the actual rendering entity. It's kind of like a tab in Chrome. So if you want to use WebKit at all, you create at least one WebView. For very simple UIs you could just create a single WebView that fills the entire screen. In that case, your UI would be a single HTML file that gets loaded into that WebView. I want to be able to separate things into windows that can be moved around, so I will personally manage multiple WebViews (one per window).
I imagine you could actually build a window manager in HTML+JavaScript, with each window being an iframe. In that case, a single WebView would suffice.
Using HTML+CSS for a GUI sounds like a really good idea. In practice, I think that both could be replaced with equivalents better designed with GUIs in mind (XML+some styling language uninfluenced by HTML). In fact, I think it's the basis behind Mozilla's XUL technology[1].
I think with my background in Web technologies it would be a lot easier for me to write a GUI using that than it would using something like Win32, GTK+, or QT (albeit, macros aside, I was quite pleased with QT when I was experimenting with it).
Show us teh codez that accomplish what you've done already!
I imagine you could actually build a window manager in HTML+JavaScript, with each window being an iframe.
You'd only need an iframe if you wanted each window to be a static .html file. You can instead just use a floating div and dynamically fill its contents. Assuming jQuery UI works with what you're doing, you can even get really pretty dialogs with almost no effort.
For example:
Note: for the above to work you will need to generate a jQuery UI theme (there are many to choose from and there's a UI to customize). Or just follow the link above to see it live on my site.
Show us teh codez that accomplish what you've done already!
I'm working in C#, but the interface to Awesomium is the same. It looks like a lot of code below, but that's only because it contains a really basic window manager:
The main thing is creating the WebCore, WebViews, passing them keyboard and mouse events, and updating the WebView textures when necessary.
In practice, I think that both could be replaced with equivalents better designed with GUIs in mind
That would certainly make writing GUIs easier, but the great thing about using WebKit is it allows you to drop in existing libraries and software like jQuery. Plus, WebKit is probably more optimized than anything I could ever write. Personally, I'd write something to translate from something like XUL into HTML+CSS. That way you get the best of both worlds.
EDIT:
Here's your UI running in my game, bamccaig.
{"name":"602057","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/2\/1\/21a3dd38df2b7de223ba539212f78b0a.png","w":1040,"h":806,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/2\/1\/21a3dd38df2b7de223ba539212f78b0a"}
{"name":"602056","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/e\/a\/ea4f468780aa03dd4d2f59e62d5e1217.png","w":1040,"h":806,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/e\/a\/ea4f468780aa03dd4d2f59e62d5e1217"}
I primarily write C# and JavaScript so I just wanted to knitpick on your code a little bit.
WebCoreOptions options = new WebCoreOptions(); options.LogLevel = LogLevel.Verbose; options.PixelFormat = PixelFormat.Bgra; WebCore = new WebCore(options);
this.WebCore = new WebCore(new WebCoreOptions { LogLevel = LogLevel.Verbose, PixelFormat = PixelFormat.Bgra });
WebCore.BaseDirectory = path + "\\Content\\UI\\";
this.WebCore.BaseDirectory = path + @"\Content\UI\";
WebKitWindow wnd = WindowUnderPoint(e.Location);
var wnd = WindowUnderPoint(e.Location);
Windows.RemoveAll(delegate(WebKitWindow wnd) { return wnd.Closed; });
this.Windows.RemoveAll(wnd => wnd.Closed);
Or maybe...
this.Windows.RemoveAll((WebKitWindow wnd) => wnd.Closed);
WebKitWindow wnd = new WebKitWindow(800, 600);
var wnd = new WebKitWindow(800, 600);
No guarantee that all of that applies.
Here's your UI running in my game, bamccaig.
Cool. ...Wait, how do you know my password?!
I primarily write C# and JavaScript so I just wanted to knitpick on your code a little bit.
Those were some helpful comments I know all of the syntax, but I guess I forget to use it most of the time I especially like being able to set member variables with the new operator. That is a darned handy feature for keeping compact code.
Wait, how do you know my password?!
That's a feature of WebKit