Some of you may recall discussions about Allegro Planet offering automated builds of your Allegro games. I wanted to share my very promising progress on the project so far.
Support for project-specific GitHub webhooks has been developed and is in Allegro Planet. Webhook calls from GitHub are collected in Allegro Planet on a per-project basis. (These will be used eventually to trigger builds).
Click here for an image demonstrating how you would add a webhook to connect your Allegro Planet Game <-> GitHub Repo.
An experimental RemoteGithubRepoBuilder demonstrates the proof-of-concept of running a build process.
Move the built file from the tmp/ directory into a publically accessible location.
Create a Build class that keeps status of an ongoing build, and notifies the user/subscriber of build events.
Add a "watch build" type of page that allows users to follow the progress (and possible error messages) of their builds.
Move work done on RemoteGithubRepoBuilder into a background job. Have it create a Build object and update that object during during build process.
Investigate Rails' ActionCable to notify subscribers.
Attach new builds to a project (ensure no filename collisions), and add a download button to the Game's project page 🎉
Early versions will likely have very limited build options and may impose restrictions on your directory structure and Makefile. Since Makefiles can arbitrarily run any commands on the system, there will need to be safeguards to ensure that the system remains secure. This may include a requirement for container builds, or might introduce an optional .allegro-planet file in your repo that include a limited scope of build settings for your project.
Project build dependencies will likely need to be included in the project's .allegro-planet file and may be restricted to allegro and allegro_flare for the time being.
Early versions will likely only support Linux builds with the clang/gcc compiler until cross-compilation can be solved.
Remote build is an advanced feature, Allegro Planet still doesn't have user accounts built.
Remote builds initially adds support for GitHub webhooks, but a more open-ended build system allowing cloning repos from arbitrary locations could be pretty cool, too.
Builds should ideally be handled completely on the Allegro Planet (or docker on the Allegro Planet) server. It's possible there will be some pressure to have builds run on remote systems and I would like to avoid that if possible.
GitHub allows the remote server to post ongoing status directly back to your repo. This feature would reduce build<->check iteration time substantially but would require users to generate a token and/or integration permissions on a repo and would add an additional time dependency on development.
Cross-platform building is still unexplored and a bit of an unknown. Ideally, the system could cross-compile binaries with the llvm compiler and not rely on any third party OSs to produce the builds. This also means that there are dlls and other dependencies needed for distribution, and that needs to be accounted for as well. None of this has been explored with much depth yet.
For my bot in #allegro which allows sending it arbitrary commands I have a Virtual Box instance to execute the binaries. It's very easy:
vboxmanage controlvm botvm poweroff vboxmanage snapshot botvm restore botsnapshot vboxmanage startvm botvm --type headless
(botvm is the name of my VM, botsnapshot is the name of a snapshot with the initial state I want it to be in before running the arbitrary command)
Then I run the binary over ssh into that VM. There are solutions which are faster (but the above seems plenty fast, the bot still replies in less than a second) and more secure (there are ways to break out of virtual box). But just saying, it should not be very hard. And with Virtual Box you can also choose to run commands inside of Windows or OSX
]]>If you're using docker, then you could just package up the dependencies in docker images, like I've done here It could potentially be also used to compile for android, which would be awesome!
]]>I personally find LXC containers even more useful and flexible than docker. Only downside is that it's only for Linux in Linux containers.
]]>If you're using docker, then you could just package up the dependencies in docker images, like I've done here It could potentially be also used to compile for android, which would be awesome!
This is amazing. I need to learn more about how this works because I'm not familiar with it. Do you know if it is possible to build windows binaries in a linux docker?
]]>Docker is a huge pain :p
]]>Elias and I disagree about this
I use docker for everything now - my build server, my blog and the TINS website all run on docker now. And on the build server most of the build jobs run in a separate docker container as well. It's the perfect way to keep dependency chains isolated and avoid conflicts. Docker ftw!
Theoretically it would be possible to set up a docker build environment for cross compilation, but I haven't tried this yet. I still do my windows builds on a fully fledged windows VM.
Remember a docker environment is linux (even if it's supported on Win10 and OS X with docker machine, that's just a thinly veiled VM running linux).
]]>And on the build server most of the build jobs run in a separate docker container as well.
What's the configuration of your build server? Are you on a cloud-hosted service?
I'm on Heroku right now, but it doesn't look like there is support to setup some external packages with apt-get for example. Not without an experimental feature, it would seem.
Remember a docker environment is linux
Would I essentially still need to setup a complex cross-compile environment? E.g. no native dockers for emulating other OSs?
]]>What's the configuration of your build server? Are you on a cloud-hosted service?
The build server is an old laptop running under my desk. It uses Jenkins to define and schedule the jobs (you can set it up to track git commits).
My website is hosted on a VPS, which basically gives you a complete linux VM to play with. I can install anything, the sky is the limit (well, as long as it fits in 20G disk space and 1G mem). My provider is Cloudvps, a dutch provider.
Would I essentially still need to setup a complex cross-compile environment? E.g. no native dockers for emulating other OSs?
That is correct. The advantage of using docker, though, is that you can separate the maintenance of the build environment from the maintenance of the server, and you can track the maintenance in git.
I think cross-compilation for windows targets should be possible, but I haven't gotten it working yet myself.