Mongrel and Docker: the power of containers

Last time we looked at how we started off the Mongrel2 webserver in a docker container. It was a very simple setup, with a single container running an instance of Mongrel with a few bits and pieces of static content.

This time, we're going to look at what makes Mongrel so interesting, and why I think that Docker suits it perfectly as a deployment mechanism. We'll shoot over the basics of handlers, and I'll summarise the handler that I created, and how Docker

Handlers

Mongrel2 doesn't deploy applications in the same way that, say, servlet containers do, and it doesn't do any processing of code itself in the way that PHP applications might. Instead, it has a construct called a handler. These are specific paths defined in the server configuration that, when requested, construct a message for the ZeroMQ message framework, and pass them to a socket. A dedicated application reads the message from that socket, takes any necessary action, and then responds to the Mongrel2 server by placing a ZeroMQ message back to a new queue.

Handler application: "thought for the day"

In this case, I've only constructed one handler - an incredibly simple one, that could have easily been managed other ways, but we'll test the water slowly. It's a simple "thought for the day" generator, that will return a json object containing a quotation, and a source for that quotation.

The code for this handler isn't checked into Github yet, but is very simple. There's a single, looping process that waits for messages, and returns one of a random set of quotations whenever it receives a message. It's just a little jar file that gets executed and stays up until terminated.

Accessing the handler

Accessing the handler from the frontend is relatively easy: I've wired up a simple AngularJS controller that just grabs the json object from the /thought path, and plugs it into some html code on the front page. Nothing too fancy.

Putting it all together

So now we have an infrastructure that looks like this:

Mongrel2 and the handler process are both running in docker containers. The communication ports for the two of them are exposed within the docker engine, but not outside of it. Mongrel's main access point (port 6767, in this case) is mapped to port 80 on the virtual machine and exposed to the outside world.

However, we're still not quite done yet. To make things even easier, we can use Docker Compose to describe this entire diagram, and suddenly we're able to build, deploy and start all of our containers from a single command. Again, we're not at a particularly complex level yet, this single file

mongrel2:
	build: ./mongrel2-main
	ports:
	- "80:6767"
	expose:
	- "5557"
	- "5558"
	samplehandler:
	build: ./sample-handler
	links:
	- mongrel2

will use the information in the two dockerfiles to build — from scratch — the entire application above and deploy it.

And that is an amazing tool, which will give us the ability to add sections to our infrastructure quickly and easily.

Automated deployments: What are the components of a system?

I've been thinking a lot recently about software deployments, particularly in my current project, which is a (relatively) standard Java EE application: application server, database, messaging framework, blah blah blah. As I'm sure a lot of projects do, we have a hotch-potch of assorted build scripts, deployment scripts, semi-automated configuration scripts.

Two or three members of the project team try to manage deployments to test systems, and have currently managed to write several thousand lines of code which act as a universal way of deploying the artifacts to the various environments which ... doesn't work. These issues seem to crop up on every project - certainly every developer I've worked with recounts stories of projects where weeks of time were lost trying to figure out why code that apparently worked perfectly would not even deploy to a target system.

The upshot of this was that I started trying to think about the process in a more modular fashion; to break it down into smaller components and see if there's a standard pattern that we could apply. And I think I've come up with a few things - some of which are obvious, but we may as well start with the obvious and see where it leads us.

The System
To create this mythical working system we need three things:

• Environment
• Build artifacts
• Deployment processes

Environment: the environment is the actual machines (either physical or virtual) that the software will run on. It includes all third party additions required to run the software, such as databases, web / application servers, messaging systems and so on. It also includes all necessary network configurations such as load balancers, firewalls and host configurations.

Build artifacts: the build artifacts are the final output of building the source code ready for deployment. They may be tailored to fit a specific environment by setting runtime variables, but other than that the source code itself should not need to know about environment-specific details.

Deployment processes: the deployment processes are the description of steps needed to take the build artifacts, transfer them to the relevant environments, and initialise them such that the system is available for use.

These three aspects of the system can - and should - be treated separately when it comes to automating the overall process of creating and deploying environments. For instance, we should not be trying to write code in our build artifacts that generates application servers that are then deployed alongside the application files; they are part and parcel of the environment, and that should be as static as possible for the lifetime of the system. Also, the deployment process should not be part of building the artifacts from the source code: it should be possible to build individual artifacts and deploy them as necessary.

All these principles stem from the idea of the separation of concerns. Our build process should not interfere with a deployment; our environments should not define the way our software is built from source code.

Building sandcastles part 2

You know, that went pretty well, all things considered.

So, there are a couple of repositories up on GitHub now:
https://github.com/nihilogist/DiceEngine
https://github.com/nihilogist/PsychicWight

I managed to set up TeamCity without too many issues, and have that observing the two repositories.
It's also observing a given set of branches of the two repositories, so that a group of remote branches can also be built, and the results tracked.

I don't have a nexus installation running properly yet - that might be part of a larger experiment, but I can certainly see the benefits of the GitFlow workflow, where feature branches / release branches / etc. are the order of the day.

The next part of the project really should be to get the war file for the PsychicWight project up and running, and get some kind of automated / semi-automated deployment going.

But given that this took me just one day to set up, I think that it's certainly a viable approach to consider at the start of a project.

Hopefully there'll be time to revisit this next week.

Two lines of code can change your life

I've written about this before (and I'll most likely write about it again), so apologies in advance if I bore people to death with it. It's the issue of education and computing. There have been a few reports recently indicating that computing courses in the UK are failing their students, the worst examples being some of the specialist games development courses: just 12% of graduates having found a job in recent times.

I write code for a living, and I enjoy my job. I'm very lucky to be doing something that I really want to do, and I think that I'm doubly lucky because it's taken me a long and improbable route to get here. And fundamentally, that route started when I was about seven or eight, and I first wrote the lines:

10 PRINT "My name is David"
20 GOTO 10
RUN

Suddenly, my television screen was filled with an endlessly repeating filler of My name is David. I'd managed to get a computer to do something that I had told it to. I was hooked. Over the next few years I would often be found hunched over the beige keyboard of my trusty Acorn Electron, typing in programs and trying to get them to work. By the time I was fourteen, just typing in other people's code wasn't enough, and I began to try and write my own from scratch, starting an epic quest to produce a text adventure game.

I don't know what it was that really spurred me on, whether it was getting to hang around with some seriously talented young developers (I kid you not, these guys had clubbed together and written an email application for the school network while I was still working on drawing a circle on the screen), or that I was very much supported by the school I was in, but despite a long time not working in the IT industry, I came back to college, and picked up Java again. And it was almost like I'd never left.

The thing is, when I was younger, it seemed so much easier to start programming. The languages were so much closer to you - heck, the Electron started up with a BASIC prompt. I can well see someone these days enrolling on a programming course and being slightly taken aback when presented with their first view of an IDE or a command line.

Now, I'm not living in the UK at the moment, so I guess I'm not really qualified to talk about the way that ICT is taught there, and I'm not well enough acquainted with the details of ICT eduction in Ireland, either. But the reports that more and more computing lessons are heading towards teaching basic office skills are a little worrying. It's not that people don't need the ability to use the word processor / spreadsheet / office suite du jour, but it would be great to see other classes available for people who really do want to stretch themselves. There are plenty of resources on the web these days - with many kids it might just take a small push in the right direction, and boom - we have another killer app developer in the running.

It sounds like we might be moving in the right direction at last though, with a few new initiatives kicking around to try and get children interested in the reality of computer science from a younger age, and I really do believe that one of the ways to do that is to show them how easy it is to get started. Show how quickly you can put together a web page. Write a little java program to print stuff out on screen or a simple game. Try and get hold of some of the old computer controlled Lego Technic, or - better yet - the new Mindstorms NXT. Given the choice between working on a spreadsheet and teaching a robot to walk? I know which one I'd choose.

People keep talking about the knowledge economy. We've got to make sure that we create a generation who are going to be up to creating that knowledge, not just using the fruits of it.

Remember, sometimes all it takes to spark an interest is two lines of code. (Though next time, please don't use GOTO. Bad habits and all that ;-) )

An ode to the Acorn Electron, or: Thoughts on Computing in Education

*beep*

Acorn Electron

BASIC

>

Ahh, nostalgia. I think my folks still have the first computer I ever used somewhere, up in the attic. An Acorn Electron, with a whole thirty-two kilobytes of RAM, and no disk drives - you had to plug in a temperamental tape deck and wait fifteen agonising minutes loading a game before you found out that you'd set something up wrong and you'd have to start all over again. But I loved it. And nearly as much as the games (Citadel! Palace of Magic! Frenzy!), I loved the idea that you could write your own programs for it, too. I spent hours trying to write my own versions of the games that I played, by typing in programs that I found in magazines, and then spending just as many hours trying to figure out which of the semi-colons or quotation marks that I'd typed wrongly.

I sometimes stop to think about how I would have turned out born just a few years later. The Electron, by booting straight into this (admittedly crude) programming environment, really made the point that you were supposed to try and do your own thing with the computer rather than just accepting what other people had written.

I don't know a great deal about the current state of computing in education, but it's something that I should find out more about. I remember that IT lessons were adventures in controlling a turtle onscreen, or (if we were lucky) controlling actual Lego connected to the serial port. It really opened your eyes to the possibilities of computers as tools. I'd like to think that there are some quick and easy programming languages for kids still out there. Whatever happens in the future, we can't just have computing lessons being restricted to learning how to use the office software suite du jour. What a way to kill any enthusiasm.

There's no shortage of resources for programming education. And the thing is, many of them are free: look at Greenfoot, for Java. It hides a lot of the complexity of the language: but that's a great way to start. It's got a graphical interface: that's great.

The problem is, of course, that to teach Java, you need to know some Java in the first place. And how do we get that kind of knowledge imparted? How many teachers are there in schools with any kind of programming knowledge? Not nearly enough. Following on from that, these programs also need to be installed, set up, and maintained. Who's going to take care of that? Much easier to just teach kids how to use the internet, a word processor and a spreadsheet.

Perhaps a special kind of week-long code camp could be run for children with any kind of interest in programming: do a week of classes in the summer holidays covering the very basics - and give pointers on places to look for further information. It might whet someone's interest, and - who knows - might produce the programmers of tomorrow.