Automated deployments: What are the components of a system?

I've been thinking a lot recently about software deployments, particularly in my current project, which is a (relatively) standard Java EE application: application server, database, messaging framework, blah blah blah. As I'm sure a lot of projects do, we have a hotch-potch of assorted build scripts, deployment scripts, semi-automated configuration scripts.

Two or three members of the project team try to manage deployments to test systems, and have currently managed to write several thousand lines of code which act as a universal way of deploying the artifacts to the various environments which ... doesn't work. These issues seem to crop up on every project - certainly every developer I've worked with recounts stories of projects where weeks of time were lost trying to figure out why code that apparently worked perfectly would not even deploy to a target system.

The upshot of this was that I started trying to think about the process in a more modular fashion; to break it down into smaller components and see if there's a standard pattern that we could apply. And I think I've come up with a few things - some of which are obvious, but we may as well start with the obvious and see where it leads us.

The System
To create this mythical working system we need three things:

• Environment
• Build artifacts
• Deployment processes

Environment: the environment is the actual machines (either physical or virtual) that the software will run on. It includes all third party additions required to run the software, such as databases, web / application servers, messaging systems and so on. It also includes all necessary network configurations such as load balancers, firewalls and host configurations.

Build artifacts: the build artifacts are the final output of building the source code ready for deployment. They may be tailored to fit a specific environment by setting runtime variables, but other than that the source code itself should not need to know about environment-specific details.

Deployment processes: the deployment processes are the description of steps needed to take the build artifacts, transfer them to the relevant environments, and initialise them such that the system is available for use.

These three aspects of the system can - and should - be treated separately when it comes to automating the overall process of creating and deploying environments. For instance, we should not be trying to write code in our build artifacts that generates application servers that are then deployed alongside the application files; they are part and parcel of the environment, and that should be as static as possible for the lifetime of the system. Also, the deployment process should not be part of building the artifacts from the source code: it should be possible to build individual artifacts and deploy them as necessary.

All these principles stem from the idea of the separation of concerns. Our build process should not interfere with a deployment; our environments should not define the way our software is built from source code.

Testing times

I'm very excited about the upcoming Dev/Test Lab feature that's coming to preview soon in Azure.

A lot of time working in an agile team is spent working on testing. Even with a dedicated QA engineer, a moderate-major portion of any development work is spent writing tests, running tests, and just flat out checking that the stuff you've written works.

It's not always easy to do that just on your local workstation, even if you're lucky enough to have a stupidly powerful machine at your disposal (spoiler alert for employers: this really does make a difference :) )

On the other hand, you can't really have a bunch of test machines, with all the associated maintenance and other costs, available for the development team to use on a whim, or because the QA team are running extended regression tests on an earlier build.

Dev/Test Lab might just help with that.

There are a bunch of tools that can be used with public or private clouds to simplify the provisioning and deployment of virtual machines and - given that many production environments now run on similarly virtualised equipment - this is a very reasonable way to run your test environments. What really has my interest in this offering from Azure is the tooling around it - making it easy to bring up and shut down systems, as well as manage a team's budget. (Sadly, this is reality, and your managers don't tend to appreciate you racking up $50,000 in VM fees because you forgot to shut down the load test system before going on holiday).

So yeah - I'm very much looking forward to kicking the tires on the preview. I might even find time to write about it.

Building sandcastles part 4: Azure

Continuing the adventures in continuous integration, I'm now looking into how services like AWS or Azure could really help dev teams along. A lot has been written about this before now, but while there seem to be lots of places talking about how easy AWS makes their dev / test / deploy cycle, I haven't seen too many places talking about all the components you might need.

So, looking at it from the ground up, we need, in the simplest terms:

• Somewhere to store our code
• Somewhere to build our code
• Somewhere to test our builds and artifacts
• Somewhere to host our final builds

This translates to:

• Some kind of SCM system
• Some kind of continuous integration system
• A private set of machines to deploy our applications to (let's assume that we're building a web application, so we'll need webserver and a database, minimum)
• A public set of machines to deploy the applications to

After mucking around with the trials of Azure and AWS, I decided to run a few experiments with Azure, and have ended up - so far - with meeting the first two requirements, with virtual machines running GitLab and TeamCity respectively.
I created a rather simple web application - no database yet, and pushed it up to GitLab, and then connected TeamCity over to that to run a couple of builds.

So far, so easy.

The next step - which feels like a pretty big one - is to build a test system. Now, it's very easy to set up a virtual machine, set up a webserver and then have a script to deploy the build artifacts from TeamCity over to it, but in an ideal world - where it's not just me working on it - it would be ncie to be a lot more flexible than that. What I really want to achieve is to be able to deploy a whole test environment with the click of a button.

That means that we need to start looking at the automated provisioning of virtual machines, because the last thing that anyone wants to do is set up a whole environment every time they want a build. Happily, there are a bunch of tools that can help us with this.

For a first experiment, I'm using Vagrant. It seems to be generally straightforward, and - in theory - relatively portable. It's been somewhat of a pain to get working with Azure, but now that it's running, I seem to be able to deploy machines relatively easily. Next steps are to actually get servers running on them.

Building sandcastles, part 3: Scala

(aka: And Now For Something Completely Different)

So, this is less about the continued experimentation in integration a bunch of SCM and CI systems, and more about random experimentation, because sometimes I just roll that way ;)

Pretty much on a whim, I brought Scala into the mix of technologies used. Following the recommendations of a couple of folk I've spoken to about this, I brought it in to the unit testing layer first, so that I can give it a go without too much impact on the actual written code. To be honest, it felt like a bit of a slog (though, having said that, I did manage to get the first unit test written and passing within an hour, after a beer, so I guess it can't be that hard).

First of all, we had to install Scala and the various IntelliJ plugins. That wasn't so bad. Second was to research how to run a JUnit test in Scala. Again, not too bad - suggestions are to use scalatest, and then have your unit tests extend the JUnitSuite class. Writing the unit test? No worries. Well ... not for the real simple ones, anyway. Running the test? Now that was the trick. First of all, you need to make sure that the version of scalatest you are using is compatible with the version of scala that you're using. Then you have to make sure that all the scala setup you did in the IDE also matches the version of scala you installed. Finally, you might need to tweak the build steps in the pom.xml file. Sadly, very few of the errors that get thrown when you have these things wrong make much sense.

For future reference (for myself, but maybe someone else will find this useful as well), this build configuration in maven seems to work quite well:

    <build>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <version>2.15.2</version>
                <executions>
                    <execution>
                        <id>scala-compile</id>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                        <configuration>
                            <args>
                                <arg>-dependencyfile</arg>
                                <arg>${project.build.directory}/.scala_dependencies</arg>
                            </args>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Something to be aware of is that the -make:transitive configuration argument seems to be redundant as of scala 2.11, and actually breaks the compilation. I've seen it mentioned in a couple of places, but taking it out made everything magically work for me... YMMV.

So anyway, I now have a very very simple test committed, which is currently passing. We'll see how things go as I try to rack up the complexity, and start on mocking out various interfaces.

Building sandcastles part 2

You know, that went pretty well, all things considered.

So, there are a couple of repositories up on GitHub now:
https://github.com/nihilogist/DiceEngine
https://github.com/nihilogist/PsychicWight

I managed to set up TeamCity without too many issues, and have that observing the two repositories.
It's also observing a given set of branches of the two repositories, so that a group of remote branches can also be built, and the results tracked.

I don't have a nexus installation running properly yet - that might be part of a larger experiment, but I can certainly see the benefits of the GitFlow workflow, where feature branches / release branches / etc. are the order of the day.

The next part of the project really should be to get the war file for the PsychicWight project up and running, and get some kind of automated / semi-automated deployment going.

But given that this took me just one day to set up, I think that it's certainly a viable approach to consider at the start of a project.

Hopefully there'll be time to revisit this next week.

Building sandcastles

(or: how do you practice at being an architect?)

Figuring out a different way of running a software project is kinda hard. Usually, you come into a project when everything is already set up and running, and you either go along with it, try to tweak it a little bit, or get the hell out. Starting from scratch isn't really one of the options.
Whilst I have opinions on some of the options, I don't really have a lot of experience at the way the different systems interact. I think that the best way of trying to sort this out is probably experience, but a problem with that is that building a large system with many interacting components is a hell of a lot of work.
So what I'm trying out is a small test project, with as many interacting components as I can, as well as a series of hoops to jump through that would usually only be present on a much larger project.

Project Sandcastle:

Going back to my absolute standard project, I'm going to build a dice roller. This time it's going to be a simple web based application, but I'm going to try and compartmentalise it as well.
In terms of infrastructure, I'm aiming for:

• an SCM system (git repository hosted on GitHub)
• a CI system (TeamCity, running locally)
• an artifact storage system (haven't decided yet, probably Nexus)

In terms of projects:

• a dice rolling engine (packaged as a jar file)
• a web application to interface with the engine (packaged as a war file, possibly with a built in server)

It's about commit(ment)s

So, I've managed a few commits to the Github project already then. Not that I have a great deal to write home about, but it's actually nice and easy to get the hang of. And the nice folks there have produced a great little Windows app to help skin over the Git interface: takes a lot of the pain out of standard tasks, though I haven't really had the pleasure (?) of dealing with multiple branches and revisions yet, so I'll reserve judgement till then.

Still in very early days of implementation yet, but there's a very basic dicebag there now, with an API that is (hopefully) easy to follow. Should also be a simple case to check it out, build it as a maven artefact and then chuck it into a project. Honestly, there are loads more updates to come. Honest.