Building sandcastles, part 3: Scala

(aka: And Now For Something Completely Different)

So, this is less about the continued experimentation in integration a bunch of SCM and CI systems, and more about random experimentation, because sometimes I just roll that way ;)

Pretty much on a whim, I brought Scala into the mix of technologies used. Following the recommendations of a couple of folk I've spoken to about this, I brought it in to the unit testing layer first, so that I can give it a go without too much impact on the actual written code. To be honest, it felt like a bit of a slog (though, having said that, I did manage to get the first unit test written and passing within an hour, after a beer, so I guess it can't be that hard).

First of all, we had to install Scala and the various IntelliJ plugins. That wasn't so bad. Second was to research how to run a JUnit test in Scala. Again, not too bad - suggestions are to use scalatest, and then have your unit tests extend the JUnitSuite class. Writing the unit test? No worries. Well ... not for the real simple ones, anyway. Running the test? Now that was the trick. First of all, you need to make sure that the version of scalatest you are using is compatible with the version of scala that you're using. Then you have to make sure that all the scala setup you did in the IDE also matches the version of scala you installed. Finally, you might need to tweak the build steps in the pom.xml file. Sadly, very few of the errors that get thrown when you have these things wrong make much sense.

For future reference (for myself, but maybe someone else will find this useful as well), this build configuration in maven seems to work quite well:

    <build>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <version>2.15.2</version>
                <executions>
                    <execution>
                        <id>scala-compile</id>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                        <configuration>
                            <args>
                                <arg>-dependencyfile</arg>
                                <arg>${project.build.directory}/.scala_dependencies</arg>
                            </args>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Something to be aware of is that the -make:transitive configuration argument seems to be redundant as of scala 2.11, and actually breaks the compilation. I've seen it mentioned in a couple of places, but taking it out made everything magically work for me... YMMV.

So anyway, I now have a very very simple test committed, which is currently passing. We'll see how things go as I try to rack up the complexity, and start on mocking out various interfaces.

Building sandcastles part 2

You know, that went pretty well, all things considered.

So, there are a couple of repositories up on GitHub now:
https://github.com/nihilogist/DiceEngine
https://github.com/nihilogist/PsychicWight

I managed to set up TeamCity without too many issues, and have that observing the two repositories.
It's also observing a given set of branches of the two repositories, so that a group of remote branches can also be built, and the results tracked.

I don't have a nexus installation running properly yet - that might be part of a larger experiment, but I can certainly see the benefits of the GitFlow workflow, where feature branches / release branches / etc. are the order of the day.

The next part of the project really should be to get the war file for the PsychicWight project up and running, and get some kind of automated / semi-automated deployment going.

But given that this took me just one day to set up, I think that it's certainly a viable approach to consider at the start of a project.

Hopefully there'll be time to revisit this next week.

Building sandcastles

(or: how do you practice at being an architect?)

Figuring out a different way of running a software project is kinda hard. Usually, you come into a project when everything is already set up and running, and you either go along with it, try to tweak it a little bit, or get the hell out. Starting from scratch isn't really one of the options.
Whilst I have opinions on some of the options, I don't really have a lot of experience at the way the different systems interact. I think that the best way of trying to sort this out is probably experience, but a problem with that is that building a large system with many interacting components is a hell of a lot of work.
So what I'm trying out is a small test project, with as many interacting components as I can, as well as a series of hoops to jump through that would usually only be present on a much larger project.

Project Sandcastle:

Going back to my absolute standard project, I'm going to build a dice roller. This time it's going to be a simple web based application, but I'm going to try and compartmentalise it as well.
In terms of infrastructure, I'm aiming for:

• an SCM system (git repository hosted on GitHub)
• a CI system (TeamCity, running locally)
• an artifact storage system (haven't decided yet, probably Nexus)

In terms of projects:

• a dice rolling engine (packaged as a jar file)
• a web application to interface with the engine (packaged as a war file, possibly with a built in server)

Run-time composition of services: like a swan?

Web Services are a fairly amazing concept, when I think about it. You have a service, running on a server somewhere in the world, that accepts requests from other places and then returns information to them. This could be as simple as getting a request for the temperature in a particular city and returning it, or as complex as being asked to search a database for, say, hospitals near a given location, and return their details as an XML document.

Other people can then build onto that, combining different web services in various orders to get more and more detailed and crazy applications. This is what makes mobile development so exciting - there are so many possibilities open to creative designers that the sky isn't even the limit anymore.

What really begins to shake things up, though, is when there's more than one service offering the same results. There are likely to be a few factors separating them - one might cost more, yet one might be more reliable, and a third might return results more quickly.

So we start creating systems that look at the alternatives available when they query these services, in order to optimise the return of results - and that's not an easy task. And then we have to consider what happens when a particular service is unavailable - do we just wait and hope that it sorts itself out sooner rather than later, or do we try to recompose the rest of the service and attempt to get round the problem.

It's only as I dig deeper and deeper into the bowels of service composition that I realise how staggering a task it is - so many variables and so much frantic querying, estimating and processing, and all of this has to be hidden from the end user. It ends up like a swan: perfectly serene and stable on the surface, but flailing around like a mad thing underneath!

News at last

Finally there's a little bit of forward movement on the Masters course - we've got the dates for the seminars sorted. Of course, the first one is scheduled for one of the few days in the next few weeks where I have something on that cannot be cancelled or moved at all, but I guess that's life...

But at least there's stuff going on. In other news there, the collection of papers to do with SOA and the application of genetic algorithms thereto continues. There's been a bit more work done on this field than I'd anticipated, which was a little surprising, and it falls into two general streams.

One takes a composed service, made up of lots of interchangeable component services, and tries to work out the best combination of services given a specific set of requirements. This is pretty similar to work that I've done already in this area, so that's encouraging.

The second uses the genetic approach to try and combine services efficiently into a single component service. This looks fantastically interesting and useful, but it would mean learning a whole new mathematical language of combinatorial algebra, and then trying to code it into Java. Which is unlikely to take place over the course of a single, part-time year.

But hey - I'm meeting up with my supervisor soon enough, and we'll have a look through the papers to see if he thinks that the gaps I've spotted are (a) actually gaps at all and (b) big enough gaps for a research project.

Service composition

Electronic journals are both big and clever, it seems. It's also very useful to be working from within the library, and thus gaining instant access to said journals from my own laptop: technology is a marvellous thing!

I've started looking into service composition techniques now, and they're looking to be really quite a complex beastie, particularly when you begin to factor in dynamic service composition which, for my sins, is what I'm attempting to research. Obviously, when you have a group of x services that can e combined in a variety of orders to achieve a variety of results, searches to try and combine them are going to need to be heuristic to some degree: the mind-boggling number of combinations rapidly rules out a brute-force search for all but the smallest number of systems. However, the number of combinations and techniques available for a heuristic search is also beginning to hurt my brain ever so slightly.

Fortunately, it appears that not many people have looked into the method that I'd like to look at (genetic algorithms). Unfortunately, this means that I'll have to start this all from scratch!