Testing is dead.

Now that I've got you riled up a bit, let me explain what I mean, and why you already love Continuous Integration (CI), even if you don't know exactly what it is yet.

The old way of developing and testing software was to write the code, launch it, and then go through as a user, clicking around, and looking to see if anything was broken. If everything seemed okay, it would be pushed into production where, again, people would click around a bit more to look for problems. This was "good enough" in that when a new feature is being built, it would get the most attention -- any bugs in that particular feature would probably get caught.

There are two problems here. One is the obvious case of being a poor use of a developers' time. Developers are expensive, and having to test by hand costs a lot of time. Every time a new feature is deployed, the entire system needs to be checked. The simple way around this is to only test the new feature -- since it's new, that's where bugs are probably going to show up...right?

Unfortunately, no. In any modern software system, there are huge chunks of code that get used for different features throughout the application -- even things that have no obvious relationship can be intertwined. Making a change to a piece of code to support one new feature can have unknown side effects for the other features where it gets used. You can tell this is the case, because any development team that relies on this hand-checking method will eventually start hearing things like "Well yeah, the new thing is working okay, but this other feature that's been working fine all along is broken now." If you're a product manager or a business person reading this, I'm certain you're nodding your head right now.

One way of solving this is by hiring a QA team to hand check everything. This still suffers from the human error issue, and is still expensive (although slightly cheaper per hour than developers doing the testing). Of course, then you introduce the overhead of the QA team and the developers having to work together to figure out the problems and fix them, and this takes you off down the rabbit hole of "not building new features because the engineers are busy keeping up with the backlog" -- not a happy place.

The best answer to this situation is automated testing. There are different flavors of this -- RSpec, PHPUnit, Cucumber, and Selenium are all words you might have heard when talking about automated testing. The idea is that you can write code to test your code. Once you have everything working in a nice balance, you know it will stay working into the future, and constant hand-checking is no longer required. Instant productivity boost!

There is a catch here. Running the tests still has to be done by hand, and the tests can take a long time to run. Large systems can easily have several minutes of automated tests, and when a developer is in the zone, being forced to wait for the tests to run breaks their concentration, their productivity drops, and that's when they open up a web browser and get distracted for a half hour by the funny internet. The other, equally bad outcome, is that developers are discouraged from running tests (because they realize the high cost of them being distracted), and they just don't run the tests anymore.

I'm sure you see what's next here. If the tests can be automated, why can't we automatically run the tests? How good of you to ask...we can. That's called Continuous Integration. The idea is this: when developers are writing code, every once in a while (usually once or twice per ticket), they commit code to a version control system (something like Git, Github, SVN, etc). This act of updating the codebase triggers the CI system to run all of the tests, automatically checking that none of the existing functionality got broken by adding the new feature (the fancy way of saying this is that the code hasn't 'regressed').

If everything is okay, the CI server logs that everything went okay, and the developers have already moved onto the next ticket. If something goes wrong, the developers get alerted, and the bug gets fixed. This means bugs get caught during development, and not in production by your users. CI saves developers time, and produces higher quality code with less bugs. It's one of those happy times in life where you get "more" and "higher quality" at the same time.

There are lots of CI solutions out there, but the CI system of choice at Cloudspace is the high-quality and open source Jenkins (the CI system formerly known as Hudson).

And of course, let's not forget the gratuitous self-promotion. In addition to our skilled engineers, we offer a hosted CI service -- Always CI. Please get in touch with us if you'd like us to help make CI easier for you.

Testing is dead. Long live testing!