Michael MacDonald

$600 Mistake

In Business, Testing on August 12, 2009 at 8:00 am

Do you test? Do you test all the time (TATFT)? Can you afford to lose $600?

I believe software testing is essential. Behaviour Driven Development (BDD) is something that I’m learning and incorporating into my development lifecycle. I use Cucumber and RSpec with my Rails projects to implement new features and to verify their behaviour.

Aside from driving your design, validating your code, and giving you and your stakeholders confidence in your app, testing saves you money. That’s the bottom line. You might not think so. You might be able to get away with zero testing if you are extremely lucky or if you are a super coder who writes perfect software. Ultimately, the time you invest in writing tests for your code will save you time and money.

I recently made a small change to my SciNews web application without writing a test. Why? Because it was just a small change and I was in a rush. I had opened the app in my browser and tried it out and everything was fine and dandy – deploy!

A month went by without any problems. Admin users of the system weren’t experiencing any issues but when a client user tried to use the service – WHAM! BAM! ERROR!! So there I was, just before midnight, on the receiving end of a support call from the client trying to figure out what just went wrong.

I tried to fix it on the spot but I didn’t understand the cause. The only change I made was a minor one about a month ago and admins had been using the system fine during that time. But this was the first client access since then. Aha! But unfortunately, after 15 minutes on the phone with the client, I was unable to solve the problem. As I later discovered, the bug affected client roles, not admins. When I tested it in the browser, I had done so using my admin account. I never tried it as a client.

In the end I had to manually perform the service for the client on their behalf. But the system had failed them and worse, it was their first use of the online service. In order to save face, compensate them for their wasted time and frustration, and to keep them as clients, the service was provided gratis. My small little insignificant change had ended up costing my business $600 and for a small, fledging business, that’s a lot of money that we can’t afford to lose. For $600, I should’ve written a test! Add in time spent on the phone with the client, the embarrassment, and the potential damage to the service’s reputation, I should’ve definitely written a test.

Will this client come back and try our service again? I hope so, otherwise my little error may well end up costing heaps more in lost revenue. This client may have become a repeat customer, so we’re talking a loss of $600/mth. And what about word of mouth? They might have referred two new clients. Now we’re talking some serious lost revenue here all because of one small minor change. Hopefully I’ve done enough with that client for them to give the service a second chance. Whether they do or not, I’ve learnt my lesson – TATFT!

  1. How times change. How the mighty have fallen, and all that … I recently quoted you – yes you Michael – and it wasn’t “TATFT”, it was more like ‘as MM used to say … “just put it up and see what happens, we live on the edge here”. Remember saying that a time or two? 🙂

    • Ha! More like how the mighty have become mightier professional! 😉

      Actually, I still stand by my original quote. In lieu of any proper testing suite (and the time, resources and inclination to implement one) and in the face of burdensome bureaucracy then the quickest and easiest way to know if something will work is to be the cowboy and live on the edge. At least that way you’ll fail fast (sorta). Sometimes the only way to know if something would work was to launch it since the development and production environments were so different.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: