How to Craft Your Path to Automated Testing

Last week, I started a new project at one of the largest banks in the world. But to keep myself out of prison, I’ll call them Global Bank.

A large part of my role is to design their automated test strategy.

And it’s really tempting to rush in and start cucumbering, selenifying, and junitizing things.

You know, like REAL programming work.

But when we create an automated test strategy, we’re actually trying to change a story.

And if we start by plugging away with tools, we’re going to get lost. It’s like trying to edit a story...without knowing what it’s about.

The Deposits Team (“Our Main Character”)

Global Bank is building new APIs to make it easy to communicate between departments and consume data. We’ll call the team I work with “Deposits”.

The Deposits team would love nothing more than to deliver a high quality API quickly.

But they can’t. Because for every release, MANUAL TESTING slows development to a halt.

The team can’t move quickly because they wait days for testers to manually setup & run scripts. And to make matters worse, testing validation is also manual. The team doesn’t fully trust the results nor have confidence in the quality of their software.

If the Deposits team can’t iterate quickly, they will either deliver late or release a crappy product into the wild.

30-Minute Releases Cycles (“Happily Ever After”)

But what if the Deposits team could move at breakneck speeds, and still produce great software?

If they could run their entire pipeline in minutes, not only could they iterate fast, but they might have time to...OVER-DELIVER. Maybe even dust off the items at the bottom of the backlog.

Cue the thunderous applause from Global Bank.

And that my friends, is why we need automated testing.

Path to Automated Testing (“The Plot”)

Automated tests help us build software at breakneck speeds without compromising on quality. It gives us confidence to deliver new value quickly without worrying about breaking old stuff.

But the biggest hurdle in getting to 30 minute release paradise is knowing where to start.

#1: DRAW YOUR CURRENT STORY

If we want to know how to change our story, we need to see the whole pipeline from code check-in → release. Find the people who know this best and interview them. It’s like a “tell me where it hurts” meeting.

“...our testers have 300-400 scripts they run through manually using SOAP UI. All of these have to be run before we approve a single release. It’s a nightmare...”Deposits Tech Lead

I’m visual, so I turned these steps into a storyboard. That also helps for future meetings.

Things to include in your story

• Steps - “Can you describe your current development cycle?”
• Metrics - “How long does step x take?” / “How long does the whole process take?”
• Major Pains - “What are some of the painful parts of this current test cycle?”

#2: FIND THE “YAWN” MOMENTS

Nobody likes a story that drags on. Try to find the bottlenecks in your pipeline. Automation will give us the most benefit with long, repetitive tasks.

After you have those, dig deep. Ask “why” things take so long. You might find a few gems:

“We run some functional tests during our build. But then QA also creates their own tests and runs them in SIT. They have an entirely different test process and we pass along bugs because we’re writing tests in silos..”Deposits Tech Lead

Yikes. A lot of our efforts in the future will look to align these teams and their processes.

#3: PLAN YOUR NEW STORY

You probably have ideas for what to automate. That’s great. I picked the top bottlenecks and drew a rough draft of what our new pipeline would look like. And now, we have a decent plan for where to begin.

It also helps to have a “success metric”. Success metrics help us show value and gauge if we’re actually solving the problem. They’re also great for aligning multiple teams / stakeholders around a vision.

It took our tech lead about two seconds when I asked him the best way to show value.

“Oh..definitely time saved. Definitely. Because that can be translated into man hours...which can be translated into dollars.”

We're working toward “A 30-Minute Production Grade Release”. The name (and plan) are works in progress :)

What does your testing story look like? Let me know in the comments!

So..

Creating an automated test strategy is like trying to change a story. But make sure you know yours first. To get started, draw out your current story, find the parts that drag, and create a new one.

Links

1. Andy Raskin, my go-to expert on telling stories
2. What is Automated Testing?
3. 5 Reason for Automated Testing

Happy Coding,

-T.O.