jump to navigation

Automation Can Be Dangerous December 6, 2018

Posted by Peter Varhol in Software development, Software tools, Strategy, Uncategorized.
Tags: , ,
trackback

Boeing has a great way to prevent aerodynamic stalls in their 737 MAX aircraft.  A set of sensors determines through airspeed and angle of attack that an aircraft is about to stall (that is, lose lift on its wings), and automatically pitch the nose down to recover.

Apparently malfunctioning sensors on Lion Air Flight 610 caused the aircraft nose to sharply pitch down absent any indication of a stall.  Preliminary analysis indicates that the pilots were unable to overcome the nose-down attitude, and the aircraft dove into the sea.  Boeing’s solution to this automation fault was explicit, even if its documentation wasn’t.  Turn off the system.

And this is what the software developers, testers, and their bosses don’t get.  Everyone thinks that automation is the silver bullet.  Automation is inherently superior to manual testing.  Automation will speed up testing, reduce costs, and increase quality.  We must have more automation engineers, and everyone not an automation engineer should just go away now.

There are many lessons here for software teams.  Automation is great when consistency in operation is required.  Automation will execute exactly the same steps until the cows come home.  That’s a great feature to have.

But many testing activities are not at all about consistency in operation.  In fact, relatively few are.  It would be good for smoke tests and regression tests to be consistent.  Synthetic testing in production also benefits from automation and consistency.

Other types of testing?  Not so much.  The purpose of regression testing, smoke testing, and testing in production is to validate the integrity of the application, and to make sure nothing bad is currently happening.  Those are valid goals, but they are only the start of testing.

Instead, testing is really about individual users and how they interact with an application.  Every person does things on a computer just a little different, so it behooves testers to do the same.  This isn’t harkening back to the days of weeks or months of testing, but rather acknowledging that the purpose of testing is to ensure an application is fit for use.  Human use.

And sometimes, whether through fault or misuse, automation breaks down, as in the case of the Lion Air 737.  And teams need to know what to do when that happens.

Now, when you are deploying software perhaps multiple times a day, it seems like it can take forever to sit down and actually use the product.  But remember the thousands more who are depending on the software and the efforts that go behind it.

In addition to knowing when and how to use automation in software testing, we also need to know when to shut it off, and use our own analytical skills to solve a problem.  Instead, all too often we shut down our own analytical skills in favor of automation.

Advertisements

Comments»

No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: