Insights — October 16, 2019

When Design Kills

UX methods for keeping your customer and your business alive

by Lisa McGee

UX Design

Skull Pattern On Gold

It may sound hyperbolic, but it’s true: interface design can kill you. Yes… you. From the hospital to the highway, we design systems that mostly help but all too frequently cause harm. Regardless of the damage done, outcomes are often chalked up to operator error. The person who was meant to use the thing didn’t use it right; they weren’t trained correctly; they didn’t take the necessary precautions.

Two such cases have caught my attention in the past year. One: The crashes of Lion Air flight 610 and Ethiopian Airlines Flight 302 caused by Boeing’s MAX 737 jets. The other: the Chernoybyl disaster depicted in HBO’s 2019 docudrama of the same name. On the surface, it would seem that, in both scenarios, operator error resulted in systems failures leading to death, destruction and tragedy. However, under the surface, you’ll find that both of these events can be traced to a long trail of misguided decisions way before the operators of the systems had opportunities to make fatal errors.

While the scale of these events may seem irrelevant to the every-day dealings of most businesses, they stand as stark examples of the consequences of design and technical debt.

Good design is the result of thinking about inputs, outputs, interactions, and results. It’s about getting in the weeds while keeping an eye on the big picture. And, it’s about making sure that every decision maker, business owner, designer, and engineer is accountable for their work and accountable to all of the people that they serve, not just shareholders.

What happened at Chernobyl

A sign on the side of a building

In 1986 a nuclear power plant in the USSR exploded, causing one of the worst nuclear disasters on record. During a safety test, power plant workers created extremely unstable conditions in the reactor, believing all the while that there was an emergency shut down button. What they didn’t know was that pushing that button would actually cause an explosion under the conditions they had created. Nuclear physicist I am not. However, it seems clear that there should never be a circumstance under which an emergency shut-down button causes an explosion. None. That’s really bad interface design.

What’s more, many soviet scientists knew about this fatal flaw. None of them did anything. No updates to the reactor, no extra notes in the manuals. The operators did not know how the system worked, and the interface lied to them.

What happened on the Boeing Max 737s

In 2018 two of Boeing’s Max 737 airplanes went down killing hundreds of passengers. In both circumstances pilots were unable to gain control of the planes during take off. After these tragic incidents hundreds of planes were grounded to investigate what went wrong.

How did this happen? In order to remain competitive in the market, Boeing needed to upgrade the efficiency of their engines. A noble cause to begin with. However, their existing 737 jet designs could not easily accommodate the new efficient engines. An overhaul to the design of the existing jets would, by law, require extensive pilot re-training. This would add significant overhead for airlines, and ultimately make the new efficient jets uncompetitive.

Their solution was to add new software to their planes that automated a change in the way they flew in order to compensate for a change to the design of the wings. Boeing downplayed the added software to avoid legal red-tape that would cause a headache for the airlines and impact the bottom line. On top of that, safety features and warning lights that could have provided crucial information and feedback about this new software were sold as an optional upgrade.

When the pilots of the Lion Air and Ethiopia Airlines flights encountered circumstances that caused the new software to kick in, they had not been trained, were not aware of the change in flight operation, and had no interface feedback to let them know what was happening. The pilots did not know how the system worked, and the interface lied to them.

Operator error, design flaw, or bad business decision?

When interface tragedy strikes, who’s to blame? Can it really be considered operator error when the interface, as designed, obscures critical information and outcomes? In both cases, the interfaces didn’t behave in the expected way. In both cases there were many people making decisions and many large pieces of metal manufactured before any buttons were pushed or levers were pulled.

A designer could have argued for a better interface. Chernobyl’s emergency shut off button needed, at a minimum, a different label. Boeing’s jets needed, at a minimum, clear indicators that an invisible software program had kicked in. The business owners should have foreseen the huge levels of financial risk being assumed by poorly designed systems. The engineer should have made a safer system in the first place.

While designers and engineers may be well equipped to consider their end users, many business decision makers are not. From the consumer’s perspective, it might be easy to believe that many businesses care more about making a profit than the safety of their patrons. In some cases that’s the truth, but more often, bad interfaces and experiences are a tangible expression of a series of complex decisions made by people across disciplines as a reaction to their present circumstances.

Even without malevolent intentions, a company whose business doesn’t prioritize consumer welfare will likely cause harm. Each organization must decide what the goals of their business are, but there are a lot of trends suggesting that the market is starting to demand higher standards from the companies they work for and patronize.

Avoiding fatal flaws in your product, service, or design

If getting top level business owner to reframe their purpose seems out of reach, fear not. There are many methods within the UX discipline that non-UX designers at any level can borrow to help to combat myopic decision making, and set up projects for success, which is ultimately good for business. There’s no world in which all products and services can be free of risk, but we can all take steps to ensure we’re not only avoiding dire consequences, but actually making something good.

Cross-functional from start to finish

At the start of any project it is critical to assemble a team of experts from every discipline and make them talk to each other. That means business owners, engineers, designers, and researchers get together to discuss their goals, fears, hopes and dreams. If you’re really advanced you could have a representative user in the room. End users are important experts in themselves.

It can be tempting to assemble this team at the beginning and then let it fade as the project moves on. Don’t do that. By keeping a cross functional team close throughout a product lifecycle you ensure that everyone sees the direct consequences of their decisions. Business owners can see how cutting corners impacts end users when they participate in usability testing and research, interface designers can ensure that their designs accurately represents the technical features of the system when they collaborate with engineers, and so on.

Make End Users Central to the Process

If you understand who is using your product, you can make sure you design it for them. At Chernobyl, a nuclear physicist would not have made the same error made by the operators working the evening of the accident. As it was, the operators did not have this innate knowledge and depended on interfaces, feedback and manuals instead. This is a great example of how understanding your end-users level of education should impact the design of the interface and the system itself. As always, the best way to ensure that you’re understanding your end user is to speak with them directly and get feedback on your product early and often.

Understand Goals, Context and Behaviors with a Task Analysis

Understanding the user is a good place to start. However, understanding demographics is never enough. It’s critical to understand what they hope to do and accomplish on both macro and micro scales. 

Task Analysis is a method that looks at what your user or customer is trying to accomplish and considers the steps, tools, and information required to reach that goal as well as the context in which action is being taken. This method ensures that each step in the user’s thought process is considered while they’re trying to complete their task, whether it’s signing up for a service, making a purchase, or operating heavy machinery. This understanding ensures that the service or interface provides the tools, steps, and information needed for accurate and easy-use. This analysis should be written out, documented and stress tested. Thinking through the most extreme scenarios in which your user or customer may need to complete a task can help to ensure that the service or interface won’t fail under duress.

Make Tradeoffs

The starts of projects are often filled with a rosy optimism. Once the project wears on and the constraints become real, decisions are made in the heat of the moment without thinking through consequences. Decide at the start of the project if you’re willing to sacrifice time, budget, scope or quality, and then think through the consequences of that decision. Releasing a subpar product quickly and iterating on it works great in some cases. If the health and wellness of your customer is on the line, quality and scope may need to take precedence over time and budget. Understanding these tradeoffs early on will help to ensure clearer headed decision making when unexpected issues arise. Because they always do.

Assign an Ethics Owner

This one is very advanced, but if you really want to make sure you do good, make it someone’s job to ensure you do. Make it someone’s job to think about the outcomes of your business decisions on your customer and the world. An ethics owner can ensure that those outcomes align with the intentions across the team. UX designers are prime candidates for owning your company’s ethics because they’re already used to thinking through systems, interactions and outcomes. If you take this step, don’t be shy about telling people. Put it on your website or your social media feed. It will make you look good, build trust with your customers, and hold you accountable.

The above suggestions are not a guarantee of success, or a guarantee that something unexpected and terrible won’t happen. As the world gets more complex, it will continue to get harder to estimate the full impact of seemingly small actions. That’s why it’s important to recognize, possibly now more than ever before, that the decisions we each make at work every day have real consequences outside of tomorrow’s meeting. Following these UX best practices will help to ensure that you’ve thought through those consequences of decisions and, as it happens, will make for a great product experience too.

(Articles only appear in the frontend.)