A painted door is an architectural design element. The designer creates an image of a door where none actually exists. This serves a number of functions, such as making a building seem larger or more symmetrical without the extra cost of adding a real entrance.
What is a Painted Door test?
Painted door tests are tools in building tech products that accomplish something similar: You show your customers an option where none really exists… yet. This serves to help you gauge reactions to a feature that has not yet been built or fully implemented. While qualitative and quantitative user research can give you feedback, it is often difficult or expensive for small teams or early-stage startups to field. The results of such research may be hard to translate into a full product if it’s gathered from focus groups or surveys. What people say in a static survey or group can be quite different from how people interact with a live product. Showing an option in the wild is a useful alternative: It can augment your data and thinking by allowing users to interact with a feature in real life.
The goal of painted door tests is to gather data, not to deceive your users, so careful and ethical use of this technique is critical. It’s important that people feel like there is a payoff for being sent down a path with no resolution. When used correctly, however, painted door tests can provide you with useful information and new insight into user behavior.
Pre-product launch
Before you build a full product, testing can give you a sense of the messaging necessary to get people in the flow. Here are three examples of tests that can be employed prior to launch:
Showing a new option to gather feedback: A team wanted to add a local payment option in a country where they had limited data. Before they invested in implementing it, they wanted to gauge interest and understand whether it would increase conversion. As an experiment, they added a non-functional version of the payment option. When someone clicked on it, they were greeted with a “coming soon” message thanking users for their feedback. To their surprise, a large number of users chose the new option, which they invested in and implemented shortly thereafter.
Running ads to test user understanding and acceptance: Startups are often exploring a number of different spaces. Ads are a good way to take a small budget and determine what the baseline conversion is. Some companies experiment with different product messages, behind which they show a sign-up for a beta test, to see what appeals to users. This is a good way to explore a broad set of messages without writing a single line of code.
Pre-selling new products: Tesla announced in September of 2015 that they would showcase the Model 3 in March of the following year. The buzz from this announcement led to over 100K deposits being placed within a day. The car didn’t officially start shipping until late 2017. Kickstarter uses a similar model: Sell products to fans before you produce, and test supply and demand before you manufacture.
The most important part of a painted door test is closing the loop. This means informing those who are interested in a potential product option that it is coming soon and thanking them for their feedback. Those who see an ad should receive a benefit for signing up to be an early customer. Those who invest in a Kickstarter should be given a bonus, such as an exclusive item or signed copy. It’s crucial to also ensure these users are the first to receive the product when it launches. The key is to use these tests to learn from users without making them feel like they are being taken advantage of or served a broken feature.
Principles for a painted door test:
Treat people fairly. Allow them to express interest. When possible, also give them a chance to provide feedback.
Ensure users don’t end up stuck at a dead end. Provide a call to action or bring them back to their previous place in the flow.
Always close the loop with users and explain what you were testing.
Thank users for participating in the test and helping you improve your product.
Offer tangible benefits, such as being the first to know when the feature will be launched.
Painted Doors in life
You can also leverage this principle in life decisions, which can often feel fraught. Running painted door tests allows you to gather more information before committing. I remember when a friend of mine got job offers from both Facebook and Google. She decided to make the commute between her home and each office to get a feel for the traffic. After a few days of testing, she decided on the closer company, because the other commute would drain her energy and prevent her from performing at her best.
Before you buy a car, consider renting the same version to see how it feels. I wish we had done this with our Honda Odyssey. While we love the car, we only realized after we had bought it and taken it home that the backdoor gate is too short for us. We’re both tall and have hit our heads multiple times since (probably losing more than a few brain cells along the way).
Before we got our dog, I spent several hours at my friend Mauria’s house, playing with her Labradoodle, Coco. Surprisingly, despite a lifetime of allergies, I had a good time with Coco with nary a sniffle. My son also has bad allergies, so my next step was to bring all three kids to spend the afternoon with Coco. They walked her, played with her, and cuddled with her with no ill effects. It was only then that we started to believe hypoallergenic dogs were really an option for us. A month later, we adopted Wonton, our mini poodle.
The examples don’t stop there. If you are unsure about joining a company, consider asking to shadow someone for a day to get more information. Sit in on a few meetings, chat with the people around the office, and get a feel for what the culture is like. If you are undecided about moving to another city, try staying there in a rented home for a few weeks to see how the environment feels. Before we moved to Santa Clara, we drove down a few times and walked around the neighborhood at dusk, something we had planned to do daily once we got there. This gave us a feel for what it would be like. If you are undecided about a long-term relationship, travel together to a foreign country for at least two or three weeks. How you handle the stress of traveling in a place where you don’t speak the language can tell you a lot about a person (Amazing Race couples who fall apart, I’m looking at you!).
Painted door tests provide you with additional information that you can’t gather elsewhere. This can help you make life decisions—especially big ones—with more context and confidence.
Testing, iterating, and learning are important practices, and they can have a positive impact—both on your products and in your life. By adding painted doors to your arsenal of tools, you are further equipping yourself to make the right decisions and invest your time and resources in the right things.
Leveraging testing the real world to build better productsRead MorePerspectives
