Robert Bleidt – Articles

Often when designing a new product, you require knowledge about customer needs, user workflows, and competitive products. Although the sales force, customer surveys, or web research may help to gain this knowledge in many cases, sometimes they may not be enough. You may be working for a start-up with no customers or sales force, or you are developing a new product in a new market.

In these cases, customer visits are a powerful qualitative research technique that can provide insight for the product development process. This is not merely the act of visiting a customer, but a formal technique of collecting information on the workflows, user archetypes, and critical needs for complex products. According to expert Ed McQuarrie (who I’ve hired in the past) Apple, Microsoft, Cadence, Tektronix, and HP are among the firms who use it. Product development expert Robert Cooper reported that customer visits ranked #2 of 18 methods of developing new product ideas in their 2008 study.

Why is this technique powerful?

• It gets you “outside your building” and sitting next to actual or potential users in their environment.
• It helps to get engineers or scientists aligned with product management or marketing.
• You can learn workflows, attitudes, or issues that would otherwise be difficult to discover.

Often during a visit, or soon thereafter, “a light bulb will light up” as a idea for a new product or feature becomes clear. A common insight you will get in a customer interview in a new market is “We can’t do that because…”, where one of your team’s implicit assumptions is challenged. Even better is seeing users use a product in unexpected ways.

An apocryphal example is the military pilot who complained to the mechanic that his afterburner did not work. The mechanic chained the plane to the test stand and carefully advanced the throttle into afterburner and it worked correctly. After stronger and stronger complaints from the surly pilot, the mechanic was put into the back seat and the plane flown to altitude. The pilot then slammed the throttle to the stop as he would in combat and nothing happened. The throttle had been quickly forced beyond the point of engaging the microswitch that controlled the afterburner. Not something learned second-hand on the ground.

A customer visits program works best for complex, multi-user, business to business products. If it’s a simple, single-user standalone product, you may be able to gather the information you need with a phone or Zoom call. Customer visits will be expensive in time and travel, but worth it for the right products and questions.

Setting Objectives – What do you ask and how?

A key component of a customer visit program is establishing the primary questions or objectives you want to learn or test. Despite the name, you do not just “visit” with a customer and hope it will lead to something, you need an advance plan of what you want to learn and who (what roles) you want to talk to.

Staying in the aviation genre, I will make up a ridiculous scenario to illustrate this: Your CEO was inspired on a recent flight to replace “those expensive airline pilots” with an AI-based voice assistant, and wants you to define the product. Further, we’ll assume that the CEO says not to worry about regulatory issues, he has that taken care of. Unfortunately, your company has no aviation experience.

Thus, your objectives might be:

• What does the product need to do?
• Could I replace both pilots, or just one?
• Are other approaches being considered such as controlling the plane from the ground?
• What reliability would customers accept? What does reliability mean to them?

Before even starting any customer visits, you are going to have to gain enough background to “know what you don’t know” so you can ask intelligent questions. Let’s say you hire a retired pilot or avionics engineer, or both, to advise you on the basic domain knowledge. This will be far cheaper and quicker than trying to do so by customer interviews. (This is perhaps an important point – customer interviews are primarily for gaining uncertain, variable knowledge, not readily obtainable facts or know-how)

Based on their advice, you identify several roles you would like to interview:

• Pilot
• Air Traffic Controller
• Avionics Designer
• Airline Operations Manager or Executive
• Aircraft Manufacturer Program Manager
• Aviation Human Factors Researcher

You have to identify companies and contacts with these roles. In this hypothetical case, your firm has no experience in aviation – It’s a new product in new market in new industry project. Here, my later remarks on the ease of gaining interviews may not apply. You are going to need some reason for target companies to meet with you, or an introduction from someone they are familiar with.

Then the question preparation begins.

Likely you have many things you want to confirm or explore. For example, some internal technical questions may be:

• What’s an acceptable response time to respond to an air traffic controller’s command¹ and execute it?
• What should the system do if a command is not understood?
• What if the autopilot (your likely interface with the plane) won’t do what’s asked, such as fly into a mountain or a supersonic dive?

Likely, there are business concerns as well:

• Who would buy this – an airline, an airplane manufacturer, or a manufacturer’s subcontractor?
• How long until first revenue – what’s the customer’s product cycle?
• Are there barriers to others entering this market? (Or does competition already exist?)

It is best to learn as many facts or as much general industry know-how and terminology from other sources (like the pilot you hired) before starting your interviews. Interview questions you should save to:

• Confirm assumptions
• Explore ideas
• Gain knowledge you could not get elsewhere

This example shows one of the issues in developing questions. If you simply walk in to an avionics manufacturer and say “We’re researching a AI-based voice assistant to replace airline pilots”, you have given your idea away to a likely competitor. You need to prepare a less detailed vision of your potential product that would get some interest, but not give away the central ideas. Perhaps you approach this visit by saying you’re researching how AI and speech recognition can be used to monitor pilot performance for safety, etc.

A related problem is that of visiting a prominent innovator in an industry. Anyone else will be doing the same thing, and ideas you discuss may be spread to the other companies who work with the innovator. Sometimes this can’t be avoided, but in many cases, you can begin with less prominent but similar firms.

Once you have established the questions for a visit, then you need to place them in a discussion guide. Ideally, this will be a sheet like this:

This guide starts with the company name and date, but then we list the participants we actually talked to, and what we learned their role might be from our viewpoint. It’s also helpful to list a cue or reminder so you can recall that person later. Note that this is the participants for one meeting. You may have several of these sheets for different meetings during a visit.

Then we have the questions section. Each question has a block to check it off as it is discussed and a priority. Most likely, you will have more questions than time, and will have to look at the priority to make sure that you try to get the “A” questions answered first.

Depending on the situation, this guide may be a printed page or an electronic document. How to Remember What a Customer Says has some techniques you can use to prepare and record interviews.

What will likely not work well is if your discussion guide is just a list of four points that you will “use as a starter”. You will not ask the same questions at each company, you will forget to ask details, and the overall quality of the data will suffer. With experience, you can make it seem like a natural conversation, but you are staying on-script with your guide. You will likely not do enough interviews to memorize everything.

If your objective is to do workflow analysis, you will also need a target list of the types of roles you want to interview.

The team and the visit

While I have done interviews by myself in areas where I have a lot of knowledge and experience, ideally you will have a team of two or three people. A good team needs the following roles:

• Moderator who asks the questions and leads the discussion.
• Secretary who records as much detail as possible.
• Thinker who is considering clarifying or follow-up questions.

A good opening for a meeting is something similar to: “Thank you very much for agreeing to meet with us. We are here to get some industry background to help in planning future products. We are not selling anything today, and everything will remain confidential. Also, we can’t make any promises about what will result, since we are just in the planning phase.”

Then start with some general questions from your “A” list that are easy to answer and start to build cooperation and rapport with the customer. Ideally, you should slowly transition to the more specific or difficult questions as the meeting progresses, keeping an eye on the time and remaining “A” questions you want to ask. If things are going ahead of schedule, you can start to phase in some “B” questions. Finally, you want to end the interview with some easy questions. Don’t just walk out after the customer tells you the price and failure points of your competitor’s product, as he will feel “used”. Taper the conversation down slowly.

Who to visit

For the organization that has never done customer visits, colleagues (or perhaps you) may be skeptical that customers or prospects, some of whom you have never met, will meet with you. In my experience, this is not a problem – I can think of only a few cases where we could not secure a worthwhile visit. Unlike a sales call, such visits don’t happen everyday, and you are giving the interviewees some status as “domain experts” by interviewing them. It’s an opportunity for them to stand on the soapbox and voice their opinions. Sometimes they will discover insights from the discussion as well.

Rarely, you will have secretive customers like Apple during the Steve Jobs era. I once joked then that the difference between Apple and the NSA was that sometimes the NSA would answer questions.

So, who and how many customers to visit?

One answer is you should plan enough visits to be reasonably certain of discovering important insights. Experts say 10 visits may get you 75% of insights, and 30 will get you 95%. In practice, usually 10 to 20 visits is a good compromise. If you have not done customer visits before, this may seem like a lot of work.

The number also depends on whether you are serving a homogeneous market, or several segments. If there are distinct segments, you need to plan on visits in each.

Finally, you should consider visiting your competitor’s customers as well as your own. Often this is a great way to learn about problems with your competitor’s products and potential markets for you.

In most cases, all these factors are present, and it’s typical to construct a matrix of visits. Axes of the matrix might be customer size, industry, region, use cases, or supplier. For example, here’s a hypothetical matrix:

	AWS	Azure	Self-hosted
Node	Smith Industries ABC Group	DEF Corp.	Franklin, Grant, and Jackson
Java	XYZ, Inc.	Project 29	Mega Group
React	ABC Group	Frivolous Work Bureau	Nano Corp. J&K Engineering
Angular	Inadvertent Technology	GH Industries	LMN Systems

In this hypothetical example, I’ve tried to cover customers who might be using a group of popular web technologies on cloud platforms or their own servers. This is a 4 x 3 matrix, which is getting pretty large – there are 13 companies already. From a practical standpoint, you might not be able to get multiple customers in each cell in the time budgeted. This is where experience comes in, and the newcomer can start with a few important cells to test how big the matrix should really be. It’s also possible in trying to fill a matrix you might find there are no users/customers in a cell, just because it is unattractive.

Sometimes, you can visit the entire relevant population. In one case, I was looking to design a new business to serve European premium auto manufacturers. So, we visited all of them.

A related question is who do you want to talk to at those customers? Well, who specifies, approves, buys, uses, and maintains your or your competitor’s product? Does the customer have internal substitutes for your product? In a sense, this is like major account selling – you need to identify influencers, gatekeepers, and decision makers. A difference is, you are not hoping at this visit to make a sale, you are collecting information for future sales.

If you have a salesperson selling into this customer, you can often get help from them in identifying the right people. Alternately, if you explain what you are looking for, the customer or prospect may help.

For example, in the 1990’s I was researching making a broadcast-quality web-based video editor. Sales connected me to engineering managers at mid and top market TV stations where I said “I’m researching a better and quicker way to edit news videos, I would like to talk to some people who do the editing, the reporters or producers who manage news stories, the managers who are in charge of your newsrooms, and of course your engineers who would install and maintain the equipment.” After a few visits, I had more understanding of local news and could name specific job titles.

What could go wrong with a visit?

Salespeople

In a mature organization, you will likely have a salesperson who “owns” several of the customers you will be visiting. This situation will need careful management. Many times the salesperson can provide valuable background on the customer’s structure and suggest thought leaders, gatekeepers, or decision-makers you should consider interviewing. Often, sales people are a great source of ideas and information.

However, if the salesperson demands to be present when you conduct an interview, they can reflexively jump into selling mode and start defending your products or explaining a solution to a customer, ruining the interview. It’s also difficult to tell the customer “this is not a sales call” when their salesperson is sitting there.

Some salespeople will give you free rein to visit the customer alone. Others may have a sale pending and be concerned a visit will interrupt it.

It’s almost always best to brief the sales person beforehand, and if possible, involve them in a review meeting afterward. What about during the visit? Here there are several options for salespeople:

• They don’t attend the visit.
• They visit someone else at the company while you are interviewing.
• They introduce you and leave after an introductory meeting.
• You split into two teams, one with the salesperson, and the other does the important interviews.

If you are able to reach agreement with the salesperson, you have to hold up your end of the agreement. This means you don’t promise future products or start selling the customer yourself.

This is not to say in some cases a salesperson should be excluded. If he or she is passive on a company and it’s your team’s first interview, they can often bring a lot of trade-craft, experience, and advice to help you learn quickly.

Engineers

Ideally, one of the people on a two or three person visit team should be an engineer. Customer visits are a great way to get engineers “out of the building” and into the real world. Often, hearing information straight from the customer’s mouth and seeing the use environment firsthand can motivate new thinking and change preconceived ideas and opinions.

What can go wrong with engineers is they can start designing a solution on the spot, and even take over the customer’s attention. If you build one-off products for single customers, that’s great – but likely in that case you are not conducting a customer visits program. Worst-case, the customer thinks you are building that solution for him, and he expects it soon, not buying product for today. It’s fine to briefly test ideas or concepts, but not to start start drawing the block diagram or specification for a product you may not decide to build.

Customers

Conversely, occasionally you will have an interviewee who considers himself a visionary. He’s already thought through the perfect solution to his problems and you just have to build it, as he explains to you in a monologue. Well,

• Maybe he is right. Listen for a while and then start challenging some of his assumptions. This is where the Thinker needs to start interjecting questions or explanations.
• After a polite interval of listening, explain you need background to understand his solution. Use that to refocus the discussion on your A questions.
• Explain you will put his valuable idea into consideration as part of this study, but you can’t promise anything.

Sometimes, you meet with a person who does not have technical knowledge you need. Don’t give up. Often they will be able to answer workflow questions, or direct you to interviews with appropriate people.

A more common problem is the customer who uses the interview to complain or vent about past products or experiences with your company. In this case, all you can do is listen and take notes until he or she is finished.

Do not open on Broadway

If you or your organization is new to customer visits, don’t start with the biggest, most important customer in your visit matrix. The first customer visit will be intense and anxious in some cases, and you will be learning new skills. Start “off Broadway” with a less important customer and gain experiences. After two or three visits, your team should have enough experience to move to more important visits.

Also, if you are studying a new market, you may have knowledge gaps that will lead to naive questions on your first visit. Armed with new background from initial visits, you will then be able to approach larger customers and speak with confidence.

After the visit

Immediately after the visit, the interview team should go somewhere reasonably private and compare notes on what you heard. Write down and review everything. Do not wait on this, as memories fade quickly. This is also a good venue to debrief the salesperson for the account.

In the next days, it’s always a good idea to send a thank-you email to the people you visited, and perhaps use that as an opportunity to clarify any details you are unsure about.

Analysis

Remember that customer visits are a quantitative research technique, not a survey. You may have some yes/no questions such as “do you normalize content on ingest?” or “do you use cloud object storage like S3?” that can be scored. Or you might be able to conclude “six of ten competitor customers would switch to us immediately if we could do X”. Most of the information, though, will be anecdotal or narrative – which is part of the value of a visit instead of a survey.

Alternative Venues and Techniques

Two other techniques for qualitative market research are focus groups and ethnographic or anthropological research. Focus groups are the most widely known and popular, and are a standard of consumer marketing. Focus groups are great for getting honest feedback on existing products or your company’s image and behavior. Focus groups are usually a small group of consumers (or in some cases individuals) who are led through a planned interview by a moderator, while you watch behind a one-way mirror or on recorded video.

If your situation does not require any workplace observation or multiple interviewees, or your travel budget is very limited, there is the possibility of arranging interviews at a user group meeting, if your company has that, or at a trade show or industry meeting where everyone is already present.

During the covid pandemic, readers have asked about using Zoom or Teams for interviews. This is not the same as being there yet. You will not be able to observe the customer’s environment, and he or she will be looking at a potentially distracting computer, with disturbing email and chat pop-ups. Can you get some information this way? Yes, but it’s not ideal.

Of course, you should not forget about qualitative techniques as well. Each has their place.

Learning More

An ideal place to start is Customer Visits, Edward McQuarrie, 2015.

A ranking of customer visits to other research techniques is given in Ideation for Product Innovation: What are the best methods?, Cooper and Edgett, PDMA Visions Magazine, March 2008. (available at stage-gate.com)

---

Notes:

1. Technically, controllers issue “clearances” not commands, since the pilot is legally in command of the flight. ↩︎