Michael Margolis

Michael Margolis is a UX Research Partner at GV. He has been spearheading UX research practices at Google Ventures for the last 12 years. His testing approach "the five act interview" is highlighted in the book "Sprint".

The Research Sprint -A pragmatic path to efficient UX research

If his name looks familiar, it’s because the UX research partner Michael Margolis appears in the world-famous book “Sprint”. He is none other than the creator of the testing process known as “the five-act interview”, used nowadays by thousands of startups worldwide.  

These 1:1 customer interviews are at the heart of his Research Sprint, a 4-day process that combines with the design sprint, and helps startups and big companies test their assumptions and de-risk product ideas, and innovation strategies.  

Michael Margolis

Michael joined GV in 2010 as the first UX research partner at a venture fund. He has conducted hundreds of research studies for GV startups, including Slack, Flatiron Health, Foundation Medicine, Gusto, Bluebottle Coffee, Nest, Lime, and Uber. As part of the design team, Michael has boosted conversion, tested new concepts, streamlined experts’ workflows, and validated product-market fit for hundreds of startups.

In 2006, Michael started at Google as a staff user experience researcher, where he conducted research for Gmail and Google Talk, led the UX research team for Google Apps, and managed Google’s UX team in Seattle. Prior to Google, Michael spearheaded user research at Walmart.com, helping build new online and cross-channel businesses. In a previous life, he produced educational software at Electronic Arts after starting out as an editor at The Learning Company.

Michael earned his bachelor’s and master’s degrees in anthropology from Stanford University. After many years in the San Francisco Bay Area, Michael is living happily ever after near Seattle.

Links

Welcome to itoday Apéro #8. I’m so excited to have everyone here. We have an amazing crowd today. So, the guess we have today with us is Michael Margolis. He joined Google Ventures in 2010. So 12 years ago as the very first UX researcher, he has been leading the team there. And working on countless of projects of UX  research.

He worked with startups including Slack, Flatiron Health, Bloomberg Coffee, Nest and many others. Yeah, he’s absolutely a star. He’s a UX researcher. Please everyone live from Seattle, USA -Michael Margolis. Welcome Michael.

Hi, thank you so much. I really appreciate your guys inviting me, I’m happy to see all of you. So I want to say before we even start, that I’m just terrified to interview you. You know, you are such an inspiration. And basically it’s hard to interview the godfather of user research.

So I hope I’m going to do my best. Thank you so much for taking time. Really it’s an honor to have you here at Itoday Apéro. There are a lot of people here who know who you are. Of course, because your name is featured in the book Sprint, about the Design Sprint, and you, you play a very important role in creating basically the whole interview methodology that is used in the book.

So your name is quite famous to these people and of course to the UX research community, but for all the people who are a bit less nerdy so maybe could you introduce yourself in a few words?

Michael Margolis

Sure. So I’m the UX research partner at GV in alphabets venture capital team. And so what I do there is I use UX research and design to help startups learn more, faster, find product market fit, de-risk big decisions. A lot of the work that I do is kind of coaching, advising, training, showing people how to use research to. I mean faster progress and to build better products.

I heard that you were the very first UX researcher in a venture capital fund? How did you get there? Yeah, as far as I know, I’m the first one. Couple other people out there doing it, but it’s pretty unusual job, so. I can give you kind of 30 years in 30 seconds of how I got here.

So my first real full-time job, you know, I studied anthropology. My first real full-time job was as an editor in educational software company back in 1991. So we’re making little floppy disk software. And so that was kind of my introduction to product design.

The group was doing a lot of testing products with kids at the time. So it was my first time of really seeing that. And also, I was the editor and building the documentation for teachers and parents. You have to explain how to use products. And so when you’re explaining things, you are kind of at the back end and you’re learning about design.

And then I went from there to work at Electronic Arts, where I learned how to ship box software. What does it take to build these things and get them out the door? And then I worked at a design strategy consulting firm. Where I got to essentially apprentice with really great researchers and designers there. So that’s where I learned how to do kind of deep customer research.

I got to work with people like Steve Portigal on Tom Williams and Sue Squires to be very client focused. And so we did these very big, very expensive projects.

And then I went to walmart.com where I had to learn kind of how to take that and adapt that and do it much faster, much cheaper. I can’t do much scrappier right? Walmart is all about everyday while costs. Yeah, and it was a place where I had to do a lot of it. So I just got a lot of reps and I was doing a ton a ton of things. So it was a ton of practice.

It’s also where I met Vanessa show who’s my partner in crime, that GV. And she is design partner. And so then I went from there to Google where I learned just a ton. So I worked with this amazing UX team. So that was in 2006. It was smallish team compared to us now, which has learned a ton ’cause at Walmart I was essentially a solo researcher.

Joining Google Ventures back in 2010

And then when I got to GV, it was really about startups, about speed. Like how do I continue to make this faster? How do I hone this for startups to be higher impact in dealing with a really huge variety of different kinds of companies across domains.

And so it’s a lot of teaching, coaching, advising. And as I said, showing people how to do this- It started, ’cause Braden Kowitz was the first designer at a venture capital firm I had worked closely with him at Gmail team. And he’d gone over in six months later, he’s like:” no, I need to research. Can you help me with some of this stuff? “Yeah, and so we just started doing that.

You told me it’s 12 years you worked at GV. Right? Now you are leading the UX research team. So how did your role evolve over the years?

It’s evolved along with our portfolio, which has changed and it’s evolved because we’ve just gotten better at kind of optimizing how we do this work. As an example, I used to spend a lot of time developing programs to teach people how to do research. How do I do interviews? How do you plan all of this? We had these giant workshops and things and what I found was that that wasn’t didn’t seem very effective.

And what I learned was the best way to teach people how to do this was to show them. So when you have a project, let’s do it together or I’ll do it with you, or I’ll do it for you, and you’ll see how I do it.

Because when I was doing these big workshops, I would train people how to do it and go through as a one day, two-day workshop. And then among six months later, they would say: “I have a project like how do I do this thing? Like I haven’t done any of the research and I don’t know how to do it”, and so just didn’t seem that effective.

And so we just got more in the habit of doing it with them. I spend a lot of time optimizing the work I do in a way that it’s transparent. So they can see me going through the recruiting process and we’re doing that together.

We’re defining who is your bullseye customer and we go through that together. It’s why a lot of the things that some of you probably seen that I’ve written, a lot of it was for that. I’m like, OK, here are the recipes. This is how. We can do it so when you need it. I’m there and we can kind of do it together and expose the team to it. They see it and they go through it. and they kind of have that wonderful ha moment when we do those studies. And then, oh, I see if we now I see what you’re talking about, now I see the power of this when we have these conversations and we do the research this way with our customers.

It’s a lot that’s really changed quite a bit. We’ve done hundreds and hundreds of studies with you know, hands on with our companies.

By the way, Michael shared a lot about the recipes of UX research.We can find them on medium, going to add the link in the chat. I really have to say that everything I know about user research, all the practice I got is from just watching your videos.

So I think it was really generous of you just script all the documentation. Yeah, you can check the links. Basically the whole GV guide to User Research on medium. That’s absolutely amazing. Everything is there.

Helping Jake Knapp and John Zeratsky creating the design sprint

A few days ago I announced that you were coming to Itoday Apéro on the design sprint Slack channel. And so John Zeratsky answered me. John is the Co-author of the book Sprint. Right. He wrote me this. He wrote, I quote him:” Michael keeps a pretty low profile, but he was instrumental to the design sprint process and had a huge impact on how we do research. Before we started working with him, it was unheard of how to plan and run the user research. So, yeah, what’s the story?

Well, it’s kind of a low profile. I was kind of hoping this was going to lead to a Netflix special. Pretty cool. So I think the way that it evolved to me, it seemed like a somewhat of a natural. Starting from what we had been doing. I haven’t been working with Brayden. I’ve been working, bring Kowitz and Jake now at Google on Gmail on some of their very early things that eventually turned into Google Hangouts.

We had ways of working already where, like we were conducting studies, figuring out new designs, iterating kind of planning and doing that. And we saw some of the side effects of planning a research study right. You schedule that and all of a sudden remarkable amount of work gets done by a team when they know, oh crap, we have a study coming up and people are gonna come in. Like we need to generate these designs.

And so there was a lot of that work that we had been doing for years. I saw the effects of that and then as I said, a lot of my tendency is to try to make things faster. And I think that the one of the big things that happened was getting to GV. I had a lot more freedom to recruit.

So when I worked within Google, there’s a really remarkable internal infrastructure team that provides a lot of the recruiting. And, for good reasons, there’s a lot of policies and a lot of rules and a lot of ways that happens and it’s for good reason as I said, but it’s slower, right? ’cause. You can’t just do it on your own.

And so when I was at GV, I had a little more freedom to just go do it. And so when I was able to take control of that, I could accelerate it pretty dramatically. Originally, starting out just recruiting with like Craigslist and finding my own people very careful. Screeners and those kinds of things. That helped me see like, oh, I can. I can really shorten it.

That was always the thing, that was kind of the limiting factor of how quickly we can get this organized. How fast can I recruit. And now as you guys know, there’s what’s changed, their economy services, right. So I can use things like user interviews. And they can help me find groups of people really, really quickly. And so that that shortens that lag time dramatically to be able to do all the interviews.

What’s the max number of interviews that I can actually survive? What do I have the stamina to get through in a day and what seems to work right? As you guys have seen you do three or four interviews and we were kind of sort of seeing some of the patterns. And after you do five that basically the whole team is like:” Oh my God, don’t make me sit through any of this for doing the same thing over and over. Let’s just like, let’s go fix it and build it and do the next thing.”

And so being able to compress those things allowed us to fit it into that that fifth day. And as the Research Center to me, like everything, everything requires research. Of course, if you’re designing it, you don’t know if it’s working until you kind of ground it in that feedback.

And so that’s kind of for the first day, but a lot of times we’re doing the research before the sprint also. So there’s a lot of work that’s happening to kind of inform that first day and leading up to it. So it kind of often bookends. I’m going to work. This is great and all of these.

Basically, it’s what we call the research sprint, right? So we got to talk about that. Was it existing before the design sprint. It’s the process that you were already using and basically they just took it and it became part of the methodology. Or is it something that you created pretty much at the same time?

I think it’s kind of like an adaptation of things, that are optimization of things that we were kind of doing before. And then we learned overtime. So there are ways that we started doing it and then it kind of got better and smarter as we did it.

So an example of things that evolved overtime where the technique that we use for the watch party. So this very organized structured way to have a team taking notes. Debriefing and then by the end of the day, knowing kind of what are the big takeaways.

We experiment to put that in place. A lot of these things were just, we tried stuff and something was really good, didn’t work great or you know, why didn’t that work so well. And we were doing enough reps of it. That we kind of tweaked and adjusted.

And so that combination of having enough reps that we were doing, where we could experiment and just having that that pressure from the teams also. They were like well five days is a lot, right. So we need to kind of get it done. Like how can we get it done, because coming back and doing more analysis or thinking about it, we just frankly, didn’t have time.

Also I’m lazy. And I don’t want to go off and do a bunch of analysis and generate slides or any of that stuff. Like let’s just do it and move on. And so it was.

I think it was really an evolution of things that we were doing. You know, I had been doing research before we did design sprints, but it just fit into that. You know, Jake and Brayden and Jay-Z, and Daniel like we fitted into that structure. It worked really well.

I would like you to tell us about the research Sprint. What’s super interesting is that when you run for example, design sprint, in the background there is a research sprint that’s going on. And something that you can use as a stand alone.

Research Sprint

You can totally run research sprint at different times of the project. Could you lead us through the whole research sprint and tell us exactly what it is?

Yeah. Kind of looks like. Yeah, we do a lot of it independently of design sprint. So I have one I’m doing tomorrow. I’m doing a bunch of interviews tomorrow, so. Kind of the basic steps of it are it starts out with defining the goals and the key research questions very clearly with the team. This is kind of a critical part. It is basically thinking about like, well, what do you, what do you need to learn actually and pressing here.

And so a lot of the underlying approach that I have to these things, it’s about the importance of being very specific. You know, speed is one part, but being very specific helps me be very efficient. So from the start, if I can help a team figure out what do we actually need to learn here?

And then. From there, if we know that, then we can kind of we go through a process where we’re defining who are the bullseye customers that you need to talk to. Maybe what I’ll do is describe this as there’s kind of this idea of kind of start at the end. I’ll talk about it in that context.

So if the first step is I figure out: OK, so. So what do we want to learn? What are the goals? What’s the outcome that we want to have as a result of this? And we all agree on that will be the insights that we want to have.

For example, let’s say we talk through an example. I was working on a project. That was helping a company develop a health plan, essentially for people with serious mental illness. And so the idea there is if we can help those people engage and take care of their basic healthcare needs along the way, we can keep them out of the ER. So things that we are talking going to be chronic issues. Somebody has diabetes.

If you can keep them engaged in healthcare won’t flare up and they don’t end up in the ER and have a lot of problems in the world. So there the fundamental question is, well, how do we find it?

Engage people with serious mental illness in their health care. That’s a hard problem. And we got to like the crux. And like, that’s really what it is. Find and engage. OK. So now I know. What we want to do that’s very specific.

Then the next question I think about is like, OK, well, how am I going to answer that? Like what’s the technique for gathering the data I’m going to need? Is that like a survey or interviews or what’s going to happen here?

And so for that case where I want are very actionable insights and lessons. That people have learned about engaging that population. How do people engage patients with serious mental illness and then also, how do we get some reactions to the startups offering right, if we have some concepts and things? So I know what I wanna learn. Who could give us that kind of information and and in terms of how right that’s getting packed to be through one on one interviews, right?

I have to do some discovery interviews and some concept evaluation with these prototypes of what the plan looks like. But who am I gonna talk to there? So there was like, OK well, who has experience in this case to engage with that population?

People have serious mental illness in the community. Like I don’t want to probably talk to the patient themselves, ’cause in this case, that’s not what I wanna. I wanna touch somebody who has experience with this and has the tips and lessons. So I wanna talk to experience case managers at Community health, mental health clinics.

OK, so now I’m kind of like working backwards and figuring this out. So then I have to go recruit those people, which I’m able to do, create Screener, do that work. And so all of this work is happening, but the first batch of things that I’ve just described, all these figuring out like what do I need to learn? Who do I, how am I gonna do it? Who do I need to talk to? That’s happening very early like.

Let’s say it’s a four day thing. It’s happening, let’s say Monday, and we’re hashing through that and then that gives me this time to actually do the recruiting. Now I know what I need to find, so I have to draft a Screener that’s very specific, that allows me to identify those people very precisely. And then you know, it takes me a few days to recruit.

Then on that, you know, 4th, 5th day or whatever and then conduct those interviews. Yeah. And again, because I was very specific. I can design the interviews to make sure I’m getting the information that I know I wanna get out of folks. So that’s kind of at a high level.

What that design this. With the research, Sprint looks like in those pieces just kind of how I think through it. It’s kind of from what do I want the outcome to be and then I can plan more efficiently.

A Research team at GV

So in our agency basically like I will be running the design sprint and I will be probably interviewing people on the day five, but it’s Eglé who is in the background basically looking for the customers. Running the ads, creating the screeners and all of this so it’s a lot of work. In our case it’s 2 peoples job. Is it something that you are doing like all by yourself or do you have a team? I’ve been doing this for a long time and I still just do it all pretty hands on myself. I have like you guys. I’m sure I have all my templates and my examples or I have quite a library of different kinds of things that I can draw from to help accelerate it. But it’s just me.

Finding the right testers

This is great, by the way. What do you think of all these services? Earlier you mentioned user testing for example. Do you think you can reach the right quality of customers or do you still need to, you know to run some ads or go to social networks to find them?

So it depends on whom I’m looking for. I have a huge range of companies I’m supporting. So there are times when if it’s something that’s fairly high incidence kind of group of people. I need software engineers. I need busy dads. I need you know, something like this then user interviews is actually relying pretty heavily, very have been very good at helping me find things that I thought would be very difficult to find.

I had to find people with very particular sets of medical conditions, OK, and they were able to help me find that. And what I’m doing is, again, I’m writing very careful screener questionnaires that help me identify those people. And these services are able to do that pretty fast.

For things that are much more specialized and harder to recruit you’re not going to find that way. I have to rely on some other techniques in other networks. So for example, when we need to talk to ophthalmologists. Not too long ago, like they’re not, they’re not responding to Facebook ads, right? They’re not responding to that and they don’t care about my Amazon gift card or whatever incentive because it’s not gonna work.

In those cases, we have to work through other networks that we have or the companies relationships to try to get those people. Those can be a little harder to pack into one day. You know the way I like to do this clump of interviews on that Friday. For example, it can be a little harder, so we do it as much as we can, but some of those people are just harder to reach. But I do rely quite heavily on those services.

Did you constitute a repository of very good, very good testers that you can call again, you know, or it’s always different. I don’t do that. So I don’t do that because I don’t wanna talk to the same people again. Actually, there are enough other people out there and what’s really important to me is that I want the companies that I’m working with to get in the habit of talking to new people all the time as much as possible.

So I don’t want to take the easy way out and just like go back to those same people. I think it can lead to a little bit of an echo chamber and I want them to be. I want more exposure to more of their users, so I don’t do that.

And then the other reason it doesn’t work for me is ’cause the variety the people I need to talk to is so large. So tomorrow I’m talking to people about their experience going through and getting genetic testing ’cause, they had predisposition for cancer, and next week I’m talking to people who are software engineers. It’s just not the same.

I mean there could be some overlap there, but it’s just not the same people and because I’m screening very, very specifically and carefully to make sure that I’m getting the bullseye people that we need for a study, there’s just not very much overlap.

Convincing gatekeepers about hands-on UX Research

I have a very personal question because I work a lot with corporates and the people we have, you know, they are mostly managers and some of them they don’t do too much research themselves. They don’t really know about the whole principle of iteration and all of this. And at first they’re always very surprised. They’re like. Just five tests?

It looks like a very small number and they react strongly to that and they’re like, no, the risk come back with these ideas of, like, giant surveys and all of this to do real research. So do you still have to face this kind of opposition? And how do you deal with?

Yeah, occasionally I get some of this. So in my experience people who object to that is ’cause, they just haven’t seen it yet. Quite frankly, that’s why as I was describing a lot of the way that I work is to try to be as transparent as possible so people can see it and watch the interviews. There’s nothing more convincing than getting people to actually participate and observe and hear the stories, and it becomes a little more personal.

And so when I get that kind of opposition. You know, I have some benefit, I guess I can explain the way we do it, why we do it, that we’ve done it hundreds of times. I can drop a lot of names of well-known startups that they look up to what we’ve done. And quite frankly in the end, it’s their company. But if they don’t want our help, that’s totally fine. So they’ll go do the research and figure it out in a different way.

So we’re available resource to the companies that are in our portfolio, but they don’t have to do it. But if we can get them to see it. It just opens it up, it opens up their eyes in a very different way. Because very often people feel like, oh, we’re already listening to our customers. We have feedback from customer support and our sales people are already talking to them. And you know we have a lot of input.

And then they see how we do it and then Oh yeah, that’s different. That’s not the way we’re having conversations. We’re not listening in that way. We’re pitching or we’re, you know, getting feedback from certain kinds of people who are out there, who are very vocal. This is so important.

Pitching vs listening

Can talk a bit about the difference between pitching and idea, and really like listening to the customer? That’s so important. Yeah, this comes up a lot. I said I’m training and coaching people to do this so often. They will do some interviews or I’ll do some, and then they’ll do some and I’ll give feedback and watch them.

And so people at startups, are very used to pitching, right? That’s what how you get your funding. It’s like how you got to where you are. It can be very successful or you can be very successful if you’re good at pitching your ideas. Pitching is usually about presenting like the the Rosiest picture and about convincing you fundamentally. Like I wanna convince you, this is a great idea.

But if you’re doing research where I want to see how are you seeing it? How do you react to? What do you think? Is it a good idea? What do you like or not like about this idea? And so the approach, mindset must be much more neutral. Like here’s this thing. What do you think of this thing? Here are two or three things. And like, let’s compare these different things. I don’t want it to just be one. And so. I don’t want to have express any kind of vested interest in one approach or another or whatever.

The thing is they were trying to learn about. Usually for me that’s easier ’cause it’s not. I haven’t spent the past, you know, year or two or more these companies building that thing. And so I’m kinda like, I don’t know, it’s there’s a thing they gave me and like, I can just kind of get feedback on it. But that pitch mindset is difficult for people to undo. Often fit to, because it’s such a strong habit. That’s what’s happening. As I said, having multiple prototypes. You guys know this, right? If you have two, three, four different things that you’re comparing, it takes a lot of the pressure off of that and then you’re just teasing out.

Building prototypes for Research

What are the pros and cons of these things? Are you involved in the prototypes? Like, are you behind the designers, telling them exactly what you need to do to integrate in the prototypes so that you can, test this hypothesis? What’s the link?

Yeah. So I work very closely with them to do as I said. So we’ve defined what we are trying to learn and then we think carefully about like, how are we gonna represent that so that we can kind of tease out those questions.

What we found is there’s this a lot of to answer your question, there’s a lot of back and forth. They’ll come up with the munition prototypes and we go through them and then we’re revising and I’m getting tons of feedback. And we’re kind of going back and forth. So I’m not looking over their shoulder exactly, but what we find is there’s this interesting exercise where, you know, it’s very common.

I’m sure familiar with this idea for a certain concepts where we’re testing the landing page, right. Just create like, what’s how would you pitch this idea to a customer, multiple ideas and how do you present it? And what we find is there’s this really valuable exercise which is just pushing them to actually write it down.

It is really valuable because we all know what it is. We know what we’re building like, OK, that’s fine. I’m the newbie here, so explain it to me. Write it down and then we can see. Sometimes it’s very difficult for them to just write it down and agree and have some alignment there. And so just that exercise, often documented, is very powerful to the team.

You told me when you were preparing this interview, that you have been in UX for like, what, 30 years, right? Yeah. Do you know, with all your experience, can you predict the result of a test before even running it? Just looking at the other questions, looking at the prototype like, yeah. How good are you?

Yeah, sometimes. So after doing so many of these, we do see certain patterns, and it’s things that you know it’s like basic design stuff that you guys would all know. People will write their messaging in a way that’s like all marketing speak and then you just need to be blunt and straightforward. Nobody is going to know what you’re talking about. Like there are things like that.

But what we see is that because again, I’m working with so many different companies and so many different kinds of users and topics that a lot of times, I don’t know anything about it. So it’s hard for me to predict. So it’s something that to me doesn’t make sense and then we go interview a farmer or a software developer. A trucker and the way they respond is just completely different because the context, the conventions they’re used to, the other tools they’re using it makes sense to them for some reason.

You know that I look at that, I think this is terrible this is never gonna make any sense. And somebody else who’s in that industry for, oh yeah, I totally get. I’m used to doing this that looks exactly like the thing we’ve been using for five years, right? And that’s why we do it, right? ’cause if they were relying, I mean, I’m not the target user. I’m rarely the target user for these companies. And so that’s why we do that work.

These are the best tests, right? It’s when you do that interview and you discover something, you know, that’s really you have no clue about. And you’re like, ah, I was wrong. This is so good. It’s much more fun than if I could just predict it all the time. Yeah.

When tests go wrong (anecdotes)

So do you have a story that you could share with us? Like a user test that went horribly wrong, you know, like, yeah, crazy tester or.

I guess the one that I think about that went horribly wrong. I mean horribly wrong. I won’t mention who the company was, but I was doing my research for a company. Just say it was in the mobility space and I had organized a bunch of interviews with users in Los Angeles. So I’m in Seattle. We traveled there with a gig team from the company, had set it up.

This whole thing at a Google office down there, that was a lot of work and for these folks, come in and in the course of the interviews, one interview and somebody you know. They explained to me how they’ve used this product and they’re telling me all these stories. And so there’s a thing that I do, which is ask them to show me their account.

So we have recruited very specifically people who were have been using the product. And what I found is it’s very helpful to walk through somebody’s account with them. Whether it’s their inbox or their dashboard like, look at it with me, partly because that flash of seeing it tells you a lot about a user and partly ’cause then in this case we could walk through the history. What I wanted to do was, kind of ’cause people don’t remember so well, but if they walk through the history like, Oh yeah, I did that time last week, we did this thing and then I could tell me the stories about.

So as we started doing this. People would ask them to show me their history and what became clear was that people had completely lied to me, so I don’t have the app. It’s not on this phone. It’s like it was obvious they had just downloaded it, you know, 5 minutes before. And it happened like four or five times in a row.

And I still to this day I have no idea what I did wrong. I mean other than just like, warning people ahead of time. I’m gonna ask you to look at this, right, so they wouldn’t lie. It was so horrible. ’cause you have the whole team watching. It sounds weird, but like it’s somewhat of a performance element. I’ve set this whole thing up and I’m doing it, the team is there and we spent all his time and it’s just a complete 100% plush.

We had to completely redo the study. That was that was bad. It was like no way to recover from that one ’cause. It was just. Yeah, well, wasted effort.

I had a horrible one, too. We organize these user tests. And one of the testers, she comes and she had a crazy cold. And I mean, she was blowing her nose every minute, literally. And it was really, really getting out of control. And the worst thing is that this was in person. So she was using my phone.

Basically she had my phone in her hand, in her dirty hands for like 45 minutes and all the time, she was, like, blowing her nose, pressing like that. It was just so painful. The worst time.

A Research highlight: Flatiron Health

I see a lot of questions are coming from, so this is gonna be my last question. Is there any research project that you have run that you are especially proud of, like something that really changed totally a product?

Yeah, I mean, it’s actually ’cause the way I think about these sometimes is that what? I’m looking to have impact on a product or a project, but because of my relationship with the companies that we work with. What I’m trying to do is actually have a bigger impact on the way they develop products. Because we’re investors and so we want the companies to be successful long term.

For me, each project is kind of a way to demonstrate, like, oh, look at the power of user research. If you were there, adopt this and hire people and do more of this, you will be successful. That’s when I think about things being successful is when I see people are hiring researchers later.

Um, that said, I think a personal favorite research project, especially these days, thinking about doing in person research is very appealing. I think currently about some of those projects, but work that I did with Alex Ingram at Flatiron Health was really fun ’cause we were studying how oncologists identify eligible patients and match them to clinical trials.

Which is super complicated process for them to do and they have research teams and so we did this couple times where we would do these kind of insane road trips. We ended up doing 35 hours of interviews with I think it was like 25 people at five different cancer centers and three States and four days. So it was really intense, but super fun project and leading to, you know, working on what was it really important project. So what stands out to me is something that was kind of this, this very crazy fun road trip.

How to summarize findings

When you have like any research projects, especially this one. It’s so big. How do you capture all these results and what technique you use to basically transmit the results to the stakeholders who make the decisions right to make sure that they have an impact.

Yeah. So that one is unusual. So the typical way that I do for most studies is kind of it’s UX watch party where we, we bang it out in a day and we capture the takeaways and everybody walks away. And because they have experienced it together, I don’t need to put together some giant deck and I don’t do that kind of work. And I’m lazy, so I don’t have to do that kind of work.

So for something that’s big like this, this was proceeding a design sprint that we did. And so what I needed to do was document the process. So what we were fundamentally trying to detail out was what is the process that people are doing and who’s involved with it. The question is what are their challenges? Green points along the way. Identifying these kinds of patients and so. I had to put together a very detailed description on those processes, so that then when we walked in and had a slide deck.

We walked in on Monday with a team to do the design sprint for this. I could present to the rest of the team who had not participated and joined me on this kind of workshop. They wanted to know what did this look like? And so with them when we started the design Sprint, everybody was kind of loaded up and understood what this process was. So in that case it was a slide deck, presenting all of that to everybody about what does this look like?

And Daniel Burke would make help me make these giant posters out of my documentation about the whole process. So it was kind of a set of steps and who’s the actor and what we’re trying to do. What were their goals, what were their questions, and what were their pain points. And so we had giant posters on the walls. It’s unusual for me to do it that way.

I just want to remind the people who will ask the question. We have a nice bottle of wine to win for them. So at the end, you can I like the best question from from our audience. So the first question. It’s gonna be Olivier. You ready? Yep, I know. So I’m designer from Belgium and my question was to rebound about the research field that you prepared actually.

The best way to teach people about UX

You mentioned that you do it alone at the moment. How would you do to help people who had never worked with UX would discover, use profile in their teams, how would you do to help them to create a surface? How do you help them help me during all the recruiting and all of that process y. How would you build maybe the protocol. These people might think that most of the time they are not aligned and so how would you make them understand?

The people in my case in Belgium have never heard about UX and discovered the profiling some companies still there.

What I’ve learned for myself and have been reminded by other people over and over, is the best way to teach people how to do this is to show them. And so I haven’t found a better way to do it. I’ve tried all these different kinds of classes and workshops and fundamentally I need to just have them play along with me. Like we’re gonna do it together. And I’m gonna go through this whole process with you and ask you a bazillion questions.

Turn my researcher skills onto the team to figure out. Help them figure out. What do you actually really need? What do you really trying to learn? And so that’s a lot of what I do. With their early step is essentially interviewing the team and figuring out like, no, no, no. What do you really wanna know? ’cause, they’ll often show up and so here’s what we want to learn, and we’ll kind of here’s what we need.

And we’ll kind of talk through it and we get to a point where I will ask this question, which is not like – what’s actually keeping you up at night. Like oh, that’s this other thing, like, oh, that’s interesting. Let’s talk about that like, OK, what is that issue? How can we help you?

But then we go through it and I do the research with them, just in this very open way. So they see me do it ’cause I haven’t and maybe I’m just not a good teacher but I haven’t found another way to get.

Would you maybe ask other stakeholder interview to better understand the concept the role of project. Yeah. Usually we start with a group in a room and so usually we promote ’cause. workers are set up. If we sense that there is not alignment like that, there’s something not quite In Sync on this team we will then sometimes revert to doing stakeholder interviews to figure out like, wait, what’s going on here.

Because sometimes there are other things that are not about the research, right? It’s about team dynamics and we realize there are a lot of other things that we are assessing when we’re working with the team. To figure out is this actually a good time to do this research.

A lot of things that we’ve also learned is there are times that I could do like the most awesome research study and it’s going to have no impact whatsoever. So we try to avoid that. So for example, if we realize the team is not aligned at all, what are the things that are keeping them up at night are completely different. They’re all going different directions.

Not a good sign if they don’t actually have resources to work on and build whatever. The thing is that we’re working on like that designer we’re trying to hire. A designer and engineer to work on this project. We wanna do the research now. Yeah, we’re not gonna do it now, ’cause, let’s wait till those people are here, and then we’ll go through this and have everybody aligned.

Otherwise, this is gonna sit on the shelf. New people show up. It’s not gonna work. So if the right ingredients aren’t in place and we can sense that now and figure that out pretty quickly. We won’t even kind of move forward. You will come in later, will kindly tell them that we’re not going to do it yet.

Blending Anthropology and UX

Hi Michael, I am joining all of you from Dubai and I’m an avid fan of the book Sprint since the time I got my hands laid on that book, my PhD journey has taken a next level. I am just a humble university lecturer and I have a lot of passion for sprints as well as ethnography. I would like to ask you about your take on blending etnographic techniques with design sprints.

When I’m taking workshops or when I’m teaching, I’m giving away a lecture. Most of my MBA students who are working, they are managing directors, CEOs and startup owners and investors. They’re much interested with a lot of pictorial research. And I’ve seen a lot of them coming in with pictures and you know we attended that, we saw the consumer touching the products like that. Or interacting with a particular product like that and we want to work on that part.

The way they’re holding the bottle or the way they’re sitting in the car. So I was much intrigued by ethnographic techniques and just until recently in one of my projects with IKEA, I was able to well blend both techniques, but I haven’t really gotten a breakthrough for that.

So I thought maybe you could let me know what are your takes on really blending techniques and design sprints. I’m much interested about phase four prototyping.

To me, it gets back to this idea of what is the fundamental question you’re trying to answer. What’s the outcome you’re aiming for and what’s the best way to deliver that? What’s the best, most efficient way to deliver that? At least the burst bite size pieces, right? A lot of this is about iterating and kind of doing it in chunks.

And so sometimes you just have to be there to see it. And when you do then I do that. So like this example I described of our traveling to these multiple States and studying oncologists, we just needed to go talk to them.

And part of that was because the oncologists are not coming to me like they’re not making time for me. And part of it is I needed to see. I want to see all the spreadsheets they were using and like all the posts. What is this messy process really look like and that’s harder to do online. Now there are a lot more tools where I can do that remotely. So sometimes I’m again because the way I’m doing this I’m balancing. My desire to be on site with doing it in a way that everybody can see me doing it.

So they’re kind of competing goals there, right? So I guess the short answer, is those techniques can be very powerful. They can be very good to help you tell the stories and to help bring other people in and have them see the context. But it’s, Ethnographic techniques are not just about pictures, right? It’s about stories and about the story stories and having those artifacts. And so you need both of those.

Just the pictures, like that’s OK and there’s a lot of waiting. I use disk out and some of these other things. But I lean very heavily on getting people to tell the stories and hearing the explanations. Because like how they’re holding it fine and depends on if you’re just doing some basic industrial design work, the work that I’m usually doing is much more.

What’s the value proposition for this business, and so that’s usually requires much more than like, oh, I can see how they’re sitting in the chair. There’s anything wrong with that? It’s just not the kind of projects I’m typically working on now.

That’s what I mentioned was quite amateur, but that did strike me that, you know, maybe. So I’m still getting there. I’m still trying decode. What’s that maybe in how and what to do. But yeah, at least it gives me a way ahead.

The thing that I would listen for, there are a lot of techniques now that make it very easy for people to avoid going and actually talking to the people that they need to talk to. And some of these are great tools, but they need to be used with other things. But if people say, well, we’re, you know, like I said before, you know, we’re using analytics, we’ve these other things. We have a lot of input from our users.

Sometimes you need to just like, leave your office or get out from behind your desk and actually go talk to these people. Be very specific, who are the people, but you actually have to go listen and watch.

And sometimes people are looking for like ways to, you know, shortcuts around that. And so what I tried to do is make it a streamlined as possible to do that work. But you kinda gotta do it. Sorry not able to do short answers.

Having empathy but avoiding bias

I think we have one last question from Andrea. Hi, Michael. Hi, Steph. I work in museums and galleries and staff knows I’ve been running design sprints for a couple of years now. But a lot of the work I do is about emotional engagement, but also economic engagement, with heritage, and I wondered about where you’ve been working, particularly with medicine.

How when you’re building empathy during an interview, do you make sure you don’t kind of melt too much and end up ALS, feeling that you’re leading for tested down a particular root of response? How do you manage to have empathy but also get clean information that you need?

It’s an issue, but it’s a good question. What I try to do is. It’s about remaining neutral. Right. And that’s what you’re kind of mentally trying to do is remain neutral and so? I think it’s important to make the noises and things where you’re acknowledging what somebody is saying. Acknowledging it and encouraging, but not necessarily reinforcing. That’s the line that I’m usually trying to ride.

You know making a simple sound like I’m trying to reflect back to them like that. Sounds like that was hard for you is different than. When that happens, right? Like so. I’m not trying to to. I’m trying to be understanding and encouraging, but not necessarily on their team to reinforce that.  Like, yeah, we hate when that happens. That’s really hard. That happened to me too, like I’m not. It’s just. I’m an observer. Trying to be kind. Uhm, in that way.

So that’s kind of what’s in my head is like, what’s a neutral acknowledgement so that the person knows that I’ve heard them. They wanna usually feel heard and like I’m understanding. So recapping and feed it back to them like oh, it sounds like you’re feeling this way about that.

In the in, the leaders say yeah, that’s how I felt like. No, no. Like you’ve misunderstood. Know correct me, which is perfect, right. But I’m not trying to add any kind of judgment or reinforcement to it.

And if they ask you for more detail around the question, again not adding detail, which then seems to be persuasive. I mean I will rephrase a question because sometimes I’m, I just would have asked it poorly, right? And so I’ll just take a step back and just try to ask it again.

We’re kind of stepping back to what we were talking about and maybe try to like rebuild ’cause some of it is managing and maintaining the rapport you have. And so sometimes if somebody starts backing at a question that I’ve asked either, maybe they just didn’t understand me, which is fine, or I’ve said something that clearly has touched a nerve in some way. And so I need to kind of backtrack a little bit and rebuild.

Add back, you know, put some more deposits into the Rapport building. And so I’ve kind of do that by like. I’m so sorry. I’m you know. I’m a little flustered, you know? Take some of the the responsibility to why it’s difficult, you know. Thank you so much for doing this. It is really helpful. This is kind of hard for me. I’m, you know, new with this, whatever.

But try to rebuild that trust ’cause that’s a lot of what you’re trying to establish in that context that, that relationship when it’s a sensitive thing.

The other, the other thing that I’ll say about this one, it’s about building empathy. So I have found in the past two years, I do it less now. There’s been this very effective thing that I found that I can do at the very start of an interview that will jump the report very quickly pretty far, which is that I’ll, you know, do my intro in which I chat about where you calling from or whatever and just like kind of the friendly hi chat.

I’m not a crazy person like establishment at the very beginning and then I’ll explain what I’m doing. And I’ll say, you know, before we jump in I just wanna check in with you, ’cause I know the past year or two years of COVID is has been really hard for people. Like, how are you doing?

And what I found is that i has been. And I’m trying to do it in a genuine way, or if it’s not, yeah, but it helped me jump pretty fast with people actually ahead. Because people will tell you. You know, hard things sometimes. Often they’re like, Oh well, I mean, this thing happened, you know, it ends like, OK, we’re deep into it pretty fast. And so just expressing that genuine but pretty neutral concern about that as a person has helped a lot

What’s the future of User Research?

Thank you so much, Michael. This is going to be me for real my last question, how do you see the future of the future future research? Wow. That’s your last question.

The future of user research my I guess in Michael’s fantasy world, the future is that it’s just gets done more and it gets done with a better quality. What I’ve seen in the time that I’ve been doing it and certainly achieving let’s say in the last 12 years. It has gotten much more common among consumer companies.

You all can buy the book, and you can do this stuff. And like, it’s just more common, more familiar how to do it. Enterprise companies were behind, and it wasn’t so common as much, people who were building those kinds of tools. But those things have become more consumerized.

Slack is a great example, right? Were design becomes very important in these big enterprise kind of products. And so they are doing more of this kind of work. They’re incorporating more of these techniques.

And the place where we still see that we as a team have disproportionate impact is on health care projects. Where we work with brilliant people solving very, very hard problems in US health care system. And they’re not used to these techniques as much.

And so we can just have huge impact there in a way that we used to on some of these other kinds of companies where people like, Oh yeah, we kind of know how to do some of that stuff.

So what I’m hoping is that continues and it just becomes more and more common and the quality of it is more and more prevalent. For some of these other, much harder kinds of problems that we all have to deal.

It was absolutely amazing having you on the show and giving your answers to all these questions.