Making the move to user research

User research helps make products and services that work for the people who use them. It takes loads of different forms including lab sessions and interviews, onsite visits and analysing data but, regardless of its form, it must be present throughout the design process. And even after the thing is live.  

Moving into a user research role

I’ve worked at the Co-op for just under 2 years. I originally joined the Analytics and Optimisation team, but for the last 10 months I’ve been a user researcher at Co-op Digital.

User research really appealed to me because it’s about listening to users as well as looking at data. My old role was heavy on the quantitative side of things: I evaluated data collected from user journeys and improved the experience for users. Good user researchers consider both quantitative and qualitative research so I’ve been working on my qualitative research skills. Now I feel even better equipped to help teams design the right thing.

User research at Co-op Digital

I applied for a user research role after seeing the work that our now Head of User Research James Boardwell and the team were doing with wills. The multidisciplinary team was working in an agile way to build a digital service to make it simpler and quicker for Co-op customers to get a will.

I saw how both data and qualitative research fed into the design process. User research formed the basis for discussions and the team could test ideas, put them in front of people and iterate them quickly. The whole team came to user research sessions so that everyone saw first-hand how users behaved when we put prototypes in front of them and asked questions. The team analysed the themes that came out of the sessions together which meant that everyone had a similar idea about where the design was heading.

Everything moved so quickly and decisions were based on things that the team had seen or heard. At each show and tell the team knew so much more than the week before – they’d added another piece to the jigsaw. They’d started small and built the right thing, quickly. I loved watching their progress.

My first taste of user research

Supporting James was my first experience as a user researcher. I joined the Wills team during a sprint focused on increasing the number of people making it to the confirmation page. I already had good experience in this from my previous role but here I also got to see James talking to people, showing them the prototype and doing qualitative research in lab sessions.

The data I’d collected told us what was happening with real people using the website, and James’ conversations with people told us why it was happening. The data showed that the exit rate from the ‘Your details’ page was disproportionately high. Qualitative research told us that people felt uncomfortable giving their personal details before knowing exactly what the service offered. Changing the order of the pages, so, giving the user more upfront information, resulted in more people completing the form.

The 2 kinds of insight complemented each other. You can read more about this in James’ post, User research and sample sizes.

Learning how user research works in a product team

I spent 6 months working with the Membership team too. User research gives us the chance to test things to make sure we’re doing the right thing for users. This way, any decisions we make are better informed.

Working on Membership opened my eyes to other ways of doing research too. It’s not just about interviews. We:

  • used qualitative website feedback and quantitative analytics to compare what users told us with what they actually do
  • visited stores to find out what our members and customers talk to colleagues about
  • spoke directly to members

It’s about analysing all available resources.

Leading my first project

Photograph of a user research session. Shows 10 members of the Electrical discovery team talking about and analysing what they've seen in the user research lab.

For the last 2 months I’ve been leading the user research on a discovery in our Electrical business. This project has helped me learn a lot about how user research informs service design through techniques like customer journey mapping and service blueprints. Service design is a fairly new way of thinking at Co-op Digital so leading this project was sometimes challenging, but we’ve got a strong user research community at Co-op Digital and support and advice was always available if I needed it.

Hard work, but worth it

I think the biggest challenge for a user researcher is using all of their observations and data to find the need, and working with the team to translate these into things we can work on.

User research encourages teams to take a more balanced approach to design. It changes the way teams work and brings the business and digital sides of things together. It’s a way to stop people jumping to conclusions about what’s ‘right’ because we’re using evidence to make decisions. And ultimately, that’s going to work better.

If learning about how people behave and why sounds interesting and you want to help teams build the right thing, quickly and cost-effectively, get in touch with James Boardwell or leave a comment on the blog.

Vicki Riley
User researcher

Small is beautiful. User research and sample sizes

At the Co-op, we use both qualitative and quantitative approaches to make decisions about products. This post is about doing qualitative research with a small-ish numbers of users – and why that’s useful.

As a rule of thumb, qualitative and quantitative approaches are useful for different things:

  • If you want to understand the scale of something (such as how many users do X or Y, or how much of something is being used), use quantitative methods, like surveys.
  • If you want to understand why people do something and how they do it, qualitative methods such as interviews or seeing how users behave with a given task (user tests) are better.  

User research isn’t a one off event. It’s a process. By researching with a handful of users at a time, iteratively, and supported by data on user behaviour, we build better digital products and services.

How many users are enough?

We don’t need to observe many users doing something to identify why they’re behaving a certain way. Jakob Neilsen, a usability expert, found through research with Tom Landauer that 5 users is sufficient. More than 5 and your learning diminishes rapidly and “after the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new”. Here’s Neilsen’s graph of these diminishing returns:

Graph shows percentage of usability problems found on the y axis and number of test users on the x axis. the graph sows that we find 100% of usability problems with a relatively small number of test users.

Source: Jakob Neilsen

Analysing user data and user research findings are complementary in developing digital products and services. Data can help identify issues to then test with users, but it can also run the other way. In user research at the Co-op, we’ll often see things while doing user research which we’ll then investigate with data. It works both ways.

Screen Shot 2017-03-09 at 11.26.18

There’s cumulative value in cycles of research

The cycle of user research shown in the diagram is how product teams work at the Co-op. We typically iterate in weekly or fortnightly cycles.

For example, the Membership team has a rhythm of fortnightly cycles. These are often focused on discrete aspects of Membership. These research cycles accumulate learning over time. They create an understanding of Membership and of users needs. Cumulatively, this gives clarity to the whole user journey.

During the last 10 months, the Membership team have surveyed 674 users and interviewed 218. The value of this research accrues over time. The team has learnt as they’ve developed the service and iterated on the findings, getting to know far more than if they’d done the research in one block of work.

That’s why observing relatively few users doing a task, or speaking to a handful of users explaining something they’ve done, is enough to provide confidence in iterating a product and to continue to the next test. This is especially true when user research is used together with data on user behaviour and even more so when it’s done regularly to iterate the product.

Error-prone humans are there in quantitative research too

It’s not uncommon for people to give more weight to quantitative data when they’re making decisions. Data is seen as being more factual and objective than qualitative research: “you only spoke to 10 people, but we have data on thousands…!”

Data hides the error-prone human because humans are invisible in a spreadsheet or database. But even though they’re hidden, the humans are there: from the collection of the data itself and the design of that collection, to the assumptions brought to the interpretation of the data and the analysis of it.

All data is not the same

Survey data, based on responses from users, is distinct from data collected on behaviour through Google Analytics or MixPanel. Poor survey design produces misleading insights.

Getting useful behavioural data from a user journey is dependent on setting up the right flows and knowing what to track using analytics software.  Understanding what constitutes ‘good’ data and how to apply it is something we’re working on as a community of user researchers at the Co-op.

Research is a process, not a one-off

Digital product teams usually have a user researcher embedded. They can also draw on the skills and experience of the conversion and optimisation team and their quantitative and statistical skills and tools. The user researcher gets the whole product team involved in user research. By doing this, they gain greater empathy for and understanding of their users (and potential users).

This diagram shows some of the methods we use to help us make good product decisions and build the right thing to support users:

Screen Shot 2017-03-09 at 11.36.18

As user researchers our craft is working out how and when to deploy these different methods.

Part of the craft is choosing the right tool

Let’s take an example from a recent project I was involved in, Co-op wills, where we used both quantitative and qualitative research.

We had customer data from the online part of the service and analysed this using a tool called MixPanel. Here’s part of the journey, with each page view given a bar with corresponding number of visitors:

Screen Shot 2017-03-09 at 15.38.11

From this, we could determine how many users were getting to a certain page view of the wills service, and where they were dropping out.

The data let us see the issue and the scale of what’s happening, but it doesn’t give us a sense of how to resolve it.

What we didn’t know is why people were dropping out at different parts of the journey. Was it because they couldn’t use the service, or didn’t understand it, or because they needed information to get before they could complete?

To help us understand why people were dropping out, we used user data to create hypotheses. One of our hypotheses was that “users will be more likely to complete the journey if we start capturing their intent before their name and email address” ie, show them the service before asking them to commit.

Through user research with small numbers of users we found a series of different reasons why people were behaving in ways Mixpanel had showed us, from confusion over mirror wills to uncertainty about what the service involved, to requiring more information.

We only got this insight through speaking to, and observing users, and getting this insight allowed us to design ways to fix it.

It’s not an exact science – and that’s OK

Research is not an exact science. Combined with user data, user research is a process of understanding the world through the eyes, hands and ears of your users. That’s why it’s central to the way we’re building things at the Co-op.

James Boardwell
User researcher