Vicki Riley on user research at Co-op Digital

We’re recruiting user researchers.

If you want to help digital teams build the right thing, and if learning about how people behave and why sounds interesting, you might be a good fit. Have a look our job description for more details.

Our user research team come from really varied backgrounds. Here’s Vicki’s story.

(Transcript) Vicki Riley: I’ve been a user researcher for just coming up to a year now, I’ve been at the Co-op for 2 years. Originally I worked on the analytics and optimisation team. That involved looking at the the data so Google Analytics to find out what people were doing on our websites. But being a user research now I can really delve into why people are behaving in a certain way and shape the products that we’re building based on that.

User research is about understanding the behaviour of people. So rather than asking people what they would do we would prefer to observe them. So with colleagues that could involve shadowing in-store, spend a day in the life of a colleague who works in one of our food stores to really get to the bottom of what they’re trying to do, what their pain points are and shape what we’re building based on what we’re seeing rather than just what we’re hearing.

A huge part of user research is kind of getting out of the building, going out to the places that our users are working or spending time. There are a lot of assumptions in Angel Square or in any business there are assumptions and without speaking to the people who use your services or use your products you’ll never really find out what’s happening, you’ll never really get to the bottom of it.

I love working with designers and interaction designers, content designers. They’re really involved in the research and they bring a different perspective to things. They have different backgrounds, different experiences, different knowledge so it
really helps to have a diverse group of people doing the research analysing the
research.

I’m learning something new pretty much every week because I’m working on different teams and moving around a lot learning from people who’ve done user research but 10, 20 years so for my personal development it’s been brilliant this last year I’ve learned more than I have in my entire career.

I’ve wanted to work at the Co-op for a long time ever since I’ve finished University. They’re a company that really makes a difference in the community and I think user research has an opportunity to shape that, to shape what we do in the future.

Vicki Riley
User researcher

 

Simon Hurst: life as a user researcher at Co-op Digital

(Transcript) Simon Hurst: I’m Simon Hurst, I’m a user researcher here at the Co-op working on Membership.

So, Membership is key to the Co-op, it’s a lot what the Co-op is. So my role entails sort of challenging assumptions that the business might have about customers or about members. So it’s understanding how people want to engage with the Co-op in the 21st century. I’m a fairly experienced user researcher now so I’ve been through digital transformation in government. I was originally first worked on the first Department for Work and Pensions service that went live.

The ability to go and do that all over again with Co-op Digital and to help a lot of
people who were coming to user research quite new and to help them along. So I think one of the best parts of my job is mentoring people as well, so I’m sort of mentoring someone at the minute and I’ve just trained someone as a user research.

So being able to share sort of things I’ve learned as user researcher with other people and equally we’ve got a good mix of people from different digital backgrounds. So even amongst the community of user researchers, there’s people with different skill sets to me I can learn from as well. So it’s just a really good mix. And stuff we’re trying to do is genuinely trying to improve people’s lives and help people.

The things I’m looking forward to right now are really starting to influence more and more how much user research is listened to, so really getting it properly embedded now. So we’ve got the roots there we now need to build on that and so it’s making sort of links in the wider Co-op to sort of share user research findings as opposed to sort of people directly on the product teams. And trying to find other user researchers to come and join us, who can join in with that and helping to develop user researchers here.

Co-op was the only other job I’ve ever applied for apart from government in 20 years, since I left college, so I have no plans on going anywhere so it’s it’s really nailing it here I think is what I want to do.

Simon Hurst
User researcher

Find out about opportunities to work with us.

Making the move to user research

User research helps make products and services that work for the people who use them. It takes loads of different forms including lab sessions and interviews, onsite visits and analysing data but, regardless of its form, it must be present throughout the design process. And even after the thing is live.  

Moving into a user research role

I’ve worked at the Co-op for just under 2 years. I originally joined the Analytics and Optimisation team, but for the last 10 months I’ve been a user researcher at Co-op Digital.

User research really appealed to me because it’s about listening to users as well as looking at data. My old role was heavy on the quantitative side of things: I evaluated data collected from user journeys and improved the experience for users. Good user researchers consider both quantitative and qualitative research so I’ve been working on my qualitative research skills. Now I feel even better equipped to help teams design the right thing.

User research at Co-op Digital

I applied for a user research role after seeing the work that our now Head of User Research James Boardwell and the team were doing with wills. The multidisciplinary team was working in an agile way to build a digital service to make it simpler and quicker for Co-op customers to get a will.

I saw how both data and qualitative research fed into the design process. User research formed the basis for discussions and the team could test ideas, put them in front of people and iterate them quickly. The whole team came to user research sessions so that everyone saw first-hand how users behaved when we put prototypes in front of them and asked questions. The team analysed the themes that came out of the sessions together which meant that everyone had a similar idea about where the design was heading.

Everything moved so quickly and decisions were based on things that the team had seen or heard. At each show and tell the team knew so much more than the week before – they’d added another piece to the jigsaw. They’d started small and built the right thing, quickly. I loved watching their progress.

My first taste of user research

Supporting James was my first experience as a user researcher. I joined the Wills team during a sprint focused on increasing the number of people making it to the confirmation page. I already had good experience in this from my previous role but here I also got to see James talking to people, showing them the prototype and doing qualitative research in lab sessions.

The data I’d collected told us what was happening with real people using the website, and James’ conversations with people told us why it was happening. The data showed that the exit rate from the ‘Your details’ page was disproportionately high. Qualitative research told us that people felt uncomfortable giving their personal details before knowing exactly what the service offered. Changing the order of the pages, so, giving the user more upfront information, resulted in more people completing the form.

The 2 kinds of insight complemented each other. You can read more about this in James’ post, User research and sample sizes.

Learning how user research works in a product team

I spent 6 months working with the Membership team too. User research gives us the chance to test things to make sure we’re doing the right thing for users. This way, any decisions we make are better informed.

Working on Membership opened my eyes to other ways of doing research too. It’s not just about interviews. We:

  • used qualitative website feedback and quantitative analytics to compare what users told us with what they actually do
  • visited stores to find out what our members and customers talk to colleagues about
  • spoke directly to members

It’s about analysing all available resources.

Leading my first project

Photograph of a user research session. Shows 10 members of the Electrical discovery team talking about and analysing what they've seen in the user research lab.

For the last 2 months I’ve been leading the user research on a discovery in our Electrical business. This project has helped me learn a lot about how user research informs service design through techniques like customer journey mapping and service blueprints. Service design is a fairly new way of thinking at Co-op Digital so leading this project was sometimes challenging, but we’ve got a strong user research community at Co-op Digital and support and advice was always available if I needed it.

Hard work, but worth it

I think the biggest challenge for a user researcher is using all of their observations and data to find the need, and working with the team to translate these into things we can work on.

User research encourages teams to take a more balanced approach to design. It changes the way teams work and brings the business and digital sides of things together. It’s a way to stop people jumping to conclusions about what’s ‘right’ because we’re using evidence to make decisions. And ultimately, that’s going to work better.

If learning about how people behave and why sounds interesting and you want to help teams build the right thing, quickly and cost-effectively, get in touch with James Boardwell or leave a comment on the blog.

Vicki Riley
User researcher

Small is beautiful. User research and sample sizes

At the Co-op, we use both qualitative and quantitative approaches to make decisions about products. This post is about doing qualitative research with a small-ish numbers of users – and why that’s useful.

As a rule of thumb, qualitative and quantitative approaches are useful for different things:

  • If you want to understand the scale of something (such as how many users do X or Y, or how much of something is being used), use quantitative methods, like surveys.
  • If you want to understand why people do something and how they do it, qualitative methods such as interviews or seeing how users behave with a given task (user tests) are better.  

User research isn’t a one off event. It’s a process. By researching with a handful of users at a time, iteratively, and supported by data on user behaviour, we build better digital products and services.

How many users are enough?

We don’t need to observe many users doing something to identify why they’re behaving a certain way. Jakob Neilsen, a usability expert, found through research with Tom Landauer that 5 users is sufficient. More than 5 and your learning diminishes rapidly and “after the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new”. Here’s Neilsen’s graph of these diminishing returns:

Graph shows percentage of usability problems found on the y axis and number of test users on the x axis. the graph sows that we find 100% of usability problems with a relatively small number of test users.

Source: Jakob Neilsen

Analysing user data and user research findings are complementary in developing digital products and services. Data can help identify issues to then test with users, but it can also run the other way. In user research at the Co-op, we’ll often see things while doing user research which we’ll then investigate with data. It works both ways.

Screen Shot 2017-03-09 at 11.26.18

There’s cumulative value in cycles of research

The cycle of user research shown in the diagram is how product teams work at the Co-op. We typically iterate in weekly or fortnightly cycles.

For example, the Membership team has a rhythm of fortnightly cycles. These are often focused on discrete aspects of Membership. These research cycles accumulate learning over time. They create an understanding of Membership and of users needs. Cumulatively, this gives clarity to the whole user journey.

During the last 10 months, the Membership team have surveyed 674 users and interviewed 218. The value of this research accrues over time. The team has learnt as they’ve developed the service and iterated on the findings, getting to know far more than if they’d done the research in one block of work.

That’s why observing relatively few users doing a task, or speaking to a handful of users explaining something they’ve done, is enough to provide confidence in iterating a product and to continue to the next test. This is especially true when user research is used together with data on user behaviour and even more so when it’s done regularly to iterate the product.

Error-prone humans are there in quantitative research too

It’s not uncommon for people to give more weight to quantitative data when they’re making decisions. Data is seen as being more factual and objective than qualitative research: “you only spoke to 10 people, but we have data on thousands…!”

Data hides the error-prone human because humans are invisible in a spreadsheet or database. But even though they’re hidden, the humans are there: from the collection of the data itself and the design of that collection, to the assumptions brought to the interpretation of the data and the analysis of it.

All data is not the same

Survey data, based on responses from users, is distinct from data collected on behaviour through Google Analytics or MixPanel. Poor survey design produces misleading insights.

Getting useful behavioural data from a user journey is dependent on setting up the right flows and knowing what to track using analytics software.  Understanding what constitutes ‘good’ data and how to apply it is something we’re working on as a community of user researchers at the Co-op.

Research is a process, not a one-off

Digital product teams usually have a user researcher embedded. They can also draw on the skills and experience of the conversion and optimisation team and their quantitative and statistical skills and tools. The user researcher gets the whole product team involved in user research. By doing this, they gain greater empathy for and understanding of their users (and potential users).

This diagram shows some of the methods we use to help us make good product decisions and build the right thing to support users:

Screen Shot 2017-03-09 at 11.36.18

As user researchers our craft is working out how and when to deploy these different methods.

Part of the craft is choosing the right tool

Let’s take an example from a recent project I was involved in, Co-op wills, where we used both quantitative and qualitative research.

We had customer data from the online part of the service and analysed this using a tool called MixPanel. Here’s part of the journey, with each page view given a bar with corresponding number of visitors:

Screen Shot 2017-03-09 at 15.38.11

From this, we could determine how many users were getting to a certain page view of the wills service, and where they were dropping out.

The data let us see the issue and the scale of what’s happening, but it doesn’t give us a sense of how to resolve it.

What we didn’t know is why people were dropping out at different parts of the journey. Was it because they couldn’t use the service, or didn’t understand it, or because they needed information to get before they could complete?

To help us understand why people were dropping out, we used user data to create hypotheses. One of our hypotheses was that “users will be more likely to complete the journey if we start capturing their intent before their name and email address” ie, show them the service before asking them to commit.

Through user research with small numbers of users we found a series of different reasons why people were behaving in ways Mixpanel had showed us, from confusion over mirror wills to uncertainty about what the service involved, to requiring more information.

We only got this insight through speaking to, and observing users, and getting this insight allowed us to design ways to fix it.

It’s not an exact science – and that’s OK

Research is not an exact science. Combined with user data, user research is a process of understanding the world through the eyes, hands and ears of your users. That’s why it’s central to the way we’re building things at the Co-op.

James Boardwell
User researcher