Why we run tests and trials

At the beginning of the year I worked on a project with 3 of our Food stores. We were looking at using a third party to replace our current home delivery service. I’m a user researcher so I spoke to colleagues and customers about their experiences with the product we were trying out.

One day, I received an email from one of the store managers asking me to visit his store. He said they’d been having technical difficulties and they’d like some support.

But my job isn’t to provide technical support – it’s to understand why someone needs it.

It dawned on me that this store manager thought the point of putting a new product in his store for a short time was to see how well the team could use it – as if we were examining the team’s ability to fit in around this new thing, rather than seeing how well the new thing met colleague and customer needs. His email suggested he thought trials are a top-down, ‘you-must-use-this-thing’ situation, not something collaborative or something that his team could and should influence.

Setting things straight

Neither tests nor trials are about seeing whether the people using the new thing have the skills or expertise to use it. They’re about finding out whether the new thing is good enough and meets the needs of the people who’ll use it. Both help us make sure that the thing we’re building, or buying in the case of trials, will benefit the business and meet user needs.

When we test

If we’re designing something to solve a problem for colleagues or customers, for example a website to help Food store colleagues find out how to do something in their store quickly and easily, it’s essential we speak to those people and see them use the thing we’re designing as soon as possible. It helps us make sure we build the right thing.

We start with research so that we understand the problem or colleague or customer need; we make assumptions about how we could fix the problem or answer the need, and we build the simplest thing we can. Then we get it in front of users – the people we’re building it for. We make improvements based on their feedback and get a new version back in front them again for more feedback. And so on.

We focus tests on either a specific set of features or specific points of interactions in a service. By testing different things continuously, we begin to understand which features work and which don’t. And when we start to get things right, we invite more people to use it.

When we use trials

Sometimes there’s a product that fits our business needs already on the market so there’s not always a significant benefit in developing our own. Instead, we ask a small number of colleagues and/or customers to try this product so we can see how it fits within the service Co-op wants to provide. If it’s a good fit, we’ll make it available to more colleagues and/ or customers.

A participant’s role is so important

My experience is that many participants don’t understand the purpose of tests and trials. I fear that if they are told by someone at head office to use a thing for x-amount of time, their feedback might not be completely honest. I think we can work harder to help users throughout the organisation understand why we test and trial and the importance of their feedback.

Starting here…

Honest feedback is useful. Sugar-coated feedback isn’t

If your Co-op workplace is chosen to be part of a test or trial, it’s because we want to learn from you, our users. Building or finding the right thing is a conversation. It’s 2-way. Good design is collaborative meaning that you, our users, shape what we build. Everyone – managers, team leaders, assistants – can, and should, influence this.  

All feedback is valuable, even if the feedback is “this new thing you’ve asked us to use makes my work life way more difficult. And it always breaks”. For the Digital team to do our jobs properly we need to be aware of the problems so we can go back and build or buy something that you’ll actually find useful.

Tips for digital teams testing or running trials

Communicate clearly and regularly with the users you’re working with. Be clear about:

  • the purpose of tests and trials in general (maybe link to this post)
  • what you’re testing or trialling with them
  • why you’re testing or trialling ie, the difference this new product could make to the bigger picture
  • how important honest feedback is and the role it’ll play in shaping a product or service
  • that there are no right and wrong answers
  • the anonymity users can expect in any evidence we share – we don’t report back to management about specific users

Better communication will mean we’re all on the same page. And that’ll mean we’ll build better services.

Let us know how we can run tests and trials better in the comments.

Gillian MacDonald
User researcher