Why we run tests and trials

At the beginning of the year I worked on a project with 3 of our Food stores. We were looking at using a third party to replace our current home delivery service. I’m a user researcher so I spoke to colleagues and customers about their experiences with the product we were trying out.

One day, I received an email from one of the store managers asking me to visit his store. He said they’d been having technical difficulties and they’d like some support.

But my job isn’t to provide technical support – it’s to understand why someone needs it.

It dawned on me that this store manager thought the point of putting a new product in his store for a short time was to see how well the team could use it – as if we were examining the team’s ability to fit in around this new thing, rather than seeing how well the new thing met colleague and customer needs. His email suggested he thought trials are a top-down, ‘you-must-use-this-thing’ situation, not something collaborative or something that his team could and should influence.

Setting things straight

Neither tests nor trials are about seeing whether the people using the new thing have the skills or expertise to use it. They’re about finding out whether the new thing is good enough and meets the needs of the people who’ll use it. Both help us make sure that the thing we’re building, or buying in the case of trials, will benefit the business and meet user needs.

When we test

If we’re designing something to solve a problem for colleagues or customers, for example a website to help Food store colleagues find out how to do something in their store quickly and easily, it’s essential we speak to those people and see them use the thing we’re designing as soon as possible. It helps us make sure we build the right thing.

We start with research so that we understand the problem or colleague or customer need; we make assumptions about how we could fix the problem or answer the need, and we build the simplest thing we can. Then we get it in front of users – the people we’re building it for. We make improvements based on their feedback and get a new version back in front them again for more feedback. And so on.

We focus tests on either a specific set of features or specific points of interactions in a service. By testing different things continuously, we begin to understand which features work and which don’t. And when we start to get things right, we invite more people to use it.

When we use trials

Sometimes there’s a product that fits our business needs already on the market so there’s not always a significant benefit in developing our own. Instead, we ask a small number of colleagues and/or customers to try this product so we can see how it fits within the service Co-op wants to provide. If it’s a good fit, we’ll make it available to more colleagues and/ or customers.

A participant’s role is so important

My experience is that many participants don’t understand the purpose of tests and trials. I fear that if they are told by someone at head office to use a thing for x-amount of time, their feedback might not be completely honest. I think we can work harder to help users throughout the organisation understand why we test and trial and the importance of their feedback.

Starting here…

Honest feedback is useful. Sugar-coated feedback isn’t

If your Co-op workplace is chosen to be part of a test or trial, it’s because we want to learn from you, our users. Building or finding the right thing is a conversation. It’s 2-way. Good design is collaborative meaning that you, our users, shape what we build. Everyone – managers, team leaders, assistants – can, and should, influence this.  

All feedback is valuable, even if the feedback is “this new thing you’ve asked us to use makes my work life way more difficult. And it always breaks”. For the Digital team to do our jobs properly we need to be aware of the problems so we can go back and build or buy something that you’ll actually find useful.

Tips for digital teams testing or running trials

Communicate clearly and regularly with the users you’re working with. Be clear about:

  • the purpose of tests and trials in general (maybe link to this post)
  • what you’re testing or trialling with them
  • why you’re testing or trialling ie, the difference this new product could make to the bigger picture
  • how important honest feedback is and the role it’ll play in shaping a product or service
  • that there are no right and wrong answers
  • the anonymity users can expect in any evidence we share – we don’t report back to management about specific users

Better communication will mean we’re all on the same page. And that’ll mean we’ll build better services.

Let us know how we can run tests and trials better in the comments.

Gillian MacDonald
User researcher

A discovery into digital Membership offers

Last year we tested out an app for members. The app we built allowed members to:

  • scan their ‘digitised’ membership card
  • check their reward balance on demand
  • choose a local cause for their 1% reward to go to

Listening to members

The feedback we got showed that people like the idea of having something on their phone rather than carrying a card around with them. However, lots of our stores don’t have tills that could scan the app.

We also realised that the features we’d included in the alpha weren’t compelling enough to launch as it was. However, feedback told us that personalised offers are something members really care about, and we felt we could improve them if we digitised them. At the moment, members receive their personalised offers on paper from the till, just before their receipt is printed. This means that to take advantage of money off they have to take, keep, bring back, and then remember to use the paper coupon next time they shop with us.

How can we help members benefit?

We wanted to find out if it’s possible to give members choice and control over the offers they get – this might be through an app or on the Membership website. We thought about how we could show members a pool of offers and then load the ones that appeal to them onto their physical membership card so they could redeem them at any Co-op store.

We worked with the Personalisation team in Food who look at members’ buying habits and match offers to members. Together, we’ve done a discovery to find out what’s possible with digital offers.

Asking questions

We looked at:

  • what our competitors are doing and where we think we can do better
  • if we made offers digital, how many offers would we give
  • how frequently the offers would change
  • if there’s a type of offer that works best in a digital format
  • who’d run the upkeep

Working quickly

Then we built a quick proof of concept, for what we’re calling a functional test. The page showed a group of offers and allowed members to choose 4 to load onto their membership card and redeem on the tills in the test lab in our head office.

It looked like this:

Screen shot of the page we mocked up. It shows a group of offers and allows members to choose 4 to load onto their membership card

This let us prove quickly and cheaply that we can build this for all members, on a website or through an app. The Membership team had already done a lot of research into digital offers in the past, so we were confident that this was something that our members would want and understand.

From a technical point of view, it’s looking promising

With a bit of tweaking, we expect the system that powers coupons at tills will also work with digital offers. It’ll be relatively straightforward to build the parts that the member interacts with, and it should be possible without needing to update our tills.

Testing our commercial assumptions

The more complicated part is figuring out which offers would be popular in a digital format. At the moment, we give paper coupons for things like dairy, bread and meat but we think that if we had the capability to switch offers in and out, offers on these things may get repetitive. So we’d look at giving offers on some smaller groups of products.

We’ll keep working in partnership with the Food team on this, but it might take a bit more time to make good commercial decisions.

What’s next?

We want to test this out with real members to prove if there’s a real appetite to use and keep using digital personalised offers.

To help us measure how many people want it, and help us build a pool of members to go to when we’re ready to test, we put a banner on the Membership website last week. So far, 21% of members who’ve seen it have indicated they’d like to help.

All being well, we’ll launch the first public trial on the Membership website and in an app this summer.

Joel Godfrey
Product manager

Pressing for progress at Co-op Digital

Yesterday was International Women’s Day. Throughout the day we shared stories on Twitter about some of the people at Co-op Digital who help us be our best selves. These people inspire, empower, encourage and elevate those around them. They help level the playing field so we hear a diverse range of voices.


Co-op Digital champions diversity full stop. We mention gender diversity specifically in this post because it’s International Women’s Day.

The importance of privacy and safety on social media

On Monday, the Social team held an event for 100 young people who attend Co-op Academies in Greater Manchester. The students were 14 to 16 years old and they’ll soon be leaving school and thinking about what’s next. The aim of the talk was to raise awareness about:

  • social media privacy
  • online safety
  • social media and your career
  • presenting yourself online

Here’s what happened.

Privacy and safety

The focus of my talk was on privacy and safety on social media networks. I warned that if we’re not careful, the range of data we disclose across various social platforms could easily be pieced together and someone with malicious intent they could steal your identity.

The takeaway points were:

  1. Make sure you know who can see your social media interactions. Some networks, like Twitter, are ‘open’ and they’re designed for networking and engaging with people you don’t necessarily know. Facebook and Snapchat are ‘closed’ and reserved for engaging with people you know personally.
  2. Create strong passwords for all your accounts and set up 2-factor authentication on your recovery email accounts. MUO gives decent guidance on this.
  3. Don’t share photos with anyone you don’t trust. Although Snapchat photos appear to ‘disappear’ they can be screen-grabbed and saved. They also exist for a time in Snapchat’s servers and they can be retrieved by the police.

Ian in front of young people with microphone presenting.

Social media and your career

Matt Eyre helps recruit people into Co-op Digital. He shared his tips on how to use social media to find your first, or your next, job. His advice is to:

  1. Have different accounts for personal and professional purposes.
  2. Check your privacy settings on all accounts.
  3. Keep your professional profiles up-to-date.
  4. Be proactive if you’re interested in working at a particular place. Get in touch with them.

Photograph of Matt Eyre reading from prompt sheet in front of students.

How you present yourself online

Choosing how we present ourselves by controlling what we share is really important. It’s about creating an image of ourselves to people we might not have met which can be useful when we’re looking for new jobs. Catherine Storey is our Social media content planning manager at Co-op Digital. She landed her last few jobs, including this one, through her careful management of personal social media accounts.

Here are her tips:

  1. Check who can see which parts of your profile on every social media account you have. It’s ok to show you have a social life, but make sure you know who can see it and the impression you might leave them with.
  2. Your Twitter handle is searchable and it says a lot about you. It’s an online representation of you, your ‘digital name’ if you like. Where possible, your digital name should be the same across each social network which helps build your online identity.
  3. Choose what you engage with carefully. When you tweet; update your status; post a photo; send a video, you expect people to see these things and it’s easy to be mindful of your output. However, when you ‘like’ a tweet, a photo or a status or leave a comment, your engagement is more passive and it’s easier to overlook the fact that these things can reflect on you too.

Photograph of Cat Storey presenting in from of students.

Aaron’s story

To finish, Aaron Omotosho talked about his (paid) work experience at Co-op Digital. Aaron spoke about the time he spent with product and service teams within Co-op Digital and his time after this completing a coding course at Northcoders. We also heard from Jonny Rathbone from Northcoders who spoke encouragingly about the opportunities out there for young people in the digital sector.

photograph of Aaron presenting in front of students.

We hope to hold more events like this in the future. It’s all part of our work to encourage a thriving tech sector in the north-west.

Ian Ferguson
Community manager

Steve Foreshew-Cain: celebrations, congratulations and goodbye

(Transcript) Steve Foreshew-Cain: Hello and welcome to this weeks Digital team update.

Firstly I’ll start with a congratulations for our Digital Engagement team, who picked up 2 awards at the Northern Digital Marketing Awards. They picked up ‘best use of social media’ and ‘best influencer and engagement’ campaign for the #ReverseAdvent. Great news, and very well deserved recognition for the team.

Also The Federation has been nominated for another Digital Impact Award, this time for social and economic transformation, which recognises outward-facing initiatives that look to make a positive impact on the wider world.

This alongside the recent partnership with Co-op Foundation and the Omidyar Network which has provided £700K of funding to offer free working space for tech entrepreneurs tacking social issues, showing the growing impact of the space. It’s something to be really proud of for the whole team.

We’re also developing a collaborative year-round events programme, open to all, that will tackle some of the major issues around creating a more inclusive digital economy and the development of a civic tech sector in the UK. This will include looking at the impact of technologies on broader social issues such as loneliness, inclusive participation and the right to privacy.

Our Co-op Academies are holding a recognition evening for their students next week and we were asked to nominate one of our colleagues to present an award. In line with the theme of the evening the colleagues presenting are those who consistently demonstrate the Ways of Being Co-op.

Annette Dockery, Annette Joseph and Scott Bennett were all shortlisted as colleagues who consistently go above and beyond in their behaviours and in bringing the Co-op Values to life. Annette Dockery through proactive commitment in bringing teams and communities together, Annette Joseph by her significant effort on the wonderfully successful Agile Immersion Course and Scott Bennett through his daily passion, proactivity, not least with the Pride campaign, and his dedication to the Co-op.

We would like to recognise all 3 of these colleagues for everything they do for us as a Co-op – thank you. The final decision was incredibly close but we have agreed that Scott Bennett will be presenting the award on the night. Congratulations Scott – we are proud to have you represent Co-op Digital.

And finally, this will be my last update. I’ll be leaving the Co-op next week. I’m leaving a team of passionate, dedicated and really talented individuals who it’s been a pleasure to work with.

Thank you for having me, but I know from here, you’ve got this

Steve Foreshew-Cain
Group Digital Director



What local cause data tells us about our members

We’re proud of the Co-op Membership scheme and what it gives back to our members and local communities. When members buy own-brand products or services from Co-op, we give 5% back to the member, and 1% to the Local community fund to support local causes as chosen by members. At the end of each funding period a new ‘round’ begins and members can choose from a new set of local causes.

Now that 2 rounds of payments have been completed, the Data Science and Community Engagement teams have been looking into the data and what it tells us about our members, the choices they’re making and what those choices might say about us as an organisation.

Learning more about the Local community fund

We looked at data around causes across 10 different categories:

  1. Art and culture
  2. Community development
  3. Education and skills
  4. Environment
  5. Health
  6. Social inclusion
  7. Sport
  8. Young people
  9. Animal welfare
  10. Other

Using these categories we’ve been able to see the number of members choosing each cause, the popularity of each category, and the amount of funding each one received. We especially looked for significant or unusual data that provide specific insights and which can help direct our future campaigning.

Here’s what we found.

Community payouts increased between round 1 and 2

Our members have raised £20 million for local causes through the Local community fund  – £9,196,993 in the first round and a further £10,825,772 in the second round.

We saw a rise in the number of causes chosen, as well as in the income that was generated for each different cause category.

Young people and social inclusion received the most funds

It’s unsurprising that our data shows members supporting causes that align with Co-op values. In fact the Local community fund is specifically targeted at causes benefitting members’ local communities. However, clear favourites do emerge under each category.

For both round 1 and round 2, the category receiving the most funds was ‘young people’. Causes relating to this raised nearly £5 million: more than £1 million more than the next category down, which was social inclusion. We have done a lot of work in this area, including through our partnership with the Red Cross.

Between the 2 rounds, the number of causes related to young people has increased from 882 to just over 1,000, showing that this remains a priority in Co-op communities. Social inclusion groups have reduced in number between rounds 1 and 2, but remain consistent in the level of support they receive.

Homelessness is an important issue to members

The high number of causes in the social inclusion category, and the many members choosing to support them, provide a clear indication that members share our appetite for tackling the problems facing communities.

Causes in the social inclusion category were amongst the most successful at encouraging members to support them – proving the most popular cause in 43% of all the communities where they were represented in both rounds.

In the case of causes that are working to address social issues, such as homelessness, this proportion can rise even higher. Across both rounds, causes supporting the homeless proved the most popular with Co-op members in 55% of the communities where they were represented.

Promotion works

As we approached the end of the second round of funding, more than 1,600 members were choosing a cause every day. This shows that interest in causes remains alive throughout the funding round, and messages about choosing causes were being received through friends and family, stores and across the media.

However we found that this number increases significantly when the Co-op, and causes themselves, begin to promote the Local community fund. The chart below shows that once social media is used, either by Co-op or by the charity, to highlight both membership and local causes, members respond by logging in and choosing.

cause-selection-promotion-04-14-nov-2017Between 11 and 13 November, driven by social media activity, the daily number of members choosing a cause rose 78%. This then increased to a total of 35,000 on 14 November when we emailed our members encouraging them to select a cause, showing clearly that email is currently our most effective tool for mobilising people.

Round 3: hoping for even bigger payouts

If you’re a Co-op member, you can now choose your next local cause by logging into your account. Alternatively, if you’re involved with a local group or established charity, you can register your interest for the next round of the fund, and use social media to raise awareness. As we can see from our data, that can make all the difference.
Simon Kirby, Community data and insight manager
Charles Gordon, Senior community data analyst

We’re building a user research lab

Co-op Digital is building an in-house user research lab on the lower ground floor of The Federation. It will be available to all Co-op teams, the Federation community and, eventually, the wider world.

We’re calling it Fed Lab.

What’s a user research lab?

A user research lab is a physical space. It’s usually 2 rooms: one room that looks like a lounge but with cameras and microphones installed, and another that looks like a regular meeting room but with a very large TV. The ‘lounge’ is for researchers and participants to do research tasks, like co-designing something or doing a task on a website. And the ‘meeting room’, called a viewing room, is where the team sit and watch the research happening live.

Labs help whole teams see first-hand how their designs are working or not working. This way, a team can learn together and work on changing designs and interactions there and then. This fast response to live user research is what makes a lab such an important tool in agile delivery.

Why we’re building a lab now

Co-op Digital’s remit is to create new digital products, services and platforms; pioneer new ways to cooperate online; and help our existing businesses make the most of digital. To do this, we put users and their needs first, and help teams across the Co-op do the same.

Until now, we’ve been using external labs for lab-based user and market research, but we know from past experience that having a dedicated in-house research space significantly ups the ante on how much time teams and stakeholders spend with users.

And we believe the more time we spend with users, the better.

In the long run, the lab will help us offer teams across the Co-op a complete in-house research service. As a result, we’ll be able to build consistency in research across the business, maintain knowledge and build on it in-house, and be more financially efficient and even profitable.

But it’s not just about us. The Federation, is a fast-growing community of like-minded digital businesses and innovators. Fed Lab will give these organisations access to the tools they need to learn more about their users too.

Where we’re up to

We’ve completed the discovery and design phase of the lab, we’ve chosen the technology, and we expect to have the lab up and running in the summer. We’re pioneering a new solution, so everything needs more care and time.  

In building the lab, we’re planning on using cutting-edge, military-grade technology that will not only offer a top-end experience for people in (and out of) the lab, but also support us in maintaining our data privacy standards and help us conform to GDPR.

As the project takes shape, we’ll share much more about the technology and the design. To keep up to date with our progress, subscribe to the Co-op Digital blog. For more information, get in touch with Kate Towsey.

Kate Towsey
User research operations