How we went from a 3-week discovery to 14 potential alphas

Running a food shop is simple in theory. You need to make sure there’s food on the shelves, there are colleagues available to help customers if they need it, and you’ve got to make sure customers can hand over their money when they want to buy something.

In fact, running a branch of a supermarket is pretty complicated. Even within that first statement, ‘make sure there’s food on shelves’, there’s a whirlwind of complexity. Getting food on the shelves involves logistics like knowing when a delivery is arriving, best before dates and in house baking.

At the beginning of March we completed a 3-week discovery to find out how we could make life simpler for our colleagues in stores. After the success of the Product Range Finder, one of our previous alphas, we wanted to find other opportunities for us to help. Now, we’re at the end of the discovery phase and we’ve proposed 14 alphas that we could work on.

Here’s how we got to this point.

Getting the right team together

We needed the right mix of people working together. It was just as important for us to collaborate with people with first-hand experience of the shop floor as it was for us to work with people with digital skills. The ‘Leading the way’ team from the Food business joined us. The purpose of their group is to help colleagues ‘go back to being shopkeepers’ by taking away some of the administration involved in running a store. Four of them joined the Co-op Digital team for the whole 3 weeks, and importantly, 3 of them had been area managers or shop managers within the last 12 months. Like we did for the first 3 Food alphas, we teamed up with digital product studio ustwo too.

Learning how things work in store

During week 1, we had around 20 colleagues from the Leading the way team come and work in Federation House to map out what happens in a Co-op store, and what goes into running one day to day.

We learnt about everything from walking around the store in the morning, ‘facing up products’ and cashing up, about what happens to unsold magazines when the issue expires, and a whole lot more. The purpose of the workshop was to uncover any assumptions. Doing this meant that anyone who didn’t have first-hand experience in store could get a decent understanding of how things work which in turn meant that our research would be less biased and more thorough.

Using filters to figure out potential

In our first week we also set up some team principles and some filters to evaluate each alpha idea on.

“Yes” ideas were ideas that we thought were good enough to carry forward to the alpha phase. Each one would:

  • have a clear user need
  • have potential for lasting value
  • empower colleagues and decentralise processes
  • keep colleagues on the shop floor

On the other hand, we had some ideas we wanted to ditch. “No” ideas were the ones that:

  • had a poor effort to value ratio
  • would add to colleagues’ workloads
  • didn’t actually need a digital solution

image shows 3 columns of post-it notes. The first column shows criteria for a 'yes' idea, the second for a 'no' idea and the third for ideas that might be good to pursue at a later date.

Week 2 and crossing the half-way point

In the second week of the discovery we spent around 30 hours in store doing ‘Lend a hand’ which is exactly how it sounds: we lent a hand to colleagues. We interviewed them and their store managers in different parts of the country. We also interviewed customers, to find out what they like about Co-op, and what they think could be improved.

After each store visit and interview, we shared what we’d learnt with the rest of the team, and we started to see themes emerge from the things we were seeing and hearing from colleagues.

image shows 3 colleagues sharing their feedback and arranging post-its on a wall.

We used those themes to create some prompting questions which we then asked over 60 Food colleagues at ‘sketching sessions’. For example, one of the themes that came out of the feedback was that it’s not always clear to colleagues how they can progress their career at the Co-op, so we asked colleagues at the sketching sessions “how can we help staff to progress?” They’d then draw something in response.

Here’s an example sketch in response to the question, “how can we sign up customers for membership at the store?” 

Sketch from colleague Phil Hesketh shows a machine that you can put your temporary card into, a screen where you choose the cause you'd like to support, and a real card will popping out of the bottom of the machine.

By the end of the sessions, colleagues had produced a whopping 562 sketches.

Getting our priorities straight

We put them all through the filter and managed to whittle the ideas for solutions down to 41. Then we fleshed them out, before prioritising them by asking:

  1. How risky is the idea?
  2. How much evidence for the opportunity do we have?

We figured the sweet spot was where we had both evidence and low risk. After looking at the 41 ideas through that lens, we got to 14 – a more manageable number!

Where we’re at now

Last week we presented back our ideas to the wider team.

group of colleagues from across the Co-op and ustwo gathered around whiteboards to hear the feedback on the 14 potential alphas.

Now it’s up to the Leading the way team to figure out which they want to go forward with, because we won’t be doing 14 alphas all at once. Just like last year’s discovery, we found a lot of opportunities, but we know we’ll solve a problem best if we can solve them one at a time.

Anna Goss
Product lead

Making better, joined-up decisions with the engineering community

This month, it’s 3 months since we set up our engineering community for software engineers, platform engineers, service managers and quality analysts at the Co-op. It’s early days but it’s already helping move Co-op engineering in the right direction.

Getting together with people who do similar jobs helps us all be more joined up which is really important, especially in a place as big as the Co-op. Without a community, we’d be working in isolation because our day jobs are within Co-op Digital, Co-op Legal Services or Funeralcare.

When we began meeting regularly, we identified the areas we need to work together to develop, including how we support training and development and coming up with development standards.

Picture of our Engineering community of practice

We’ve created infrastructure standards

I was really pleased to see that practices such as Continuous Delivery and Infrastructure as Code were already well established when I joined Co-op Digital 6 months ago. However teams were working in isolation at that point. Lots of them had similar problems and were tackling them in different ways. This meant that getting some of the services we were launching to a point where they were secure, reliable and supported was trickier than it needed to be because there was quite a bit of rework involved.

To make things simpler, we spent time during our community of practice meet-ups to create shared standards for our platform infrastructure. There’s still plenty to do and these things are never really finished of course, but we’re now in a much better shape and future projects will follow a much easier path. Most importantly, teams are more empowered to get on with stuff and do their job.

We’re also working on standards for how we’ll support cloud infrastructure across several teams. This work will sit with our Digital Operations team which is forming steadily.

Making better technology decisions

Out of that also came a clear need to provide better support around making technology decisions. We want teams to be empowered, but at the same time there’s always going to be a limit on how many different technologies we can support and maintain. Our approach has been to try and provide really great guidance so teams can make decisions in context rather than needing meetings to make decisions. It’s all still quite early days so again we’ll hopefully come back again soon and update on how it’s getting on.

We’ve been hiring

C6VVNKlWgAMmd3r

We’ve worked with some great external companies while we’ve been adding gradually to our in-house expertise but we’re at a stage now where we’re looking to bring in a significant number of software and platform engineers. The Co-op Digital team and the wider engineering community of practice is looking forward to new talent joining us. From there, the culture of the team will grow and strengthen.

If you’re interested, take a look at our Work with us page for the roles we currently have open. We’ll be recruiting for engineers for the Funeralcare team shortly.

In the meantime, sign up to the blog and follow Co-op Digital on Twitter.

Rob Bowley
Head of Engineering

Small is beautiful. User research and sample sizes

At the Co-op, we use both qualitative and quantitative approaches to make decisions about products. This post is about doing qualitative research with a small-ish numbers of users – and why that’s useful.

As a rule of thumb, qualitative and quantitative approaches are useful for different things:

  • If you want to understand the scale of something (such as how many users do X or Y, or how much of something is being used), use quantitative methods, like surveys.
  • If you want to understand why people do something and how they do it, qualitative methods such as interviews or seeing how users behave with a given task (user tests) are better.  

User research isn’t a one off event. It’s a process. By researching with a handful of users at a time, iteratively, and supported by data on user behaviour, we build better digital products and services.

How many users are enough?

We don’t need to observe many users doing something to identify why they’re behaving a certain way. Jakob Neilsen, a usability expert, found through research with Tom Landauer that 5 users is sufficient. More than 5 and your learning diminishes rapidly and “after the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new”. Here’s Neilsen’s graph of these diminishing returns:

Graph shows percentage of usability problems found on the y axis and number of test users on the x axis. the graph sows that we find 100% of usability problems with a relatively small number of test users.

Source: Jakob Neilsen

Analysing user data and user research findings are complementary in developing digital products and services. Data can help identify issues to then test with users, but it can also run the other way. In user research at the Co-op, we’ll often see things while doing user research which we’ll then investigate with data. It works both ways.

Screen Shot 2017-03-09 at 11.26.18

There’s cumulative value in cycles of research

The cycle of user research shown in the diagram is how product teams work at the Co-op. We typically iterate in weekly or fortnightly cycles.

For example, the Membership team has a rhythm of fortnightly cycles. These are often focused on discrete aspects of Membership. These research cycles accumulate learning over time. They create an understanding of Membership and of users needs. Cumulatively, this gives clarity to the whole user journey.

During the last 10 months, the Membership team have surveyed 674 users and interviewed 218. The value of this research accrues over time. The team has learnt as they’ve developed the service and iterated on the findings, getting to know far more than if they’d done the research in one block of work.

That’s why observing relatively few users doing a task, or speaking to a handful of users explaining something they’ve done, is enough to provide confidence in iterating a product and to continue to the next test. This is especially true when user research is used together with data on user behaviour and even more so when it’s done regularly to iterate the product.

Error-prone humans are there in quantitative research too

It’s not uncommon for people to give more weight to quantitative data when they’re making decisions. Data is seen as being more factual and objective than qualitative research: “you only spoke to 10 people, but we have data on thousands…!”

Data hides the error-prone human because humans are invisible in a spreadsheet or database. But even though they’re hidden, the humans are there: from the collection of the data itself and the design of that collection, to the assumptions brought to the interpretation of the data and the analysis of it.

All data is not the same

Survey data, based on responses from users, is distinct from data collected on behaviour through Google Analytics or MixPanel. Poor survey design produces misleading insights.

Getting useful behavioural data from a user journey is dependent on setting up the right flows and knowing what to track using analytics software.  Understanding what constitutes ‘good’ data and how to apply it is something we’re working on as a community of user researchers at the Co-op.

Research is a process, not a one-off

Digital product teams usually have a user researcher embedded. They can also draw on the skills and experience of the conversion and optimisation team and their quantitative and statistical skills and tools. The user researcher gets the whole product team involved in user research. By doing this, they gain greater empathy for and understanding of their users (and potential users).

This diagram shows some of the methods we use to help us make good product decisions and build the right thing to support users:

Screen Shot 2017-03-09 at 11.36.18

As user researchers our craft is working out how and when to deploy these different methods.

Part of the craft is choosing the right tool

Let’s take an example from a recent project I was involved in, Co-op wills, where we used both quantitative and qualitative research.

We had customer data from the online part of the service and analysed this using a tool called MixPanel. Here’s part of the journey, with each page view given a bar with corresponding number of visitors:

Screen Shot 2017-03-09 at 15.38.11

From this, we could determine how many users were getting to a certain page view of the wills service, and where they were dropping out.

The data let us see the issue and the scale of what’s happening, but it doesn’t give us a sense of how to resolve it.

What we didn’t know is why people were dropping out at different parts of the journey. Was it because they couldn’t use the service, or didn’t understand it, or because they needed information to get before they could complete?

To help us understand why people were dropping out, we used user data to create hypotheses. One of our hypotheses was that “users will be more likely to complete the journey if we start capturing their intent before their name and email address” ie, show them the service before asking them to commit.

Through user research with small numbers of users we found a series of different reasons why people were behaving in ways Mixpanel had showed us, from confusion over mirror wills to uncertainty about what the service involved, to requiring more information.

We only got this insight through speaking to, and observing users, and getting this insight allowed us to design ways to fix it.

It’s not an exact science – and that’s OK

Research is not an exact science. Combined with user data, user research is a process of understanding the world through the eyes, hands and ears of your users. That’s why it’s central to the way we’re building things at the Co-op.

James Boardwell
User researcher

International Women’s Day: we need more diversity in tech

Today is International Women’s Day (IWD), a day to celebrate the social, economic, cultural and political achievements of women. It’s also a day to discuss equality and sadly, more often than not, lack of equality between men and women, boys and girls in many areas of life.

The importance of being seen

I’m an architect which means I plan and design systems both technical and human for Co-op Digital. Throughout my career I’ve often been the only woman in the room. That won’t be a huge surprise because it’s hardly new news that the world of tech, digital and design has always been male-dominated.

And that’s a problem for the next generation of women because as the saying goes: if you can’t see it, you can’t be it. In other words, girls aren’t likely to aspire to take on roles and be part of a community that they’re under-represented in.

yellow background and black text. text says: 'If you can't see it, you can't be it.' Followed by #beboldforchange

Time for change

At Co-op Digital we’re committed to trying to break out of the catch 22 situation and reduce the imbalance of men to women in tech. We actively support Ladies of Code, She Says Mcr, Manchester Geek Girls and Ladies that UX. We also made a pledge to support gender diversity at conferences so that no one from Co-op Digital will speak at events or be part of panel discussions of 2 or more people unless there’s at least one woman speaking or part of the panel (not including the chair).  

Celebrating with a screening

To celebrate IWD me and my colleague Gemma, a principal engineer, arranged a screening of Hidden Figures in Manchester for families with children aged 6 and above. Our aim was to inspire young people, particularly girls, to consider a STEM-based (science, technology, engineering and mathematics) career.

image is from Sunday screening and shows Gemma speaking at the front of a cinema packed with families.

The film’s about 3 female, African-American mathematicians who had a massive impact within NASA in the early days of the space program. They succeeded in engineering, mathematics and software development despite facing gender and racial bias. The film’s based on a true story so it seemed like an appropriate and important thing to show to mark IWD.

image shows the leaflet we gave away at the screening. The text on the front says: 'International Women's Day private screening Hidden Figures. Sponsored by Co-op, Autotrader, Tech North'

Thanks to sponsors from Co-op Digital, TechNorth, Autotrader whose support made it possible to make it a free event.

We’ve made a good start…

The screening was fully booked and we also inspired and supported 3 sister events screening Hidden Figures or Codegirl in Sheffield, Nottingham and Liverpool. As the crowds left the cinema, I overheard 2 brilliantly positive things from young, female attendees. “Can I go to Madlab, Mum? I want to make something!” asked an 8 year old, and a slightly older girl asked, “Nana, do you think I could be an astronaut?”

…but there’s still a long way to go

Sure, over recent years women and minorities are marginally better represented in this sector and that’s in part due to the committed people at organisations like Manchester Digital who created a ‘diversity toolkit’ to address issues around equality last year.

But women are still grossly outnumbered. 

If you’re a parent, consider taking your children to one of the many free creative and code clubs in the north west. Or if you’re are curious about a career in the sector yourself, come along to one of the meet-up groups in our thriving northern tech community.

We need to continue to be bold for change and fight the good fight every day, not just today.

Danielle Haugedal-Wilson
Digital Business Architect
Co-op Digital champions diversity full stop. We mention gender diversity specifically in this post because it’s International Women’s Day.

Funeralcare: taking the beta to Edinburgh

Since April 2016, the Funeralcare team at Co-op Digital has been working to make life easier for our colleagues at our funeral homes across the UK. Our aim has always been to reduce the time our colleagues spend juggling and filling in paper forms so that they can spend more time with their clients – people who are grieving for their loved ones.

It’s been awhile since we wrote an update on our work. Back in August Andy Pipes, our Head of Product Management, said that we were rethinking how we deliver our at-need funeral service (an ‘at-need’ service is the immediate assistance someone might needs after reporting a bereavement).

At that point we’d built:

  • a ‘first call’ service that logs details of a death and automatically alerts an ambulance team by SMS to take the deceased into our care
  • a funeral arrangement service which captures the client’s decisions, the costs, and keeps colleagues in various locations from funeral homes and the central care centre updated
  • a hearse booking system, staff diary and staff assignment service
  • a coffin stock control system, and a way for clients to browse the existing coffin range
  • an audit system that captures certain steps in the service

Since then we’ve been busy testing with colleagues and iterating.

We’ve added new features

As we’ve learnt where the gaps are in the service, we’ve added new features. They include a digital mortuary register and a digital belongings log to record possessions.

Deceased can come into our colleagues’ care at any time of the day or night and it’s vital the funeral director knows where that person has been taken. To help, we’ve developed a digital mortuary register so that ambulance staff can book the deceased in and the funeral director can see where the person has been taken.

image shows a screen with the first page of the digital mortuary register. the options are 'booking in' and 'booking out'

Another new feature is a digital belongings log. Often, when someone is brought into our care they’ll have jewellery on them or other personal belongings with them. This means that when a funeral director at a funeral home gets a call from the grieving family to check up on jewellery, they don’t immediately know what the deceased came in with because the paper record is with the deceased at the mortuary. To make this easier and more efficient, we introduced a digital log instead of needing multiple phone calls between different locations.  

Live trial and user testing

We’ve been testing in 2 ways. From September to November we continued to visit funeral homes all over the country to observe how colleagues work but we were also doing usability testing on each of the individual features in the bulleted list above with colleagues in mock labs. We tested and improved each feature separately until we thought we’d built enough of a service to be valuable to colleagues. At that point, in December, we rolled out a beta trial in Bolton.

interaction designer Matt researching which content is most valuable to one of our colleagues with a paper prototype.

We asked colleagues in Bolton to use the service in parallel with their current process which involves whiteboards, post-its, paper diaries, fax machines and the old, often painful-to-use software. Letting them use it for real is the best way to learn what’s working and what’s not. It drew our attention to 3 major things we’d overlooked during usability testing.

  1. We thought we were being helpful by preloading the local churches and crematoriums but we hadn’t given colleagues the option to create new ones.
  2. We found that the calendar couldn’t cope with all day events.
  3. We discovered that colleagues help each other out so having restricted access for specific roles creates a problem if someone is off ill and cover is needed.

Testing the beta with a small number of colleagues helped us catch problems like these before we rolled the service out to more people.

Trialling the service in Edinburgh

We want our service to be useful everywhere but we’ve been told many times by colleagues that there’s no such thing as a ‘typical’ funeral. They vary from region to region for reasons including local traditions, operational set up, affluence, traffic as well as legislation. Because our aim is to give time back to colleagues so they can spend it with their customers, we need to create something that works for all users not just our colleagues in Bolton. That’s why we are launching our at-need funeral service trial in Edinburgh in March.

We’re still learning

The beta has shown us that funeral arrangements are made up of multiple interactions like choosing flowers, booking venues and signing off obituary notices. Funeral arrangements are iterative with lots of tweaks along the way, so iterating the design is the only way we can cope with all the new things we keep learning.

We know that standard software packages don’t solve every problem. By involving colleagues throughout we’re building something that meets their needs and will improve things for both colleagues and their customers.

We’re transforming the Co-op Funeralcare business but we believe that what we’re doing here will actually help transform the entire industry. To help us do this, Co-op Digital is working towards having a dedicated digital product teams within the Co-op Funeralcare business.

If that sounds like something you’d like to help with we’re looking for an agile delivery manager and a product manager.

You can read more about the agile delivery manager role and more about the product manager role.

Come to a talk about the digital transformation of our Funeralcare business on 28 March. We’re particularly interested in speaking to product managers, delivery managers, software developers and platform engineers. You can get your free ticket at Eventbrite.

Carl Burton
Product lead

Warning! Your MVP may cause discomfort (but it’s worth it)

We recently posted about how Co-op Digital and the Co-op Legal Service (CLS) combined their digital and legal expertise to build a service that makes it simpler to get a Co-op will. Together, we built something that’s both legally robust and easy to understand. In other words, it meets the needs of our customers.

But bringing together 2 contrasting ways of working so we could deliver this was tricky. The challenge was wider than combining the 2 disciplines. It involved building trust in the agile way of working with the wider Co-op business.

We start small

The digital team works in an agile way. Part of being agile is about getting value to your user as soon as you can through a minimum viable product (MVP). This means building the smallest usable thing that might be useful to them. Then, you watch how real users interact with it, listen to what they say about it and iterate and improve quickly based on what we learn from research.

Being perfect’s not the point

Releasing an MVP helps us build something useful at each stage of delivery and it’ll help us build the right thing. The point of working in this way is to avoid building and overspending on something that doesn’t meet user needs. So, releasing an MVP actually makes sound business sense.

But it takes time to learn about the needs of your users which means it takes time to build the best solution. This is a daunting process for anyone who isn’t used to working in this way, because an MVP is very rarely pretty.

In fact as Reid Hoffman, Founder of LinkedIn says:

“If you’re not embarrassed by the first version of your product, you’ve launched too late.”

A reputation to uphold

When we released our MVP, real Co-op customers were going to be using it to give details about their situation and to book in a call with one of our will writers. It’s understandable that anyone who’s unfamiliar with this way of working would be nervous about releasing a product with known problems. We’re operating in a competitive environment and what if potential customers were put off by a poor user experience? What if they went to a competitor instead? Would releasing an MVP put the Co-op’s wills services’ reputation on the line?

Reaching a compromise

The biggest sticking point was that the digital team wanted to release a very stripped back version that didn’t cater for a whole customer segment. CLS had an assumption that launching such a minimal service without the option to make a ‘mirror will’ (something often used by spouses) would put potential customers off.

As David Bland says in his post Spruce, the corporate minimum viable product:

“The challenge with a minimum viable product is that you decide what’s minimum, but the customer determines if it’s viable.”

The digital team had to trust CLS’s judgement on their customers and release something more developed than we might usually expect an MVP to be. We were happy to do this because we knew we could learn lots from doing things this way.  

Understanding each other better

The more the 2 teams worked together, the more the trust grew. CLS came round to the idea of releasing an MVP (or something close to one) after we explained:

  • it’s possible to iterate to fix any customer concerns in a matter of days
  • we could ‘turn off’ the beta instantly and we could also control the traffic to the online version and only allow access to a small percentage
  • the phone call part of the journey could act as a backup and we could help customers over the phone if they had any problems online part of the service

Data-driven decision making

Once we’d launched there was a mindset shift in the team and the wider business. Together we looked at data and tied that to user research instead of relying on assumptions.

Tracking user behaviour with analytics tools really helped confirm that releasing as early as possible was the right thing to do. It was like having a window to view a customer’s behaviour and we used the data to help make decisions about the product development.

We could see at which points customers were stopping their journey and this helped us prioritise work. For example, we knew that an automatic postcode lookup feature would be useful here. It was coming up in user research regularly as something that would help smooth the user experience. However, when we looked at the data in our analytics we found that the vast majority of people were filling in the address fields manually just fine. So we decided to de-prioritise building postcode lookup. There were other areas that needed attention before this.

Taking a leap of faith was worth it

The metrics tools helped us show stakeholders and the Co-op Legal Service the connection between our product improvements and the bookings and sales. We could also show that the online business is generating a new set of customers that’s not cannibalising the original service. We knew we could potentially scale this up which is really positive from a business point of view.

In the next few weeks the digital part of the team will start transitioning over to Co-op Legal Services who will continue to iterate the product.

Find out more about Co-op wills.

Ben Aldred
Product engineer

Supporting the Manchester Digital Skills Festival

Last week, Co-op Digital sponsored the Manchester Digital Skills Festival, an event that promotes careers and collaboration in tech, digital and design. Students, graduates and educators from local schools and universities had the chance to meet more than 180 digital organisations from across the north west.

Photograph of hall with attendees and speaker inside Manchester Town Hall

Co-op Digital contributed to an experience day where a group of 13-14 year old pupils from a local Co-op Academy came into The Federation; a talent day for students and graduates with an interest in the industry and a conference day aimed at starting conversations between digital organisations and educators.

Getting young people interested in digital

During the festival the need for digital organisations to engage with people at a younger age was flagged (again) as a good way of improving diversity in the industry.  

With this in mind, we invited pupils from a local Co-op Academy into The Federation. We gave them an opportunity to get a feel for what it’s like to work in the tech community by spending time with some of our communities of practiceThey also took part in user research, coding and agile delivery workshops.

Pupils working collaboratively on a lego project

There was a chance for digital organisations to talk to educators from local schools and universities. The 2017 skills audit was a big talking point and Rob Bowley, Head of Engineering, was part of a panel discussing key issues flagged in the report.

Bringing more digital people to the Co-op

Principal Engineer Gemma Cameron talked about the Co-op’s culture and values and how they help us build products and services that meet the needs of our members, customers and communities.

Over the past year and a bit Co-op Digital has attracted loads of fantastic digital talent. People who care about doing the right thing for our Co-op colleagues, members and their communities. We’re looking to encourage diversity in the digital and design community, and we’ll be recruiting more great people throughout 2017.

You can find out more about a career at Co-op Digital and follow Co-op Digital on Twitter.

Matt Eyre