At one point or another, most digital teams have been convinced that their assumption about how to fix something will work but when they’ve tested it, they’ve found they’re still way off solving the problem.
That’s ok.
It’s fine to ‘fail’ as long as you do it quickly, learn from it and make changes to make things better. It’s part of working in an agile way. We should talk about failing more often. So, here’s an example of how we failed fast and learnt quickly in the Membership team.
Making assumptions based on user research
We’d seen from user research that most people, especially those who were new members, didn’t understand what a co-op is, how one operates and why it’s a different way of doing business.
Most people, especially those who are new members, don’t understand it even though we include loads of info on co-ops when we send out membership cards. But it looks like people either don’t read it at all, or, if they do, they don’t remember the information. Without that understanding, the Co-op Membership is just another loyalty card to most people.
During user research sessions when we talked about the idea of a co-op, people seemed interested. Not everyone, but some. The problem seemed to be not with the quality of information being given, but where and how in the user journey it was given.
It seemed if we could more effectively convey the concept of a co-op, that would be enough for some users to become more engaged. Certainly they would be better able to make an informed decision whether they wanted to get involved. They’d become true co-operators as opposed to just loyalty card users.
Making changes based on our assumptions
We designed an interaction where the information about co-ops and Co-op Membership was introduced to people as part of the online registration. Our hypothesis was that at this point in the user journey the member is more committed and more likely to have time to read this information and be more receptive to it.
By chunking the content into sections and importantly making it dismissable, the user would be able to digest as much or as little as met their needs, rather than being faced by the entirety of the proposition in one hit.
We know people don’t read things online. In fact you’re lucky if people read more than 20% of what you stick on a screen so we kept that in mind with the design.
Here are 2 examples of pages from the prototype.
Then we tested the new design
During 2 rounds of research we spoke to 12 people (you can read more about our views on samples sizes in James Boardwell’s blog post ‘Small is beautiful’). The group included a mixture of ages, online capabilities and length of time being a member.
Before showing them our new design we asked each participant to fill in a short questionnaire to find out what they understood about Co-op Membership. We then interviewed them, and showed them the prototype that was intended to help them understand the idea of a co-op.
At the end of the session we asked them to fill in the same questionnaire.
Results showed we hadn’t got it right
As we expected, before looking at the prototype people didn’t understand:
- what rewards they earned as a Co-op member
- what a co-op is
- elements of the Co-op such as the dividend, democracy and engagement
And the post-prototype results weren’t any better – the new design had had zero effect on users’ understanding.
Picking ourselves up. Trying again
We’d seen people read the information, but they didn’t take it in. Although we were giving them more control, we were still imposing a bulk of potentially irrelevant content rather than letting the user discover it in their own time, and reading as much or as little as met their need.
For some people, some of the information would have been both relevant and necessary – but for most their primary need at this point was to find out ‘what’s in it for me’ and everything else was a distraction.
So we iterated again. This time we wanted to give people a positive interaction that let them get only what they wanted, at a time when they needed it.
We added a ‘what’s this?’ drop down within members’ accounts to explain both rewards and Co-op points. Here’s how the current design looks.
We’d seen in research that many people often didn’t know exactly what they got for being a member so adding this was important.
Better results this time
During research we watched people time and again interacting with the drop down, unprompted. Responses were often comments from the user such as ‘ahhh, so that’s how it works’ or ‘I didn’t know that, I thought they’d send me vouchers’.
If there wasn’t that immediate, unprompted reaction we’d then follow it up with questions such as ‘what made you click on that’ and ‘what did it tell you’. This made us confident in what we were seeing had met the need we’d identified and so we released it. We know people are making use of it. Google Analytics tells us those drop down controls have been clicked 250,000 times since we released it on 14 February.
So after failing often and learning from, and iterating on, what users were saying to us, we’ve made good progress on helping people understand what rewards they’ve earned as a Co-op member.
We’re still researching how best to help people get a deeper understanding of what a Co-op is including elements of the Co-op such as the dividend, democracy and engagement. Those are things we haven’t solved yet, but we will. And it’ll probably involve a lot of failing fast.
Simon Hurst
User research
I spoke to a customer yesterday who still did not know how to spend her rewards. I asked her what she knew about her vote, but she said she was not interested in voting for people that she doesn’t know.
Hi Sandra, thank you for your comments.
I see this a lot in the research we do with people. In my research I’ve seen that people sign up for schemes because they think they’ll get ‘something’ in return. They rarely seem to understand what. They also don’t want to take the time to learn because people have busy lives, as soon as they leave our store they are thinking about getting the kids to school, getting to what, what to have for tea etc… So they don’t take in the detail. They only learn how it works through direct interactions. This occurs at the till and with our store colleagues. I personally believe store colleagues have an absolutely vital role in help people understand and get the most out of membership- they can do a job that an online service simply cannot do.
I have shopped at co-op for years, but I seem to remember that Paul Flowers escapade cost all of us a lot of loyalty points for a good few years, the co-op did not belong to it’s customers then, did it? we would have said why give him a golden handshake? Your loyal customers paid the price for that disgusting individual, and I have no doubt that were the same thing to happen again, it would not be the criminal paying.
what people need to understand is that they are ‘owners of the business’ – and a direct comparison given with owners and shareholders in other corporations, who have a control of the business decisions dependant on their financial holding (this is a concept most people today are familiar with ) – with an explanation that in a coop, for a £1 shareholding. every member has an equal say- and this is the most significant difference – profits are there to be shared with all our 4m members, after reinvestment to keep the business strong. We have no other ‘owners’ or ‘profiting shareholders’ apart from our 4m members.
This is enough to pique interest where there is any, to other democratic and financial benefits of coops.
So many people are unused to the concept of shared ownership and responsibility that starting from the reward gets them involved but is in fact a red herring, making them think this is a loyalty scheme.
Hi Barbara. Thank you for your comments.
I think you make a really good point, and I especially like ‘enough to pique interest where there is any’. As the blog post says, when people are exposed to it then they can make an informed decision of ‘this isn’t for me’ or ‘actually, I’d like to know more about this’. The difficult thing is how we do this. It has to be at a time when the member is receptive to the message and in a way that they can consume it. Again, I’ve seen this handled well by staff and often the route in has been when they customer asks ‘why do I pay a £1?’
I’d disagree with the member ‘needing to understand’ though. We want the member to understand. The member need is to be able to get what they came to the Co-op for, be it milk for their tea in the morning, or bread for the kids packed lunch.
Back in 1844 members joined because they needed an alternative way to do business as they were being ripped off, and being sold flour with chalk in. They needed a different way to do business. We need to understand what the equivalent need is for membership in 2017.
Simon
Having spent over 20 years in direct marketing at the Co-operative Bank, I can vouch for how your agile approach to active testing and research through a continual process of “Do – Analyse – Learn – Do Better – Analyse..” is exactly right.
Of course, what has to be defined before turning the wheel is “what good looks like” and that can take many forms… through the business lens it’s higher value member relationships (our current mutual membership proposition), through the community lens its participation of members in their communities (our current purpose) and through the lens of co-operation participation by an increasing minority engaging in our democracy (our core purpose).
After 40 years of watching our Co-op try and fail to get all of the above right… it will be a fascinating experience to see how you make a success of you exciting venture…. but I think we’ll have to be patient…. I suspect you’ll need some computer power to analyse the multiple variables in customer-member behaviour that will need to be unpicked, analysed and tested before we get the optimum balance that will make me us a 21st Century Co-op making a real Co-op Difference…. I look forward to reading/hearing more of your news!!
Frank
Hi Frank, thank you for your comments.
I completely agree with what you’ve said. I also think from your comment that you recognise that this is actually really, really hard to get right!
A lot of our research has been about getting the service working and meeting the immediate member needs. So, people needed to be able to quickly and easily register to be a member, they needed to understand how the basics of membership worked, things like that. However, user research is never just about testing websites so we’ve talked to people about their community, and their view of businesses so we’ve also been learning about community and engagement. I’m now working on the ‘more engaged’ strand of membership full time so we’ll be looking to focus on tackling this now, but as I said early, it’s hard.
Research on membership will continue, working in agile, researching, building and testing is a constant, iterative cycle. This work will be lead by a fellow user researcher, but we work closely together so we’ll both be sharing what we learn and, more than likely, blogging about it as well.