How we tried to increase temporary card registration with flyers

Recently, in his post How we’ve helped users understand Membership, user researcher Simon Hurst said that “it’s fine to ‘fail’ as long as you do it quickly, learn from it and make changes to make things better.” It made me think about my most recent example of failing fast and how useful it was for the ‘more members’ part of the Membership team to do a quick, inexpensive trial so we could test an idea.

The problem with temporary cards

You can become a member by signing up online. You register your details, pay your £1 and your Co-op Membership card is sent to you through the post. You can also sign up in our food stores. You pay you £1 and you receive a temporary card to use there and then. The idea is that you’ll go online to register your temporary card later.

However, our user research and data show this isn’t what’s happening. 58% of temporary cards we’ve sold haven’t been registered. This is a problem because:

  • around £1 million of 5% reward is sitting in a pot, and can’t be spent until the temp cards are registered
  • we can’t get in touch with customers to let them know the balance they have because their temp card isn’t registered
  • until they register the card, customers can’t access all the member benefits. For example, they can build up their rewards but they can’t spend them or choose a local cause to support

To try and increase the number of temporary cards being registered we ran a few trials in stores. We dubbed one of these ‘the flyer test’.

Encouraging temporary card holders to register

Here’s our hypothesis:

Photo of post it notes stuck on a whiteboard with hypothesis on them. Hypothesis reads: We've seen/we've heard That people aren’t registering their temporary cards We believe this is because They don’t know they have to do anything with it, and the instructions given aren’t clear So if we Give them better instructions We'll see More members registering We'll know this is true when We see an increased temporary card conversion rate

To test this hypothesis we asked colleagues on tills in 10 stores to watch out for customers who were swiping a temporary card. When they spotted this happening, we asked them to hand those customers a flyer which had a call to action on it: ‘register your temp card’. The flyer also explained the benefits of registering the card to try and nudge people into registering.

Image shows front and back of flyer. Front says: Register your card online to claim your member rewards. Back lists things that members are missing out on if they haven't registered their cards online.

We included a vanity URL so we could track how many people registered their cards after receiving a flyer. Simple.

Learning early

We had our hypothesis and agreed our test. Our first failure was cracking the logistics of designing, printing, delivering leaflets across the country. That was hard, and so was making sure our store colleagues understood why we were doing this. This was our first learning: there are colleagues across the business great at doing this, and working with them is better than working alone.

We hadn’t fixed anything. And that’s hard to take

We sent flyers to 10 stores across the country and asked them to hand them out for the next 4 weeks. We put Google Analytics tracking in place and we decided on our measure of success: 10 visits to the URL a week, with 50% of those going on to register their card.

The test went live and we eagerly refreshed the Google Analytics report each morning waiting to see an improvement in temporary card registration. There were none. Nobody was visiting our URL.

We called the test stores. Maybe they hadn’t been handing the flyers out? Turns out they had. And what’s more, colleagues liked them because the flyers were an easy, concise way to tell customers why they should register their cards.

But they weren’t working for customers.

Over 4 weeks, 35 people visited the URL, and 3 of those people registered their cards. We hadn’t hit our measures. The test had failed.

We learnt lots, quickly

The trial taught us that:

  1. People don’t naturally move from a physical thing (a flyer in a shop) to a digital thing (our website). Even if you spell out all the really great reasons why they should. If moving from physical to digital was a natural thing for people to do, they probably would have already registered their temporary card.
  2. Involving wider team members early on is important because they may have ideas, sometimes tried and tested ones, about how to get stuff done.
  3. We should test an idea from as many angles as we can before we go ahead and roll it out further. We based our hypothesis on user research, then came up with an idea that we thought would test it. If we had looked at the data as well, we would have seen that there are only around 50 active temporary cards per store, and that these cards are only seen around around twice a month. So…
  4. Targeting active temporary cards isn’t the best way to solve the wider problem.

Learning a lesson cheaply, and on a small scale

We often say it’s okay to fail, but it’s still disappointing when you’ve put time and effort into something. You start picking it apart. Maybe we picked the wrong stores? Or the wrong time of year? Or the wrong colour flyer?

No, those things don’t matter – our idea just wasn’t that great.

Failing is ok, as long as you recognise when to let your idea go and move onto tackling a problem another way. So yes, we failed but we only failed in 10 shops, not all 3,000. We didn’t spend much money, we didn’t inconvenience our users and we were open about how the tests were performing in our weeknotes and our show and tells.

Most importantly we learnt enough to inform where we should focus our efforts next.

We’re moving away from encouraging users to do something towards giving them the tools they need to do it there and then – our next trial will test if customers would register their temporary cards on a tablet in store.

Joel Godfrey
Digital business analyst