Co-op Digital Blog

I’m holding my hands up and admitting my team recently made a mistake. We’re guilty of using high-tech laptops to build software intended to run on far less powerful devices.

Testing in context is something all digital teams should do automatically. But while chatting to a couple of other software engineers, I learnt that forgetting to do this is a common developer sin. Or at least, it’s more common than it should be.

I hope that by writing this post it’ll help keep it in the forefront of people’s minds.

What we were working on

I’m part of the Membership team. Recently, we wanted to find out if members would find it useful to check their rewards balance at the start of a shopping trip, so we’ve been building a prototype that members can use in stores.

Where we fell down

The prototype we were building was difficult to run on the device we’d be testing it on, ie, the small Android tablets in stores. The tablets weren’t chosen for their performance, they were chosen so we can keep the cost down while we’re still experimenting. To make the prototype run faster, we’d been using the browser on my laptop while we we doing the development work and showing demos to the team. In other words, we’d prioritised speed over accuracy.

A few days after spinning up the prototype, we tried it on one of the tablets. It didn’t work in the same way. We’d wanted to get it into the hands of real users as soon as possible so we could check our assumptions and find out how well it met user needs so by this point we’d booked in some user testing in store. Unfortunately, we couldn’t fix it in time.

Not a disaster, just a delay

Working in an agile way allows us to overlook relatively small things and make little mistakes like this. The beauty of agile means we’re never going to spend an outrageous amount of time or money correcting mega problems. For our team, we had to delay the user testing which was frustrating (we should’ve known better), but it wasn’t a disaster.

Easy to fix

We spent a day accounting for the differences between the Android tablet and the browser on my machine. We used a service called ngrok.io so the tablet could use the website running on my laptop and we could develop against the low-powered device. This way we could test over a representative 3G network. It’s what we should have done all along.

But easy to avoid too

As I said, testing prototypes in context or in an environment as close to reality as possible is basic but sometimes, for whatever reason, there’ll be occasions when we trip up.

To avoid this:

• keep in mind the device, or types of devices, your users will be using your product or service on, both before and after you begin building (this is the big one, obviously)
• involve your quality assurance testers (QAs) as early and as frequently as possible
• run the software on realistic devices as early and as often as possible…
• …but never assume that emulating a device in a browser will be exactly the same as it would on a device

We’ve learnt our lesson. Let this be a reminder to you!

Paul D’Ambra
Software engineer