Customer experience

Taking a content design approach to how AI could help our colleagues

Co-op
A colleague in a Co-op store is standing in the aisle of a store holding a small handheld device. She is inputting information into the handheld device. There are jars of jam and containers of coffee on the shelves behind her.


Our ‘How do I’ (HDI) website was created by content designers pair-writing with store and operational colleagues. The aim was to provide operational policy information, in a way that was easy to understand, in a busy store environment.

Store colleagues rely on ‘How Do I’ to comply with legal regulations and maintain high standards of customer service. Colleagues tell us it’s useful, but difficult to find some information quickly. Our Content Design and Data Science teams worked together to test how using generative artificial intelligence (AI) and a large language model (LLM) could help.

It proved to be a great opportunity to learn from how content designers can work with teams who want to make the most of AI capability.

Taking a content design approach

As a Content Design team at Co-op, we create content that is evidence-based, user-focussed, and based on shared standards to meet our commercial goals. We want to keep these content design principles at the centre of our approach to AI generated content.

The teams designed a process that combined a Co-op built AI and a Microsoft LLM. It means that when a user enters a query, a Co-op built AI system looks at a copy of our ‘How do I’ website and finds the information that is most likely to be relevant. It takes this data and the original question, and feeds it all to a Microsoft LLM. The LLM then generates a response and passes it back to the user as an answer.

How the AI works

There are a number of illustrations to show a process of how the AI works in steps. Illustration 1: hands using a phone: Colleague types a question into AI HDI Illustration 2: a screen with a magnifying glass and options: AI search engine looks up relevant information. from HDI. Keyword and semantic search. Passes the question and relevant info to LLM Illustration 3: Letters LLM in a file: LLM generates a response and sends it back to the AI Illustration 4: Mobile showing a list :Answer is provided to colleague

All of the content on the ‘How Do I’ (HDI) website was created and designed according to content design principles. As a result of the way LLMs work, without content design expertise, LLMs generate new content that is not subject to the same rigorous user-focussed design processes.

We needed to test how the AI was working to make sure it does not give misleading, unclear or inaccurate information. We analysed search data and worked with colleagues to identify the common queries they search for. This helped us to build an extensive list of test questions covering a wide range of operational, legal and safety related themes.

Testing and analysing the AI responses

When we tested the AI system with questions, we used the language our colleagues used. We asked simple questions and complex questions. We included spelling mistakes and abbreviations, then we analysed the AI system responses.

We took a content design approach and used our content guidelines to assess the responses. Validating the accuracy of responses included fact checking against the original ‘How Do I’ content to understand whether the AI had missed or misinterpreted anything.

We used this analysis to create a number of recommendations for how to improve the content of the AI responses.

Accuracy

Almost all the AI system responses provided information that was relevant to the question. But analysis showed it sometimes gave incorrect, incomplete or potentially misleading information. ‘How do I’ contains a lot of safety guidance, so to avoid risk for our colleagues, customers and business, we needed to make sure that any responses are always 100% accurate.

Accessibility

The initial AI system responses were hard to read because they were stripped of their original content design formatting and layout. Some of the responses also used language that sounded conversational, but added a lot of unnecessary words. LLMs tend towards conversational responses, which can result in content that is not accessible. It does not always get the user to the information they need in the simplest way.

Language

The AI did not always understand some of our colleague vocabulary. For example, it struggled to understand the difference between ‘change’ meaning loose coins, and ‘change’ meaning to change something. It did not understand that ‘MyWork’ referred to a Co-op colleague app. This meant it sometimes could not give relevant answers to some of our questions.

Using content design to improve the AI

Our Content Design team is now working with our data science team to explore how we can improve the AI system’s responses. We’re aiming to improve its accuracy, the language the AI uses, and reduce unnecessary dialogue that distracts from the factual answers. We’re also exploring how we can improve the formatting and sequencing of the AI responses.

This collaborative approach is helping us to get the most out of the technology, and making sure it is delivering high quality, accessible content that meets our users needs.

Based on the content design recommendations, our data science team have made changes to instructions that alter parameters for the AI, which is also known as ‘prompt engineering’. This affects the way the AI system breaks down and reformats information. We’re experimenting with how much freedom the AI has to interpret the source material and we’re already seeing huge improvements to the accuracy, formatting and accessibility of the responses.

Impact of the innovation of this AI work

“The ‘How Do I’ project has been hugely innovative for the Co-op. Not only in the use of the cutting edge technology, but also in the close cross-business collaboration we needed to find new solutions to the interesting new problems associated with generative AI. We’ve worked closely with Joe Wheatley and the Customer Products team, as well as colleagues in our Software Development, Data Architecture and Store Operations teams. We’ve been able to combine skills, experience and knowledge from a wide range of business areas and backgrounds to build a pioneering new product designed with the needs of store colleagues at its core.”

Joe Wretham, Senior Data Scientist

The future of AI and content design

AI has so many possible applications and its been exciting to explore them. This test work has also shown the critical role content design has in making sure we are designing for our users. AI can create content that is appears to make sense and is natural sounding, but the content needs to help users understand what they need to do next, quickly and easily.

Content designers understand users and their needs. This means understanding their motivations, the challenges they face, their environment, and the language they use. The testing we’ve done with the ‘How do I’ AI system shows that AI cannot do this alone, but when AI is combined with content design expertise, there are much better outcomes for the user and for commercial goals.

The content design team at Co-op have been exploring how they can balance current content design responsibilities with exploring skills and new areas for development in AI.

Blog by Joe Wheatley

Find out more about topics in this blog:

How user-centred design reduces risk for colleagues and our Co-op 

Matt Tyas

We’ve followed the Horizon Post Office scandal with empathy for everyone that it has impacted and is still affecting. It’s clear that the postmasters and their families were failed on many levels and we cannot address them all here.   

Looking at it from a digital technology perspective, it shows how important it is to build systems using user-centred design. Working in a user-centred way plays a valuable part in designing the right solutions for colleagues and customers. Listening to them, and questioning technology and processes, provides confidence that you are meeting their needs. It also mitigates the high-level risks and consequences of not testing or having active and open feedback channels. 

How we work in product teams to understand user needs  

User-centred design is based on understanding the tasks users need to perform and the environments they are in. It reduces the potential for us to negatively affect anyone who interacts with the Co-op.

We have specialists within our teams that make sure that our services are user-centred and delivering value to the Co-op. That value could be commercial, or creating efficiencies in how we work. 

Although skills often overlap, each specialism is an important part of a product team. Collaboration between disciplines helps us to consider everything within a user’s experience and design the right solutions. 

User researchers 

User researchers talk to the users of our services and provide insights to help the team make decisions. They empower team members and stakeholders to fully understand user needs and build confidence through testing. User researchers also help to identify and mitigate any problems with our services.  

Interaction designers 

Interaction designers are sometimes known as UX (user experience) designers. They help create accessible interfaces and consistent user experiences to solve user problems.  Interaction designers do things like sketching, creating digital prototypes and producing digital designs for a product or service.

Content designers 

Content designers create and organise information in the clearest way to help users complete tasks. They work closely with user researchers, interaction designers and engineers to make sure the content is accessible and easy to understand. 

Service designers 

Service designers design the end-to-end journeys of our services. They help teams to think about all channels to help users complete their goals. They align their work with business needs and measurable value. 

Product managers  

Product managers focus on the product vision, providing direction on objectives, strategy, the Co-op’s goals and wider market. They help to assess the value of work, prioritising it into plans that meet the team goals and contribute to sustainable growth.

Product owners  

Product owners translate strategy and objectives into tasks for designers and engineers to enable the team to deliver the product. In smaller product teams the product manager will also perform the duties of the product owner. Both roles work strategically and need to communicate with the team on how to achieve goals.  

Delivery managers 

Delivery managers enable their team to build and iterate user-centred services. They remove obstacles to progress, helping the team to explore better ways of working and deliver outcomes more effectively. 

Engineers 

Engineers craft the code that makes our digital products work for our users. Our engineers build software with users in mind and follow standards to ensure people have the best experience when they use our products. 

Quality coaches 

Quality coaches embed quality into every stage of product development, working with product, design, delivery and engineering specialists. They take a risk-based approach to tackle any problems early and deliver a high quality product or service.

Subject matter experts 

We work closely with the people who do the jobs we’re designing for (or the customers they serve). They are the experts, and we listen to their expertise and experiences, often co-designing solutions with them. 

Supporting teams 

At Co-op we take a service-first approach and the technology teams that support us make sure that our digital products are secure, robust and accurate.  

Why we start small and iterate  

We gradually improve products and services over time, which is sometimes called an ‘agile’ way of working. By using quick cycles of experimentation, learning and releases we can deliver value early and change direction quickly. If we learn something new about our market or spot any problems, we can fix it straight away and build everything else around a solid foundation.  

We define the most important features first, then work on the less important features over time. 

How we test to help us learn and improve  

We test to validate new ideas or create a better solution to an existing service. We use mock-ups, sketches, and other low-fidelity visuals like coded prototypes. By testing early, we can develop onto higher fidelity versions and products with more confidence.  

When we release products early and often, we reduce the risk involved in complex solutions. We also create value for Co-op and our customers or colleagues sooner. We test results consistently to see what’s working and what needs to be better. 

Why we collaborate and empower our team members 

We value collaboration and empowerment across teams. A product team owns their product and should be in full control of making changes to it.   

We collaborate closely with other teams and stakeholders to make sure that we’re considering all the factors that influence a product’s success.  

This means decision making sits closely with the experts of the product and its users, so that we can move quickly and gain the most value from our time. 

How user-centred design helps us avoid mistakes 

We make a minimum version of our work live as soon as we’re sure that it is working for our colleagues and customers. If a simple version is working well and doing what it needs to do, then we can build additional features on top.  

Fixing problems early or before we make something live, also helps us to save time and money. We avoid the expense of making changes on a higher fidelity product later. Most importantly, we minimise exposing our customers and colleagues to systems that impact them negatively or cause them harm. 

At Co-op we always want to do the best we can for our members, customers and colleagues. User-centred design is an important part of making sure we do this for our digital products and services.   

Thank you to the Content Design community and Customer Products team for their collaboration on this post.

Matt Tyas – Head of Design.

More information on topics in this blog post:

How our users influenced our new forms guidance

Co-op

The Experience Foundations team recently updated our guidance on forms in our Experience Library.

Diagram of a web form with markers showing where different form elements are place and what they do

Originally, this piece of work was about making sure we included all the components we knew our community needed. But as we got further into the research, we found our community needed guidance on aspects we hadn’t considered.

In the Co-op Customer Products team, we value having the autonomy to be flexible and divert from a plan when we need to. So, with the aim of meeting newly-discovered user needs, we pivoted our work.

A recap: the importance of familiarity in design

Co-op has many business areas and many products and services within them. In most, there’ll be at least one form that, for example, asks a customer for personal details to register for something, or asks for a customer’s payment details so they can buy something. Although our business areas are diverse, it’s important that all of them use a common design language to create familiarity. This means that interactions work in the same way in each service and each one feels like it belongs to Co-op. This helps us build trust with our users.

Starting with research

As always, we started with research. This involved one-to-one conversations with colleagues from a wide range of teams and disciplines to better understand their needs. The conversations helped shape our focus and we ended up with a list of form components that our community needed. Our goal was to design, build and release these components into the Experience Library.

New information = new direction

However, during the conversations, a new theme emerged around the structure and layout of forms.

Although our original research didn’t highlight this as an area of need, feedback from newer members of the community made it clear that this was important but there was ambiguity.

Some of the questions they asked included:

  • What spacing should I use between field sets, labels and buttons?
  • Is it better to use single or double columns for laying out forms?
  • Where should I position buttons?
  • How should I show optional or required fields?

We realised our community needed more than form components and guidance on when and how to use forms – it needed guidance on designing single or multi-page forms from the ground up.

Getting a deeper understanding of the problem

The outcome we were aiming for was for all design colleagues to be comfortable and confident setting up forms for the products and services they look after. So we needed to understand the practices that already existed, and also what change was needed.

Here are 4 things we did to deepen our understanding.

1. Carried out user research

We facilitated conversations with newer members of the design community. We asked questions like:

  • When designing a form, what did you feel unsure about?
  • What guidance did you expect to find in the Experience Library for designing a form?
  • Is there anything else you feel would have helped you in designing a form?

These open questions helped us understand which areas needed clear guidance.

2. Reviewed Co-op forms

When we started the forms work, we reviewed forms across Co-op products and services. We went back to the analysis we did but this time we focused on layout and structure and therefore the usability rather than individual components.

This helped identify variations in form design across Co-op.

3. Analysed other design systems

We looked at the guidance other design systems had on form design. An important take-away was how some design systems used visuals to explain guidance.

4. Revisited best practice

We revisited forms specialists Caroline Jarrett and Adam Silver’s work on forms and considered how it applies to our form design at Co-op.

Designing the ‘Form design’ page

Content designers and interaction designers worked together to define the topics that our guidance should cover. We had some difficult conversations to help us understand different takes on the same topic and often challenged each other’s view. Referring back to the insights allowed the team to have those difficult conversations. We reflected on different perspectives and continually iterated on the content. Through this process we were able to define our stance on things like button positioning. Once we were aligned, we added detail and referenced the insights we’d found in the research.

We also found the need to visualise some of our guidance. For this, we defined a visual language that can be used on diagrams in the future.

Diagram showing how a form in one column is easier to use than a form in 2 columns

We shared early versions of the page with people from the Design, Product and Engineering communities to review. We value different perspectives, and want others to contribute to our work. By designing in the open, our community sees our approach, which helps build trust. Showing them the depth of our process encourages buy-in and the early feedback in the reviews was positive.

A ‘people-first’ design system

Our new Form design page wouldn’t exist without the feedback from our community. We designed it for them, based on conversations we had with them. Delivering guidance that meets their needs shows that we’re listening, we’re collaborative and this builds trust with our colleagues. Our work is less about a page in a design system, and more about the people that use it. We’ll keep listening and iterate when we need to. Like the rest of the Experience Library, this page will evolve with our community’s needs.

Imran Afzal, Lead Designer

Simulating in-store experiences with physical prototyping

Co-op1 Comment

The Customer Experience (CX) team has been working with our Co-op Food colleagues to look at how we can improve customer service in our stores. When the CX team help the wider Co-op business solve problems, our process usually involves prototyping. Because we often work in the digital space, our prototypes are often on a screen too.  

This challenge however focuses on in-person experiences in our stores. So, for this piece of work, testing in a physical space and in a more tangible way felt more appropriate. 

Before trialling in a store, we wanted to test our ideas in a low-risk environment where we wouldn’t be in the way of day-to-day store life but where we could still involve colleagues who bring other expert knowledge.  

We used a ‘desktop walkthrough’ method to simulate the in-store experiences. 

We are writing this post to share: 

  • why we chose the desktop walkthrough method as a prototyping tool 
  • how we used it to get a better understanding of our trial logistics 
  • what we learnt about using a less familiar method 

Exploring the problem with a team of experts 

To discover how we can improve customer service in store, we needed to understand the current customer experience and identify pain points.  

We formed a small team of colleagues across Food Operations, Insight and Research, and store managers to help us focus on the right things. Each discipline has its own perspective and involving the right people means we’re more likely to focus on the right things.  

Defining the problem and prioritising 1 concept to tackle

Based on our research, we identified 3 areas we could explore that would help our customers receive (and our colleagues to be able to provide) better service. They were: 

  1. Technology – how might we use new and existing technology to make improvements across different parts of the customer journey? 
  1. People – how might we help our colleagues to prioritise service through training and recognition? 
  1. Insight – how might we make better use of the insight we have on our customers, colleagues and stores to make improvements to customer service? 

We chose to explore the ideas focused on people because we identified the most amount of value, opportunity and feasibility here. We specifically wanted to look at how we might recognise colleagues who were great ‘customer service advocates’ in stores.  

We defined our hypothesis and used it to develop a plan for our trial in a real store. We established the basics of good customer service, and we defined the role of a customer service advocate.  

Choosing an inclusive and lightweight way to test  

To choose the right prototyping method for the scenario, we revisited what we wanted to learn. Our learning objectives were to: 

  • get a shared understanding about the end-to-end customer experience 
  • understand the important interactions between colleague and customer journeys 
  • identify other problem areas so we can address them 

We decided to try a desktop walkthrough because: 

  1. It brings experts from different areas together, in one room, without distraction so we could explain why we had arranged the walkthrough and what we planned to do afterwards in real stores. Each person has a unique perspective and can raise challenges the rest of the group wouldn’t necessarily consider. 
  1. We could figure out our next steps without getting in the way of or taking time away from in-store colleagues. 
  1. We had a hunch it might help us realise things relating to the physical space we otherwise likely wouldn’t have with a different method. For example, shelving and fixtures tend to be tall and make it difficult for colleagues to see each other providing good service.  

The set-up 

As the name implies, the walkthrough takes place at a desk. The Format team shared a generic store floor plan which we printed out and laid on the desk. Then we added 3D card shelving, tills and self-checkouts on top of the paper layout to recreate a mini-scale, realistic-as-possible store. We used figurines to represent colleagues and customers. 

photograph shows 3D card shelving, tills and self-checkouts on top of the paper floor plan
We added cardboard tills, self-checkouts and shelving on the floor plan.

Walking through scenarios 

We chose to walk through common scenarios for store colleagues. For example: 

  • opening the store  
  • navigating around the store at the times when there are fewer colleagues on the shop floor  
  • operational tasks such as unloading deliveries or scanning gaps on the shelves – times where a colleague is less available to directly help customers  
  • customer interaction trade-off scenarios like helping a customer to find an item while being asked over headset to pack a Deliveroo order 
image shows the full floor plan and has figurines at either side that represent customers and colleagues
We grouped customer and colleague figurines around the floor plan as we walked through scenarios.

We also took note of real colleagues’ shifts, lunch breaks and list of tasks too so we could get an idea of how busy the space would be. Weaving this into our walkthrough brought an additional layer of understanding for the people in the room. 

A desktop walkthrough meant we got a bird’s eye view of colleagues moving through our model store for the duration of their shift. It also helped us see where, when or why colleagues interact with customers. 

image shows the back of a colleague figurine facing the store floor [plan and other figurines in the distance
Customer team member number 3 is in the stockroom dealing with a delivery here

Building value for our CX team and the wider community 

Our desktop walkthrough was a quick, cheap way to prepare for an in-store trial. Bringing our ideas to life in this way meant we picked up on things that might not work in stores and we could adapt our concepts without wasting time or money. A lot of this was down to 2 ex-store managers who joined us for the walkthrough – their input was invaluable. Their first-hand experience of working in – and running – stores meant they could sense-check our assumptions which made the scenarios we walked through far more realistic. We made changes to our experiment plan based on their insight and we believe this contributed to the success of our first store trial. 

Since our desktop prototype we have progressed to trialling our customer service advocate concept in stores and continue to learn and adapt. 

Steph Clubb, Lead CX visual designer  

Hannah McDonald, CX strategist 

Introducing Co-op’s Customer Experience Strategy team

Co-op

Co-op recently created a new Customer Experience (CX) Strategy team. This post explains why our team exists, our purpose and how we work. 

What we mean when we say ‘customer experience’

Customer experience (CX) is how a customer thinks and feels about all interactions they have with a brand. Customers no longer base their loyalty on price or product. Instead, they stay loyal to brands that offer the best experiences. This means brands can gain a competitive advantage by providing customers with a consistent, personal and rewarding experience.

We need to consider CX across the whole customer journey 

At Co-op, we offer a varied range of products and services. Customers can come to us to buy both pet food and pet insurance. They can pick up today’s dinner from a Food store or prepare for their future through Life Services. They can place an online food order or plan their funeral. And along the way, they have many different interactions with us. 

Speaking to a Co-op colleague in a Food store is just one of the many interactions that customers can have with us

By meeting or exceeding customer expectations every time they connect with us – whether in physical or digital spaces – we create better experiences for them. This means customers will be more likely to continue to use our services and to recommend Co-op. In the long term, this helps us gain a competitive advantage through: 

  • better retention 
  • more effective cross-selling 
  • bigger customer networks 

Lots of our colleagues are already working to create better customer experiences. But an approach that works across the whole business and considers the entire end-to-end experience for customers is a new and exciting opportunity for us. This is where the new CX Strategy team comes in.

Our CX Strategy team is responsible for the holistic customer experience across Co-op 

The CX Strategy team works in partnership with colleagues across the business to create seamless journeys that solve customer problems and improve their experience. 

We: 

  • collaborate with business areas, working alongside them to develop actionable CX strategy 
  • shape strategies based on customer insights 
  • join the dots across different teams, systems and processes 
  • define opportunities for improving the end-to-end experience for customers 
By considering the whole end-to-end journey customers have with us, including the ways we connect then with their community, we can improve their experiences

The CX Strategy team is partnering with teams across the business 

As well as CX strategists, the CX Strategy team is made up of experts in content strategy, research and service design. When we partner with a business area team on a project, it’s important that we begin by understanding the current landscape. We ask the team to share their expertise on their business area and customers with us. We then work with them to map customer journeys and identify points of friction. As we move through the process of exploring and setting the strategy, we’re able to distil our focus and make recommendations. This helps us create a realistic implementation plan that the business area team can put into action. 

A diagram outlining the working approach that the CX Strategy team use

So far, we’ve: 

  • worked with Life Services to create a customer experience strategy grounded in insight, making changes across a customer journey that crossed two business areas to generate new revenue streams 
  • worked with Nisa to understand the current wholesale customer experience and identified opportunities that have the potential to increase sales by millions  
  • mapped how we’re measuring CX at Co-op 

Next, we’ll be working with Membership, our customer service centre, Co-op Power and Food. We’ll be focusing on creating customer experience that works for our customers, members and communities and that also benefits our business. 

The CX Strategy team 

Co-op colleagues can join our ‘Customer Experience Spotlight’ talks 

We’re marking CX Day 2021 with a series of CX best practice talks on Tuesday 5, Wednesday 6 and Thursday 7 October.   

If you’re a Co-op colleague, you can sign up to join our lunchtime Customer Experience Spotlight talks to find out how Insurance, Life Services and Food are championing CX.