Our data hackathon at Federation House


(Transcript) Alex Waters: So we are in Federation House in the event space. We are hosting the Co-op’s first ever data hackathon.

Data Hackathon 2017, we’ve brought about 50 data analysts from around the whole Co-op. From all the different business units, from different functions, they’re all together today working on some data challenges. We’ve got a few different ones. We’ve got ones around energy consumption, ones around internal costs, ones around using social media data. They’re being creative, they’re adding in extra data where they can find it, they’re not being asked to solve specific questions, they are being given the challenges to solve themselves.

Mike Yates: So, me and my team decided to work on an energy task. So a few teething problems at the start, as you’d expect, trying to get all the data and work out how to use the systems, but getting there now. We’ve potentially found a £27 million saving but I probably shouldn’t say that on camera, it’s probably not right! I need to double check the figures. But potentially, some exciting stuff that we’ve found.

Pralita McCourt: My team are working on looking at how the spend on travel and accommodation was across the business last year. So we’re looking at that 2016 data and analysing for any patterns and any cost saving measures that could be adopted for this year and next year.

Mike Yates: So we’ve got guys I’ve never worked with before, so 2 guys in my team they’re kind of more, what I’d say is back-end system guys and kind of architecture guys which is something that I never normally get exposed to so it’s great to kind of see how they think and how they work. I’d say data is everything in that, you know, we’re ultimately trying to understand people, that’s kind of a lot of what life is all about, and I think data’s a great way, it’s not the only way – it’s a great way to get some real insight into that.

Pralita McCourt: It feels good to be part of it, it’s exciting. It’s good meeting up with other analysts from across the business that you wouldn’t normally come into contact with on a day-to-day basis.

Alex Waters: Right now, I think it is probably the most exciting time to work in data in the Co-op than it has been for many years. I really do expect that as we go through the next couple of years the Co-op is going to get better and better at using data and it will be recognised nationally as a brilliant place for data.

A discovery into data at the Co-op

We’ve been looking at how we handle our data. Over the years we’ve had recommendations from both in-house and consultancy teams about how to do this, but now we want to break the cycle and finish what we started.

Above all we’ve been thinking about how we can take a more ‘Co-op’ approach to our data. We pulled together a multidisciplinary team from across the business to look into this and they’ve become know internally as the ‘data layer’ team (explained in more detail by Rob in his being trusted with data post).

So, where to start?

We want to be trusted with data, and use data to inform what we do. The purpose of the discovery was to explore how we should go about creating the right conditions, both online and offline, to support this; and where to start.

We wanted to understand:

  1. How we deal with data now.
  2. The options for doing things differently.
  3. What we should test, explore and do with the data we have.
  4. What we can, and should, do with data in the longer term.

The purpose of this discovery was to stay at a high level when thinking about data rather than focusing on any one area in particular. We knew that would come later so didn’t want to get into specifics at this stage.

What we did

photograph of 4 members of the team gathered around post-its on the team wall going through the things we've learnt from user research

We needed to understand the problems and opportunities we faced: not just in the Co-op, but in the wider world too. This meant referring to a mix of research, from reading previous data reports to holding interviews and workshops.

The data science team

We started by speaking to the data science team here at the Co-op to get an idea of the challenges the business faces when it comes to data. They identified 47 points that needed to be addressed. We discussed each one and prioritised them in terms of urgency.

Inside the Co-op

We also spoke to different Co-op groups and the Executive team. We wanted to understand how people around the business approach data, how they’re working with it and how we can meet our Co-op ambition of being more trusted with data.

People outside the Co-op

To get a better external perspective we met 4 private companies that work closely with data. We found examples of how easily accessible data can improve the way a business runs, and creates an environment for identifying new product and service opportunities.

The public

Speaking to the public gave the team a different view of data and consent. We like to speak to our members regularly about data in order to be better informed. Understandably people feel strongly about how their personal information is used, and approaches change depending on how this usage is explained and how customers feel about the business in question.

Where we ended up

photograph of walls in the data layer team area showing what the team had found by the end of the discovery

Three main themes came out of our research:

1. We have multiple versions of the truth

Teams work from different databases that don’t necessarily stay in sync, or use consistent definitions. This makes it hard for users (colleagues) to find the best source for information, and be sure they are interpreting data correctly.

2. We under-leverage our data for analysis and insight purposes

Individual teams own lots of data and use it only within their team. We could be much better at sharing data and insights across the Group so it could be helpful for everyone. The problem is partly down to technical constraints, and partly a reflection of our how widespread we are and how differently we work: even within the Digital Group itself.

3. Consent and preferences aren’t understood holistically

Customers opt in to marketing for individual parts of the business, but we don’t have a clear central understanding of the consent an individual has granted across the group. This means that we may not be giving the best customer service that we could.

None of these themes came as a surprise – but the value in the work came from prioritising how we should move forward.

What’s next?

We identified 4 alphas which will be the first step in our vision for a Co-op that is built on, and led by, relevant, transparent and trusted data.

First we need stronger data governance across the whole of the Co-op group, so that our data can become more consistent, more joined up, and easier to find and understand.

Secondly, we should have a single view of each individual so that  all of our information on members and colleagues matches up. We’ll look at membership to help with this.  

Thirdly we want to look at what we mean by ‘consent’ when we ask members to trust us with their data, and we want to be sure this meaning is exactly the same across all Co-op businesses: from Food to Funeralcare.

Finally, we should develop a data science lab so that our data scientists and analysts have the right environment, tools and processes to develop and improve the work they are doing. This will also help us to attract more of the kind of people that will make the Co-op a centre of excellence.

These 4 alphas will start from September, and we will share our progress as we go. It’d be great to hear your thoughts.

Jack Fletcher
Interaction / service designer

Rob McKendrick
Head of Data Engineering

Our data team is growing, and we’re looking for talented people to join us in a number of different roles. Find out more about working for Co-op Digital.

What we learnt from talking to our members about data

On the Friday before our AGM, we held an event at Federation House so we could continue the conversation about how the Co-op uses and shares members’ data. We wanted to invite our members to help us shape our data policy in person. It was an open invitation and 63 people took the time to chat to us.

We ran 3 workshops to find out:

  1. How the Co-op compares to other businesses when it comes to being trusted with data.
  2. What data are people willing to share for social benefit and commercial benefit.
  3. What types of usage of data people are happy with.

Being trusted with data

IMG_5889

In this workshop, we asked people about the organisations they trusted (or didn’t) to use their data, with examples from banks, telecoms, member organisations, broadcasters to bring the subject alive. Depending on their personal experiences with those organisations, people have very different views around who they trust to hold their data securely and use that data sensibly.

An important thing we learnt was that the organisations people trust with their data have very clear reasons for why they hold different data and how they use it. Some people thought that, as the Co-op, we might end up with large amounts of data from across our different businesses (insurance, food, electrical and legal services). They wanted us to be clear about how we use those different types of data. So, as we build new data stores, we need to make sure that we’re careful and transparent when sharing members’ data across the Co-op.

Willingness to share

IMG_5906

Our second workshop asked if members were OK with sharing different types of data with the Co-op – information like their age, gender, salary, religious beliefs. We split the group into 2. One half was thinking about sharing data for commercial reasons, to improve our products and services. The other half was thinking about sharing data for social reasons, like community research.

We gave the groups a scale of how comfortable they were with sharing, from “Not at all” to “Yes please”.  People have many different opinions and different worries, concerns or reasons for sharing. Most people made up their minds with a firm “no” or “yes”, but the reasons for their answer varied widely, and some people changed their mind as the group debated the issues.

What does this tell us? Well, if we’re going to be trusted with holding more data, we’re going to have to give people choices around how their data might be used, both within the Co-op and externally.

Play your consent right

The third workshop was a game where people voted about whether they would give consent to companies to use their data for specific purposes. For example, if people would consent to the Co-op using their habits of purchasing pet food from Co-op stores to let them know about special offers in pet insurance.

Feelings were pretty much summarised by 2 responses:

  1. “I’m an individual – don’t assume what I’m interested in by age, postcode, gender.”
  2. “Maybe I shouldn’t provide more data about myself in case I miss out on special offers that the Co-op targets at particular people.”

We’ll need to think about how to use data to help people find the things we think they are most interested in, whilst not precluding people from other offers.

IMG_5914

Hearing from data experts

We live-streamed the opening discussions and you can watch the keynote and panel discussions on our Youtube channel.

A massive thank you to our experts who gave up their time to travel to Manchester, speak, answer questions and mingle throughout the event. So thank you:

What’s next

It’s not too late to join the conversation because we’ll keep talking to our members about data. Last week we published a post called Speaking to our members about how their personal data is used. We’re going to look at and analyse the feedback we got from the survey more closely to pick up more themes and add what we found out at this event

In the coming weeks, we’ll join Data Leaders to plan improving how we use data across the Co-op. We will also be working with our colleagues in Data Protection. And of course, we’ll discuss all of this with our Members’ Council and advisory boards.

Rob McKendrick
Head of Data Engineering

Our data team is growing, and we’re looking for talented people to join us in a number of different roles. Find out more about working for Co-op Digital.

Speaking to our members about how their personal data is used

Last month, the data team blogged about how we want to become trusted with our members’ data. We said that one of the pieces of work we’re doing to help us strengthen our position of trust is around understanding how members feel about personal data. We wanted to find out what our members thought about their data being used and shared for various things.

A survey to start

We asked members to take part in a survey. 148 people completed it but not everyone answered every question.

Here are 5 things we’ve learnt so far.

1. 50% want more transparency around how data is used1.2.50%-want-to-know-more

Half of the people who took the survey said they’d like to know more about how personal information is used by organisations and businesses, but they don’t know how to find out. To set the Co-op apart from the rest, we should be clear about how we’re using this information and, if people want even more information, we should make that easy to find.

This leads us to our next learning…

2. Members would like to understand how their data is used

We asked members whether they thought they understood enough about how organisations use their personal information.

Three out of 10 members said they feel they know enough or everything about this. Which leaves 7 out of 10 wanting more information or having no idea about how personal information is used. That’s good to know. It indicates we should be explaining more about what we do, and what we don’t do, with member data.

3. What people do before trusting someone with their data

When it comes to thinking about trusting an organisation with personal details, only 2% of people said that didn’t do anything before hitting the submit button.

We asked: “Which of the following do you routinely do before submitting personal information?” The most popular answer was: “Make sure I amend my marketing preferences.” “Check for secure symbols/assurance marks on the website” was a close second.

Only 3 out of 10 people said that they read the terms and conditions or small print though. We think that we could go further in making our privacy notices easier to understand and more likely to be read and understood.

4. Members are happy to share their data for the right reasons

We asked our members if there were any reasons why they’d be happy for us to safely and anonymously share their data. Unsurprisingly for Co-op members, 70% said they’d be happy if it benefited the local community! Some of the comments around this were “Because the Coop is for local people”, “Local community is important” and “Because it’s for a worthwhile reason”.

One respondent said that although they don’t like the idea of their data being shared, the best reason to do it would be if it helped the local community. That person emphasised that their data must be safe and whoever has the data must be accountable. We completely agree and we’ll continue to prioritise data safety. We will also continue to speak to our Member Council and Members as we go.

5. People trust us

Over three quarters (76%) of our respondents said that they trust us to keep their data safe and to use it sensibly (79%). This is also good news. However we can’t sit back and relax on that one. We know that we must do the right thing, not the easy thing in order to bring our Co-op difference to data. We’ll continue to work in the open.

image shows text that says: People trust us and a graphic of two hands shaking an the Co-op logo

Turning learnings into policy

We’ll look at and analyse the feedback we got from the survey more closely to pick up more themes. But the conversation around whether we use and share member data, and how we do it, isn’t closed now. We’re still listening to your thoughts on this. Over the next few months we will start to discuss improvements to our policies around data and, with a lot more information and discussion start to make those policies real.

Tell us what you think

We’re holding an event Shaping our Co-op: Data trust and transparency event at Federation House on Friday 19 May, 5:15 to 7:30pm. You can register for tickets.

We’d like to speak to members regardless of how much you know about how your data is and can be used, as well as people with an interest in data consent. The only prerequisite is that you have an interest in doing the right thing for members.

Rob McKendrick
Head of Data Engineering

Being trusted with data

Being ‘trusted with data’ is something we talk about a lot. It is, of course, what we’d like to happen to the Co-op. To help us reach that point, we’ve done some work to define what it means to us and we’ve just begun to look at how we’re going to develop our ideas in 3 areas.

We spoke to lots of groups within the Co-op including our Digital Advisory Board before deciding that we believe being trusted with data is made up of 3 things. These are:

  1. Integrity.
  2. Transparency.
  3. Meaningful consent.

Image shows a triangle with 'Co-op: trusted with data' in the centre and the each of the 3 corners has one of the following words in it: 'integrity', 'transparency' and 'meaningful consent'.

Here’s how we’re developing these 3 areas.

Being transparent with how we’re using data

We want to build the right set of tools and technology to manage and link our data across the Co-op. We’re calling the concept our ‘data layer’, and we’re using ‘layer’ rather than ‘hub’ because we think our approach should be consistent and integrated across the Co-op, as opposed to being centralised. Most importantly, like Mike said at the Co-op AGM 2016, we’re committed to becoming transparent with how we use our data.

To do this we want to understand more about which processes, habits, culture and tools we should adopt to meet our members’ data needs. Our first step to building a data layer is to start a discovery using a multidisciplinary team and user research-lead approach to find out:

  • what we need to do to be trusted with data
  • which expectations can we set or stretch
  • how can we communicate with them on the topic of data
  • how a data layer looks in terms of engineering and data management and what opportunities that offers

We’ve just kicked off this piece of work so we’ll talk more about this later in the year.

Data integrity

Data integrity is about about making sure that data is correct, well-managed and secure. Our Head of Data Governance and Integrity, Ian Thomas will tackle this in 2 ways:

  1. By helping us think about data standards, usage and regulatory requirements as part of everything we do. Ian’s already working directly with our Membership team to simplify how we handle member data. This will make it easier to be transparent around how we hold it. We’re also working closely with Rob Bowley and the engineering team on information security.
  2. By working towards consistent standards and approaches to our data across the Co-op with the wider data governance community. We believe that by co-operating with our colleagues we can share good practice and improve our governance.

Consent to use and share data

The third area we’re looking at is consent for us to use and share data. This is a topic that organisations don’t like talking about if they don’t have a position of transparency. We’re going to start the conversation with our members about data consent because we think that speaking to them will help us do the right things with data and will help us gain their trust.

Initially we’ll use our Member Voice surveys to start asking questions about data consent, and we’ll be speaking about it at our AGM. We’re also holding an event called Shaping our Co-op: Data Trust and Transparency Event to get feedback from our members on trust and consent. In combination with the data layer research we aim to get a comprehensive view of what we need to build in order to be truly trusted with data.

We’ve just begun these important pieces of work and we’ll keep talking about them.

Rob McKendrick
Head of Data Engineering

Championing a better way of doing data

Blue background with white text that says 'championing a better way of doing data.'

We want to bring the Co-op difference to data. That means going beyond what is simply required by law, and instead infusing the way we collect and handle data with the Co-op’s values.

Practically, we want the Co-op’s data to be: correct and up to date; secure; available to those who need it within the Group and easy to find, understand, connect and augment. That will help us make decisions based on data. We’ll arrive at better decisions more quickly because the information we need will be easy to find and use. It will also help us spot new opportunities across the business, quickly, creating new opportunities because we are joining the dots. We’ll also be able to build better relationships with our partners because data that is well-maintained and with consistent standards can act as common language between us and them.

So, how do we get there? Well, we all have a role. We’ll need to set common standards and provide tools and ways of working needed: data principles.

As importantly, we need to create a culture at the Co-op that isn’t complacent about data and problems with data, but instead fixes those issues at source. We should think and care about how data is used once it is created. Everybody has a role to play in data. Thinking about data and asking how to use it and why will become a habit.

Some of this isn’t new and many people at the Co-op have been doing good work for a long time. Helping and supporting those people to continue to do their jobs is important. That’s why we’ve been convening and meeting with Data Leaders, and why we’re including colleagues from data teams across the business to work out what values we want to hold our data to from now on.

Data and the Co-op values

To help us think about this, we’ve started to look at how Co-operative values like self-help, self-responsibility, solidarity and equity might manifest in data.

We’ve come up with a Data Principles alpha to help colleagues working with data at the Co-op. The principles are based on workshops we’ve had with colleagues, and we’re going to be running more user research sessions to make sure that they are relevant and helpful for colleagues at every level. We’ve done a few versions of data principles, and based on colleague feedback on previous iterations we’re sharing what we’ve learnt publicly.

Important themes

1. Data is part of everything

The data function does not work in isolation. Everyone does their bit to collect and create  good data, which can be used as the basis for making decisions. We are focused on what Co-op members and customers want and need, and respond to that quickly. Colleagues have the necessary tools to do so, and are trained in how to use data and to spot opportunities.

2. Clarity is for everyone

We will communicate how we use and collect data in a way that both specialists and non-specialists can understand. We’ll use consistent terms and standards that are externally recognisable, as well as use plain English to help members meaningfully consent to how the Co-op uses their data.

3. One version of the truth

Major data sets will have a designated owner and steward, who is in charge of keeping them updated, accurate and complete according to defined goals. All significant data sets will be listed and visible to all staff in a Central Data Catalogue, rather than relying on local duplicate, or inconsistent versions.

4. Co-operating safely

We will use data across the business where appropriate and ethical. We encourage co-operating about data, safely and securely, working together for mutual benefit.

We’re still testing these and we’re keen to hear colleague, customer and member thoughts on them. If you have feedback on these principles, leave a comment below and join the conversation.

Catherine Brien
Data Science Director

Local causes and location

Charlotte King recently talked about the changes that we’ve made to make it easier for our members to select a local cause to support with their 1%. I’ve been looking at those local causes, what type they are and where they are in the United Kingdom.

Image showing the spread of local causes being helped by Co-op Members

This image is a scatter plot of those local causes that our Co-op Members are supporting when they buy products and services from us. 

This shows the relationship between the people who live in the UK and the Co-op. The Co-op is supporting local causes right across the nation so where there’s a community, there’s likely to be a good cause that our members are supporting.

What do the colours represent?

When a local cause applies to receive support, they’re asked to classify themselves into a category like ‘Health’ or ‘Environment’ and the colours above are distinct for each category.

I’d love to give you an interactive map where you can explore the data for yourselves but I can’t do that today so instead, I’ve done some exploring for you. 

This scatter plot below just has the categories Health (yellow) and Community Development (purple).

Image showing the spread of local causes being helped by Co-op Members

Both types are widely spread but something I noticed was that there seems to be a split between the urban centres of Greater London and Northern England. London appears to be more densely covered by Health causes whereas places like Greater Manchester, Merseyside, Leeds, Tyneside are more balanced with Community Development causes.

Here’s another plot, this time with Social Inclusion (black) and Young People (blue).

Image showing the spread of local causes being helped by Co-op Members

Social Inclusion causes seem relatively concentrated in certain areas of the country. These tend to be the main cities. Causes within the category of Young People are both numerous and well spread, there seems to be a nation wide concern for causes that will help the next generation. 

Don’t forget we’re looking for the next round of good causes to support in May 2017, the closing date for applications is 16th December.

Alex Waters
Data Science.