Digital development standards

 

Last updated 25 June 2018

Use these standards to check you’re meeting the minimum quality level for custom software development in Co-op Digital. Doing this helps reduce risk.

We hope these standards will:

  • help new teams check that they’re setting up in the right way
  • help existing teams identify if they are going off-track
  • be used by teams to assess the health of their own service
  • be used by any third parties tasked with assessing a service

These standards do not say how you should structure your code, precise aspects of process or how to apply specific technologies –  this will vary depending on the context of your service or project.

1. All code and config is held in a professionally-managed repository

Why is this important?
Losing code, unintentionally overwriting code, confusion over which version of code is correct and the inability to build the application in a repeatable way will all significantly slow down the speed of development and impact system quality. The code and configuration for the system must be safe and available for people to collaborate on.

What it may look like
You can hold your code and config in one or more repositories depending on the shape of their service and its maturity. Any type of repository can be used (for example Gitlab, Github, Bitbucket) as long as it:

  • is a central repository
  • is accessible to the team members who need it
  • conforms to company security standards
  • is available when your team and service needs it
  • is frequently backed up

See if you’re on track
Could you:

  • show someone your repository or repositories?
  • explain how you maintain your system configuration?
  • explain how frequently your code and/or config is backed up?
  • demonstrate that you check your code and/or config in on a regular basis, ideally at least once per-day?
  • show someone where do you store your application build scripts, infrastructure build scripts and deployment scripts?

2. All systems have automated tests

Why is this important?
Good automated tests allow the team to keep moving forward knowing that they haven’t broken anything.

What it may look like
The types of test, number of tests and what they test will vary between systems and can depend on the technology used. Your system may have many unit tests and a few coarser-grained tests or it may balance unit tests, integration tests, API tests, GUI tests and so on. You should be able to run all or a subset of the automated tests easily.

See if you’re on track
Could you:

  • show someone the automated tests in your repository or repositories?
  • demonstrate that your tests (almost) always pass?
  • explain what happens when your tests break?
  • explain when you run your tests and show the output from your last test runs?
  • run your tests now?

3. All application deployment is automated

Why is this important?
Manual deployment to environments is risky and frequently error-prone, especially during time-critical deployment windows. Automating the deployment removes most of this risk.

What it may look like
Deployments to any managed or controlled environment are automated using a tool like Jenkins. A managed environment is any shared environment such as production, pre-prod, QA, training, or system integration test (SIT) environment. Automated deployments may be triggered from previous build pipeline steps or may be manually triggered. If you have to interact manually with the process, keep this to a minimum, like only asking for:

  • the build number
  • the target environment

All other environment-specific information must come from a persistent location such as a repository or a Jenkins build job.

See if you’re on track

Could you:

  • show someone how you deploy to your environments?
  • confidently state how long a deployment takes?
  • describe which tools you use to automate your deployments?
  • show how your deployment scripts are backed up?

4. All environment-building is automated

Why is this important?
Manual building of infrastructure components and environments is time-consuming and error prone. The information on how to build them must be widely available and up to date. Chasing down problems caused by unexpected environmental differences can waste large amounts of time.

What it may look like

Creation of infrastructure components in any managed or controlled environment are automated using a tool such as Terraform. Managed environments would be any shared environment such as production, pre-prod, QA, training, or SIT. You should be able to rebuild the whole environment using the tool with minimal or no manual intervention.

See if you’re on track
Could you:

  • show someone the automated environment building scripts in your repository or repositories?
  • rebuild all your managed environments from these automated scripts?
  • say how long it would take to rebuild your production environment if you needed to?

5. The system can be rolled-back to a known working state

Why is this important?
If a deployment to a managed environment fails, the service must continue working safely or to be restored to a working state as soon as possible. If your software isn’t working it’s not delivering value.

What it may look like
This could be an automated rollback or a short sequence of manual steps (for example shutdown application, restore database, re-deploy previous version, restart application). Any rollback must ensure that the data is restored to a safe, workable state and that data is not lost as a result of the failed deployment.

See if you’re on track
Could you:

  • show the measures you take during development to ensure you can safely roll back to a known working state if a deployment fails?
  • describe what happens if a production deployment fails?
  • identify who authorises a rollback?
  • explain how you would validate the system after a rollback?
  • prove that you have tested a rollback in a production-like environment?

6. The system infrastructure meets the standards of the Digital platform and Digital operations teams

Why is this important?
Most or all of the infrastructure your application needs will be commodity. Following common standards allows you to engage specialists to design, define, diagnose issues and fix problems with the infrastructure, saving a lot of time and effort both at design time and during operation.

What it may look like

You should engage with the digital platform team as soon as you are looking to put any form of working service in the cloud (other than wireframes). The system infrastructure you create must conform to the digital platform standards.  No service will be able to enter the beta phase unless it has agreed:

  • an infrastructure plan with the Digital platform team
  • the management approach to the system infrastructure with the Digital operations team

See if you’re on track
Could you:

  • show that you have spoken to the Digital platform team about your infrastructure?
  • prove that you have an approved infrastructure platform?
  • demonstrate how your infrastructure platform is kept up to date?
  • show that you have spoken to the Digital operations team about your system management?

7. Data is stored according to security and data standards

Why is this important?
Failure to follow security standards may result in service outages, reputational damage, financial loss, customer lawsuits, criminal prosecution, and huge amounts of work repairing the damage.

What it may look like
There are Co-op standards that govern the type and level of protection of data, functionality and infrastructure elements. You must engage with your security business partner to discuss your application and determine where there are likely to be security issues and with the data governance team to ensure their use of data complies to group standards. Together, you can explore the best solution for your service or project

See if you’re on track
Could you:

  • show that your system’s use of data been reviewed by the security and data governance teams? What types of data does your system hold and how do you keep it safe?
  • show someone the results of your last data classification exercise?
  • demonstrate that your system conforms to all the relevant data security standards like Payment Card Industry (PCI), Data Protection Act (DPA) and so on?

8. Errors and events are logged in a way that they can be centrally monitored

Why is this important?
The primary way of determining the health of an application should be through its logs. These logs should be in an understandable format and be sent to a location where they can be monitored by all relevant stakeholders.

What it may look like
Make sure your service logs errors and diagnostic information as set out in the monitoring and logging digital patterns, with specific attention to the logging format.

See if you’re on track
Could you:

  • show someone some recent logging entries generated by your application?
  • point out where your log entries are written to?
  • demonstrate how your logs are monitored and who monitors them?
  • explain what happens if an error entry is written to the logs?

9. The system architecture is documented

Why is this important?
A lot of information is discovered and created as a system evolves, and this information is vital for wider stakeholders and also for your future selves. A lot of information about the system is captured as tests, code and config. Information that can’t be captured this way must be documented.

What it may look like
Information about the main components of the system and its wider architecture is captured in wiki pages. This includes:

  • the overall structure of the system
  • key architectural decisions
  • ways to help people navigate the architecture, including infrastructure architecture, deployment architecture, and development process as well as software structure and data structure

See if you’re on track
Could you:

  • point someone at a wiki location where I can find your system architecture documentation?
  • demonstrate that the documentation is up-to-date?

10. The system is well structured

Why is this important?
A well structured system, with clear separation of concerns, consistent conventions and a coherent structure makes it easier to develop and support the system. All systems tend towards chaos unless you take action to prevent this (see the paper on the Big Ball of Mud).

What it may look like
Each system should evolve its own set of principles on which it is partitioned in order to separate out concerns. The system may be split into multiple services where each service deals with a specific part of your domain (invoicing, payments, catalogue, and so on) using an approach such as Domain-Driven Design or splitting it into microservices.

Your services should provide interfaces and protocols that play nicely with others (see the Co-op Architecture Principles). The system may employ multiple logical layers and spread these over multiple physical tiers, either for the whole application or its constituent services. You may have conventions or rules for the location and packaging of business logic (for example, not in stored procedures and not embedded in the UI but in separate, testable packages). As you learn more about your application, the patterns you apply may change and evolve but you should retain a coherent approach to the architecture.

See if you’re on track
Could you:

  • draw up the essential structure of your system?
  • ask any member of the technical team to draw the essential structure of the system and end up with roughly the same drawing?
  • identify which patterns, principles and conventions you use to ensure good separation of concerns?
  • explain how these patterns, principles and conventions combine to provide a good separation and structure?
  • demonstrate how you decide where a given piece of functionality lives?
  • explain where this is in your architecture documentation?

11. Your team has its own context-specific development standards

Why is this important?
These development standards lay out broad guidelines about the outcomes we expect from development and some of the principles we expect to be adhered to. They explicitly do not set out how you should structure code, precise aspects of process or how you should apply specific technologies since these are specific to the context of your team. However, that set of team-specific standards and conventions should be agreed and written down to avoid wasted effort and to help the maintainability of the application.

What it may look like
Your agreed standards and conventions must be captured in an easily accessible format. To reduce the number of things needing to be documented, these can refer to existing tool documentation, pattern documentation or rulesets for code analysis tools and the like held in repositories. If the standards and rules change they must be updated. You should be able to pick on a current piece of development (both the code and the process used to produce it) and assess it against the team’s development standards.

See if you’re on track
Could you:

  • show someone where I can find your team development standards?
  • prove that your team’s standards up-to-date?
  • show someone some recent code that conforms to your team’s standards?

12. All code and config must be peer-reviewed before it is deployed to production

Why is this important?
We need to keep codebases reasonably consistent for them to be maintainable. Reviewing is a good way to do this and will lead to better designs and fewer defects.

What it may look like
This could be anything from a full, formal code review after a chunk of work, to pull requests (PRs) on short-lived branches, to constant pair or mob-programming. If the code review is formal there must be a mechanism to track any updates required and ensure that those changes are made to the code. If pairing or reviewing PRs, avoid having the same pairs or the same people reviewing each other’s PRs to ensure consistency across the codebase.

See if you’re on track
Could you:

  • explain how code and config changes are peer-reviewed in your team?
  • prove that this process applies to all code and config changes?
  • show someone where this is written into your team development standards?

13. You conform to other relevant Co-op Digital and Co-op Group standards and guidance

Why is this important?
The Co-op Group has various communities of interest who have formal or semi-formal roles in ensuring that the best interests of Co-op are met by teams implementing digital services. Engaging with them and conforming early to the principles and standards prevents wasted work and delays later in the service lifecycle.

What it may look like
You should engage with the architects in your area of the business, with the digital platform team and with the relevant communities (engineering, design and so on) to ensure that you have good sight of the standards, guidelines and patterns that apply to each aspect of the system. They include our:

If you are looking to introduce new products or technology talk to your local Principal Software Engineer or to the Technology Direction Group directly through Slack.

See if you’re on track
Could you:

  • demonstrate an awareness of the Co-op architecture principles?
  • explain how your service conforms to them?
  • show someone a completed architecture self-assessment?
  • demonstrate that you have engaged with the various communities to determine what other standards are commonly adhered to?
  • explain which ones you conform to?