Skip to main content

Creating resources in Cloud Platform

Namespaces

Cloud Platforms host a Kubernetes cluster which is logically divided into namespaces aka environments. We deploy applications into these namespaces using Helm. Each application will generally require three namespaces/environments - dev, preprod and prod.

Sometimes it makes sense to group applications together into a single namespace, for example you may have a related UI and API that belong together. This is fine but we would still require multiple environment namespaces for dev, preprod and prod.

Sometimes we need additional environment namespaces, for example some probation applications also have a staging namespace to work with the Delius staging environment. This is fine but will mean some extra work in the helm configurations when copying the template applications.

There is a plan to incorporate into the bootstrap process the creation of new namespaces (or updating of existing ones) for new applications. This would automatically create (or update) dev, preprod and prod namespaces; this document will be updated when the process is complete, with details of how to add other namespaces.

Requirements

Do I need a new namespace?

If your application is in a new service area you will definitely need namespaces for the dev, preprod and prod environments. If your application fits nicely alongside existing application(s) you can skip creating a new namespace and reuse the existing ones. However, you will probably need to create a Kubernetes certificate resource for your new application.

Creating new namespaces

If you do need new namespaces then you will need to add some terraform to the cloud-platform-environments git repository.

Each new namespace needs new Kubernetes resources such as an ingress (to allow inbound traffic), access roles (to give humans and build tools access to the namespace), a certificate (to use TLS) amongst others.

dev namespace

Initially we will create a dev namespace. We’ll refer to this as <your-new-service-name>-dev where <your-new-service-name> is whatever you’ve called your new service.

  • In the cloud-platform-environments repo create a new branch
  • Look in directory namespaces/live-1.cloud-platform.service.justice.gov.uk
  • The github Kotlin and Typescript template projects are both configured to use the hmpps-templates-dev directory, and it can be used as the basis of your project’s namespace configuration for the -dev, -preprod and -prod environments with appropriate changes.
  • Create a copy of this template directory, and rename it, appending dev to your new service name, e.g. <your-new-service-name>-dev

You will now have a bunch of Kubernetes definition files in your new directory. There will be several places that need updating:

  • Change the defaults under directory resources/variables
  • Replace the template namespace references with your new namespace name (e.g. replace all instances of hmpps-template-kotlin with <your-new-service-name>-dev). Some projects will not have a github repository at this point, the following stages after creating the namespaces will create this for you. The cloud-platform.justice.gov.uk/source-code: for you project will be https://github.com/ministryofjustice/<your-new-service-name>.git
  • When finished perform a quick search for the word template in your new directory in case you missed any
  • Add your GitHub team to the rbac yaml - copy the existing subject of kind Group but replace hmpps-sre with your GitHub team. (Note: please copy, do not replace the hmpps-sre Group - it is required for administration tasks on the namespace).
  • Update the certificate’s DNS name. By convention we prefer <your-new-service-name>-dev.hmpps|prison|probation.service.justice.gov.uk - where you choose one of hmpps, prison or probation
  • Check that the certificate’s DNS name isn’t longer than 63 characters - if it is you’ll need to choose a shorter name
  • Make sure the certificate’s property spec.secretName doesn’t include the word dev. This secret name should be the same for all namespaces/environments.

Once your updates are complete push your branch back to the repo and raise a new Pull Request. You’ll then need to ask on MOJ Slack channel #ask_cloud_platform for a review of the PR.

Once approved, merge the PR.

A Concourse job will run in the background which should eventually create your new namespace. When the namespace is ready the command kubectl get namespaces | grep <your-new-service-name>-dev should return your new namespace.

Updating Cloud Platform namespaces

The key to being able to deploy with Github Actions is storing Kubernetes credentials within Github itself. The good news is that Cloud Platforms can facilitate this by creating service accounts that can populate the environments section of Github, and both apply permissions for approval/deployment and store the secrets so that they can be referred to by Github Actions.

Furthermore, these secrets can automatically be rotated, improving security. This section details how to create the service account and populate the environments

Namespaces and mappings

It is assumed that each deployment environment has its own namespace, but this namespace can be shared by a number of components within a product (eg. ui and api).

This is the important bit: for this to make sense, the environments should all match: (links show examples within the template repo)

There are also three configurations required to be added to the namespace - these can all be found under the hmpps-templates-dev/resources folder:

  • Github service account (resources/serviceaccount-github.tf)
  • Application Insights secret (resources/applicationinsights.tf)
  • Time definition (additional lines within resources/versions.tf)

Github Service acccount

Within the hmpps-templates-dev namespace definition there is a file called resources/serviceaccount-github.tf

This should be copied into the appropriate namespace’s resources directory, and the following changes made:

  • github_repos

Each of the repositories that uses this namespace should be listed on row 5

  • github_environments

This refers to the var.environment variable within resources/variables.tf - this is one of the parameters that needs to match with all the others.

  • data “github_team”
  • resource “github_repository_environment” “env” -> reviewers

These configure the review constraints for the environments. Add the appropriate github teams that are authorised to review and approve deployments for this environment, based on the existing entries.

Application Insights

The new template projects require the application-insights key for the appropriate environment to be defined within its own secret. This is defined in the appinsights.tf file which should be copied to the resources directory of the namespace.

These keys are managed by AWS SSM parameters; there’s one for dev (labelled ‘t3’), one for preprod and one for prod; the env variable can be used to match these particular environments so the file can be copied as-is. However, in other cases, but in the case of (for example) staging, the appropriate suffix will need to be hard coded, so if staging traffic will go to the pre-production Application Insights endpoint:

data "aws_ssm_parameter" "application_insights_key" {
  name = "/application_insights/key-preprod"
}

Time Definition

So that tokens can be rotated, the following entry should be added within resources/versions.tf:

    time = {
      source  = "hashicorp/time"
      version = "~> 0.9.0"
    }

Once this PR has been approved and the Terraform has run, the Github repositories to which this namespace applies will have a corresponding environment entry within the settings page, environment link.