Deploy Services

This page explains how to deploy the FRIDGE services to a FRIDGE tenancy. This process includes configuration for various components such as Argo Workflows, MinIO, network policies, and other infrastructure settings. It does not deploy the Kubernetes clusters within the FRIDGE tenancy; instead, it assumes that Kubernetes clusters have already been deployed.

Deployment¶

A FRIDGE consists of two Kubernetes clusters: an access cluster and an isolated cluster. The access cluster hosts the Harbor container registry and an SSH server for accessing the isolated cluster. The isolated cluster hosts the FRIDGE services. The deployment process uses Pulumi to manage the infrastructure as code. You will require appropriate Kubernetes contexts set up for both clusters.

Pulumi Backend¶

You can use any backend you like for Pulumi. The Pulumi documentation details how to use various backends. For local development and testing, you can use the local backend:

pulumi login --local

Access cluster¶

First, navigate to the infra/fridge-dual-cluster/access-cluster/ folder. You will deploy the access cluster first, as it hosts the Harbor container registry and SSH server required to access the isolated cluster.

Virtual Environment¶

Set up a virtual environment for this project. You can use the following commands:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Creating a stack¶

The infra/fridge-dual-cluster/access-cluster/ folder already contains a Pulumi project configuration file (Pulumi.yaml), so you do not need to run pulumi new to create a new project. The Pulumi.yaml file defines the project name and a schema for the configurations for individual stacks.

To create a new stack, you can use the following command:

pulumi stack init <stack-name>

Configuring your stack¶

Each stack has its own configuration settings, defined in the Pulumi.<stack-name>.yaml files. The configuration can be manually edited, or you can use the Pulumi CLI to set configuration values. You can set individual configuration values for the stack using the following command:

pulumi config set <key> <value>

Some of the configuration keys must be set as secrets, such as the MinIO access key and secret key. Those must be set using the Pulumi CLI using the --secret flag:

pulumi config set --secret minio_root_password <your-minio-secret-key>

It is critical that you set all required configuration keys before deploying the stack. In particular, you will need to supply a public SSH key that will be added to the SSH server in the access cluster. If you do not do this, you will not be able to access the isolated cluster later.

For a complete list of configuration keys, see the Pulumi.yaml file.

Kubernetes context¶

Pulumi requires that the Kubernetes context is set for the stack. For example, to set the Kubernetes context for the dawn stack, you can use:

pulumi config set kubernetes:context dawn

This must match one of the contexts in your local kubeconfig. You can check the available contexts with kubectl:

kubectl config get-contexts

Deploying with Pulumi¶

Once you have set up the stack and its configuration, you can deploy the stack using the following command:

pulumi up

Isolated cluster¶

You will deploy the isolated cluster next, as it hosts the FRIDGE services. Navigate to the infra/fridge-dual-cluster/isolated-cluster/ folder.

However, two additional steps are required before deploying FRIDGE to the isolated cluster.

1. SSH port forwarding: You must set up SSH port forwarding from your local machine to the isolated cluster via the SSH server in the access cluster. This is necessary because the isolated cluster has a private API server endpoint, which is not directly accessible from outside the access cluster. You can use the following command to set up SSH port forwarding:

   ssh -i <path-to-your-private-ssh-key> -L 6443:<isolated-cluster-api-server>:443 fridgeoperator@<access-cluster-ssh-server-ip> -p 2500 -N

   Replace <path-to-your-private-ssh-key>, <isolated-cluster-api-server>, <ssh-user>, and <access-cluster-ssh-server-ip> with the appropriate values for your setup.

2. Kubernetes context: You must set the Kubernetes context for the isolated cluster stack to use the local port forwarded to the isolated cluster’s API server. Edit the kubeconfig file for the isolated cluster to point to https://localhost:6443 for the API server endpoint. Then, set the Kubernetes context for the stack using the Pulumi CLI:

   pulumi config set kubernetes:context <isolated-cluster-context>

Once these steps are complete, you can deploy the isolated cluster stack using the same pulumi up command as before, after setting up the virtual environment and creating the stack, and setting the required configuration keys.

FRIDGE deployment targets¶

Currently, FRIDGE is configured to support deployment on Azure Kubernetes Service (AKS) and on DAWN AI. FRIDGE uses Cilium for networking, and thus requires a Kubernetes cluster with Cilium installed.

In the table below, you can see the components need to be deployed to each target. Some components are pre-installed on DAWN.