# Hopsworks helm chart


[![Spin boot up test (OVH)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-test-ovh.yml/badge.svg)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-test-ovh.yml)
[![Hopsworks workflow tests in kube](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/workflow-tests.yml/badge.svg)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/workflow-tests.yml)
[![Hopsworks load tests in kube](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/load-tests.yml/badge.svg)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/load-tests.yml)
[![Publish Main](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-publish-main.yml/badge.svg)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-publish-main.yml)
[![Publish Release](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-publish-release.yml/badge.svg)](https://github.com/logicalclocks/hopsworks-helm/actions/workflows/helm-publish-release.yml)

{{ template "chart.versionBadge" . }}{{ template "chart.typeBadge" . }}{{ template "chart.appVersionBadge" . }}

{{ template "chart.requirementsSection" . }}


## To deploy

### Setup your cluster

We have described several scenarios for creating kubernetes clusters [here](https://github.com/logicalclocks/hopsworks-helm/wiki/1-Install). [Below](#minikube) we add and example with minikube + docker driver

### Update dependencies
```sh
./helm_dependencies.sh
```

### Create namespace
```sh
kubectl create namespace kyverno
kubectl create namespace hopsworks
```

### Install the helm chart

```sh
helm install hopsworks-release . \
    --wait --timeout=600s --namespace hopsworks \
    --values values.dev.yaml
```


### Install in Minikube

<a id="minikube"></a>
Prepare minikube first
```sh
# To see the steps set -x
set -x
bash tests/launch_minikube.sh hopsworks
kubectl create namespace hopsworks
kubectl apply -f tests/local_path_provisioner.yaml
```
Then install hopsworks
```sh
helm install hopsworks . \
    --timeout=3600s --namespace hopsworks \
    --values values.minikube.yaml
```

**NOTICE** Using the minikube registry addon also for the hopsworks base images is not recommended. The reason is that the minikube addon is configured as a proxy through cache to save system images from minikube restart to minikube restart. With the proxy option enabled, we cannot push images. Thus, the presets will fail.

**NOTICE** Due the way we are configuring the docker registry the base images syncing takes a bit of time. Thus, even when the app is ready in under 20 mins, the images might take a bit longer to sync.

**NOTICE** the script will launch a metallb configuration. Feel free to set any IP range you want.

**NOTICE** that we create a folder named `local_minikube_path`, that folder is the PVC for the docker registry. Thus, if you want to keep the images between cluster hard restarts, you should not delete that folder.

**NOTICE** we enable the registry addon in minikube for: storing the system images between restarts of minikube and for the hopsworks images. Its data is also saved inside the `local_minikube_path` for caching the installation. By default we use the minikube registry addon as a managed registry for the base images. 

### Upgrade the helm chart

- **NOTE** If you plan to upgrade the helm-chart, consider the following issue in the `spark-operator`, https://github.com/kubeflow/spark-operator/issues/1554 . The spart-operator would recreate the serviceAccounts if they are not marked to do the contrary as `--set spark.spark-operator.serviceAccounts.sparkoperator.create=false`. Thus, the operator would stop working if this option is not false. Just add the previous value in case of upgrading the helm chart

## Deployment delete

> :warning: **The hopsworks helm subchart has a variable named `wipeWhenUninstall`**! The resources listed there, would be removed if they are related to the namespace in which the helm chart was installed in a post-install helm hook. Unset the variable if this behaviour is not needed. Otherwise, everything, including persistent volumes will be wiped out.


### Delete all Hopsworks Certificates before deleting the cluster

**Before** removing all resources from the cluster you should first delete all `HopsworksCerts`.
```sh
kubectl -n hopsworks delete hopsworkscerts --all
```

This will also revoke the issued certificates.

### Helm uninstall

```sh
 helm uninstall hopsworks-release -n hopsworks --timeout=900s
```

### Delete namespace to delete all changes

```sh
kubectl delete namespace hopsworks
```

## Nodes on privileges

We enfornce explicit `securityContext` in each one of the containers we create. Yet, we cannot control those that are external to our helm-chart. Notice also that, some of our containers (just a few) need to run privileged. For example, to patch the container engine to trust our generated certificates, the nodes need to be reached through a privilieged container.

Check warnings coming out of a deployment with a restricted namespace, e.g. `kubectl label --overwrite ns hopsworks pod-security.kubernetes.io/warn=restricted pod-security.kubernetes.io/warn-version=v1.24` and ping back the team if you are not sure it is expected.

## Gernerating schema.json and helm docs
We use [helm schema plugin](https://github.com/dadav/helm-schema?tab=readme-ov-file) to generate schema based on the values.yaml file. We also use [helm-docs](https://github.com/norwoodj/helm-docs) command to generate a README file with a table of all values in the subchart. 
Here is a very simple example of using helm schema and helm-docs together in a yaml file: 

```yaml
# -- (int) this is my new field 
myField: 1

# @schema
# additionalProperties: false
# properties:
#   test:
#       type: integer
# @schema
# -- (object) this my new object
myObject:
    test: 10 
```

Then you can run the following to generate both README and values.schema.json based on the above file
```bash
helm schema -r -p --no-dependencies
helm-docs
```

You should always run the following commands to update the schema whenever you add or update the values.yaml file of any of the charts. For example if updating a value in the hopsworks subchart, you should run the following:

```bash
cd charts/hopsworks
helm schema -r -p --no-dependencies
helm-docs
```

The command will update the values.schema.json based on the values.yaml and associated comment annotations, you should check the output of the file to make sure that is the intended output otherwise you can use any annotations as needed to satisfy your value definition, check [helm schema readme](https://github.com/dadav/helm-schema?tab=readme-ov-file) for more information about the different supported annotations. 

To generate the helm-docs on the parent helm chart to include all the subchart values as well run the following command 
```bash
helm-docs -u -g .
```

To generate the helm schema on the parent helm chart, you should run the following command
```bash
helm schema -r -p --no-dependencies -i .
```

You can also use the following script to validate that all the schemas and docs were updated correctly
```bash
./.github/test_lint.sh
```

{{ template "chart.valuesSection" . }}