feat(nubus): Update chart to version 0.37.0-pre-jtorres-ox-extensions-to-data-loader

docs/enhanced-configuration/idp-federation.md

@@ -44,9 +44,9 @@ We will provide additional documents regarding user provisioning in the future,
   - UDM REST API:
     - Build a provisioning solution by yourself using the [UDM REST API](https://docs.software-univention.de/developer-reference/5.0/en/udm/rest-api.html).
     - The API gives you full control over the contents of the IAM in order to create, update or delete users and groups.
-  - Directory Connector:
+  - Nubus Directory Importer:
     - It is based on a Python one-way directory synchronization for users and groups.
-    - We will provide more details on this approach soon one the tool is made publicly available.
+    - Please find more details in the [upstream product's documentation](https://docs.software-univention.de/nubus-kubernetes-operation/latest/en/howto-connect-external-iam.html).
 - Ad-hoc provisioning (AHP)
   - This feature is currently not available in the openDesk Keycloak, but there are plans by the Supplier Univention to make it available.
   - Ad-hoc provisioning creates an user account on the fly during a users first login.

docs/enhanced-configuration/separate-mail-matrix-domain.md

@@ -9,6 +9,10 @@ SPDX-License-Identifier: Apache-2.0
 * [Example configuration](#example-configuration)
   * [Mail domain](#mail-domain)
   * [Matrix domain](#matrix-domain)
+    * [DNS](#dns)
+    * [Webserver](#webserver)
+      * [Content Security Policy](#content-security-policy)
+      * [.well-known](#well-known)
 
 # Use case
 
@@ -59,7 +63,9 @@ or via environment variable
 export MATRIX_DOMAIN=my_organization.tld
 ```
 
-This setup requires also a different DNS setup:
+### DNS
+
+The following changes apply to the standard DNS:
 
 | Record name                      | Type | Value                                  | Comment                                                                            |
 | -------------------------------- | ---- | -------------------------------------- | ---------------------------------------------------------------------------------- |
@@ -67,6 +73,14 @@ This setup requires also a different DNS setup:
 
 *Note:* `matrix.opendesk.domain.tld` in the "Value" column can also be the IP address where synapse TLS port is listening to.
 
+### Webserver
+
+#### Content Security Policy
+
+The webserver of `my_organization.tld` should add `*.opendesk.domain.tld` to it's CSP header.
+
+#### .well-known
+
 If you want to use other Matrix clients,
 e.g., Element Messenger for [iOS](https://apps.apple.com/de/app/element-messenger/id1083446067)
 or [Android](https://play.google.com/store/apps/details?id=im.vector.app),
@@ -82,4 +96,4 @@ you need to create a JSON file with the following contents that is served from
 ```
 
 This ensures clients know where to find the Matrix protocol endpoint when users specify `my_organization.tld`
-as their homeserver.
+as their homeserver.

docs/getting-started.md

@@ -52,7 +52,7 @@ files.
 > All configuration options and their default values can be found in files at `helmfile/environments/default/`
 
 For the following guide, we will use `dev` as environment, where variables can be set in
-`helmfile/environments/dev/values.yaml`.
+`helmfile/environments/dev/values.yaml.gotmpl`.
 
 ## DNS
 
@@ -115,13 +115,13 @@ All available apps and their default value can be found in `helmfile/environment
 | Memcached            | `memcached.enabled`         | `true`  | Cache Database                 |
 | MinIO                | `minio.enabled`             | `true`  | Object Storage                 |
 | Nextcloud            | `nextcloud.enabled`         | `true`  | File share                     |
+| Nubus                | `nubus.enabled`             | `true`  | Identity Management & Portal   |
 | OpenProject          | `openproject.enabled`       | `true`  | Project management             |
 | OX Appsuite          | `oxAppsuite.enabled`        | `true`  | Groupware                      |
 | Provisioning         | `oxConnector.enabled`       | `true`  | Backend provisioning           |
 | Postfix              | `postfix.enabled`           | `true`  | MTA                            |
 | PostgreSQL           | `postgresql.enabled`        | `true`  | Database                       |
 | Redis                | `redis.enabled`             | `true`  | Cache Database                 |
-| Nubus                | `nubus.enabled`             | `true`  | Identity Management & Portal   |
 | XWiki                | `xwiki.enabled`             | `true`  | Knowledge management           |
 
 Exemplary, Jitsi can be disabled like:
@@ -199,18 +199,27 @@ cluster:
       - "127.0.0.0/8"
 ```
 
+If your load balancer / reverse proxy IPs are not already covered by the above `cidr` you need to
+explicitly configure the related IPs or IP ranges:
+
+```yaml
+cluster:
+  networking:
+    incomingCIDR:
+      - "172.16.0.0/12"
+```
+
 ### Ingress
 
-By default, the `ingressClassName` is empty to choose your default ingress controller, you may want to customize it by
+By default, the `ingressClassName` is empty to choose your default ingress controller. You may want to customize it by
-setting:
+setting the following attribute to the name of the currently only supported ingress controller `ingress-nginx` (see
+[requirements.md](./requirements.md)) for reference) within your deployment if that is not the clusters default ingress.
 
 ```yaml
 ingress:
-  ingressClassName: "cilium"
+  ingressClassName: "name-of-my-nginx-ingress"
 ```
 
-**Note:** Please check the [requirements.md](./requirements.md) for the supported Ingress controllers.
-
 ### Container runtime
 
 Some apps require specific configuration for the container runtime. You can set your container runtime like `cri-o`,

docs/migrations.md

@@ -6,14 +6,17 @@ SPDX-License-Identifier: Apache-2.0
 <h1>Upgrade migrations</h1>
 
 * [Disclaimer](#disclaimer)
-* [From v0.9.0](#from-v090)
+* [Releases upgrades](#releases-upgrades)
-  * [Automated migrations](#automated-migrations)
+  * [From v0.9.0](#from-v090)
-    * [Updated IAM component Nubus](#updated-iam-component-nubus)
+    * [Automated migrations](#automated-migrations)
-      * [Manual cleanup](#manual-cleanup)
+      * [Updated IAM component Nubus](#updated-iam-component-nubus)
-* [From v0.8.1](#from-v081)
+        * [Manual cleanup](#manual-cleanup)
-  * [Updated `cluster.networking.cidr`](#updated-clusternetworkingcidr)
+  * [From v0.8.1](#from-v081)
-  * [Updated customizable template attributes](#updated-customizable-template-attributes)
+    * [Updated `cluster.networking.cidr`](#updated-clusternetworkingcidr)
-  * [`migrations` S3 bucket](#migrations-s3-bucket)
+    * [Updated customizable template attributes](#updated-customizable-template-attributes)
+    * [`migrations` S3 bucket](#migrations-s3-bucket)
+* [Related components and artefacts](#related-components-and-artefacts)
+  * [Development](#development)
 
 # Disclaimer
 
@@ -24,14 +27,16 @@ Though we try to ease the pain when it comes to 0.x upgrades. That is what this
 Limitations:
 - We assume that the PV reclaim policy is set to `delete`, so expect that PVs get deleted as soon as the related PVC was deleted and will cover an explicit delete for PVs.
 
-# From v0.9.0
+# Releases upgrades
 
-## Automated migrations
+## From v0.9.0
 
-### Updated IAM component Nubus
+### Automated migrations
 
-openDesk is integrating the latest [Nubus](https://www.univention.de/produkte/nubus/) development from Univention. The new redundant and scalable LDAP requires migration activities. These have been automated to avoid manual interaction. The `run_2` of the openDesk
+#### Updated IAM component Nubus
-upgrade migrations executes the following steps
+
+openDesk is integrating the latest [Nubus](https://www.univention.de/produkte/nubus/) development from Univention. The now redundant and scalable LDAP requires migration activities. These have been automated to avoid manual interaction. The `run_2` of the openDesk
+upgrade migrations executes the following steps:
 
 - Stage PRE:
   - Delete service `ums-keycloak`, as it will be recreated headless.
@@ -40,7 +45,7 @@ upgrade migrations executes the following steps
 - Stage POST:
   - Restart Keycloak.
 
-#### Manual cleanup
+##### Manual cleanup
 
 Currently we do not execute possible cleanup steps as part of the migrations POST stage. So you might want to remove the no longer used PVCs after successful upgrade:
 ```
@@ -49,14 +54,14 @@ kubectl -n ${NAMESPACE} delete pvc shared-data-ums-ldap-server-0
 kubectl -n ${NAMESPACE} delete pvc shared-run-ums-ldap-server-0
 ```
 
-# From v0.8.1
+## From v0.8.1
 
-## Updated `cluster.networking.cidr`
+### Updated `cluster.networking.cidr`
 
 - Action: `cluster.networking.cidr` is now an array (was a string until 0.8.1), please update your setup accordingly if you explicitly set this value.
 - Reference:[cluster.yaml](https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/main/helmfile/environments/default/cluster.yaml)
 
-## Updated customizable template attributes
+### Updated customizable template attributes
 
 - Action: Please ensure you update you custom deployment values according with the updated default value structure.
 - References:
@@ -65,7 +70,28 @@ kubectl -n ${NAMESPACE} delete pvc shared-run-ums-ldap-server-0
   - `monitoring.` prefix for `prometheus.*` and `graphana.*`, see [monitoring.yaml](https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/main/helmfile/environments/default/monitoring.yaml).
   - `smtp.` prefix for `localpartNoReply`, see [smtp.yaml](https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/main/helmfile/environments/default/smtp.yaml).
 
-## `migrations` S3 bucket
+### `migrations` S3 bucket
 
 - Action: For self managed/external S3/object storages, please ensure you add a bucket `migrations` to your S3.
 - Reference: `objectstores.migrations` in [objectstores.yaml](https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/main/helmfile/environments/default/objectstores.yaml)
+
+# Related components and artefacts
+
+openDesk comes with two upgrade steps as part of the deployment, they can be found in the folder [/helmfile/apps](../helmfile/apps/) as all other components:
+
+- `migrations-pre`: Is the very first app that gets deployed.
+- `migrations-post`: Is the last app that gets deployed.
+
+Both migrations have to be deployed exclusively at their first/last position and not in parallel with other components.
+
+The status of the upgrade migrations is tracked in the ConfigMap `migrations-status`, more details can be found in the [README.md of the related container image](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/images/opendesk-migrations/README.md).
+
+## Development
+
+When a new upgrade migration is required, ensure to address the following list:
+
+- Update the generated release version file [`global.generated.yaml`](../helmfile/environments/default/global.generated.yaml) at least on the patch level to test the upgrade in your feature branch as well as trigger it in the `develop` branch after the feature branch was merged. The set value gets overwritten during the release process with the release's actual version number.
+- You have to implement the migration logic as a runner script in the [`opendesk-migrations`](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/images/opendesk-migrations) image. Please find more instructions in the linked repository.
+- You most likely have to update the [`opendesk-migrations` Helm chart](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/charts/opendesk-migrations) within the `rules` section of the [`role.yaml`](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/charts/opendesk-migrations/-/blob/main/charts/opendesk-migrations/templates/role.yaml) to provide the permissions required for the execution of your migration's logic.
+- You have to set the runner's ID you want to execute in the [migrations.yaml.gotmpl](../helmfile/shared/migrations.yaml.gotmpl). See also the `migrations.*` section of [the Helm chart's README.md](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/charts/opendesk-migrations/-/blob/main/charts/opendesk-migrations/README.md).
+- Update the [`charts.yaml`](../helmfile/environments/default/charts.yaml) and [`images.yaml`](../helmfile/environments/default/images.yaml) to reflect the newer releases of the `opendesk-migrations` Helm chart and container image.

helmfile/apps/nextcloud/values-nextcloud-mgmt.yaml.gotmpl

@@ -37,7 +37,7 @@ configuration:
     port: {{ .Values.cache.nextcloud.port | quote }}
   collabora:
     # internalWopiUrl: ""
-    wopiAllowlist: {{ join " " .Values.cluster.networking.cidr | quote }}
+    wopiAllowlist: {{ join ", " ( concat .Values.cluster.networking.cidr .Values.cluster.networking.incomingCIDR ) | quote }}
   database:
     host: {{ .Values.databases.nextcloud.host | quote }}
     port: {{ .Values.databases.nextcloud.port | quote }}

helmfile/apps/nubus/values-opendesk-keycloak-bootstrap.yaml.gotmpl

@@ -42,8 +42,7 @@ config:
     # We use client specific scopes as we bind them to Keycloak role membership which itself is linked
     # to LDAP group membership to ensure a user cannot access an application without the required
     # group membership.
-    # ToDo:
+    # ToDo: Ensure all applications verify the token's signature to ensure it is not tampered.
-    # - Jitsi does currently not care if it gets scopes/claims as long as the user is authenticated.
     clientScopes:
       - name: "read_contacts"
         protocol: "openid-connect"

helmfile/environments/default/charts.yaml

@@ -120,7 +120,7 @@ charts:
     registry: "registry.opencode.de"
     repository: "bmi/opendesk/components/platform-development/charts/opendesk-jitsi"
     name: "opendesk-jitsi"
-    version: "1.7.9"
+    version: "1.9.2"
     verify: true
   mariadb:
     # providerCategory: "Platform"
@@ -254,7 +254,7 @@ charts:
     registry: "artifacts.software-univention.de"
     repository: "nubus-dev/charts"
     name: "nubus"
-    version: "0.38.0-pre-provisioning-consumer-integration"
+    version: "0.37.0-pre-jtorres-ox-extensions-to-data-loader"
     verify: true
   opendeskKeycloakBootstrap:
     # providerCategory: "Platform"

helmfile/environments/default/cluster.yaml

@@ -18,11 +18,14 @@ cluster:
     # Kubernetes cluster network CIDRs.
     cidr:
       - "10.0.0.0/8"
+    # IP addresses or IP ranges of the reverse proxy / load balancer to restrict the requesting source
+    # for defined services.
+    incomingCIDR: []
     # Ingress-gateway IP - only relevant for "NodePort" cluster services.
     # When ingress and egress gateway use different ips, which results that pods can't self-discover their incoming ip,
     # you need to provide the public (load-balanced) ingress gateways ip address.
     ingressGatewayIP: ""
-    # LoadBalancer status fiel - only relevant for "LoadBalancer" cluster services.
+    # LoadBalancer status field - only relevant for "LoadBalancer" cluster services.
     # The IP/DNS of your load-balancer will be fetched for some components from 'status' map of services.
     # Most providers use '.status.loadBalancer.ingress[0].ip' to store public ip. You can modify the chosen field here.
     loadBalancerStatusField: "ip"

helmfile/environments/default/images.yaml

@@ -113,7 +113,7 @@ images:
     # upstreamMirrorStartFrom: ["2023", "12", "14"]
     registry: "registry.opencode.de"
     repository: "bmi/opendesk/components/supplier/nordeck/images-mirror/jitsi-keycloak-adapter"
-    tag: "v20240314@sha256:8abe8209a59c2d646fa3e7136f6a6ea15f3f692106bba92c0e777f8d383edc12"
+    tag: "v20240808@sha256:73db5ae9bb617d076b6a877500dbeece38fd15f8ae933ffadbd72d9aa82ad8da"
   jitsiPatchJVB:
     # providerCategory: "Community"
     # providerResponsible: "Nordeck"
@@ -413,11 +413,9 @@ images:
     # upstreamRepository: "nubus/images/notifications-api"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "9", "4"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/notifications-api"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/notifications-api"
-    registry: "artifacts.software-univention.de"
+    tag: "0.27.0@sha256:d99173199f20c701b29b8a3c1a46465085a873b37f413882e7d2e106e258c35a"
-    repository: "nubus/images/notifications-api"
-    tag: "0.31.1@sha256:cc51d3d7c25f2bef2bda4f3baad0c29dc428b798be78697d58739eb6a79636d2"
   nubusOpenPolicyAgent:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -467,11 +465,9 @@ images:
     # upstreamRepository: "nubus/images/portal-frontend"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "9", "4"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/portal-frontend"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/portal-frontend"
-    registry: "artifacts.software-univention.de"
+    tag: "0.29.0@sha256:3af3d5d24f690557b4a644d5720113dca0c802465b0e43466b49db27acd37939"
-    repository: "nubus/images/portal-frontend"
-    tag: "0.31.1@sha256:499266cb7c645775dd2d0c9f386498114ab0dcda67b014db0e86735d757b4187"
   nubusPortalListener:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -489,11 +485,9 @@ images:
     # upstreamRepository: "nubus/images/portal-server"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "9", "4"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/portal-server"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/portal-server"
-    registry: "artifacts.software-univention.de"
+    tag: "0.27.0@sha256:e1ad659feb4a1948d07e6e7d99b94b6bdbd4525d96f4cf9a010b75189f0082fc"
-    repository: "nubus/images/portal-server"
-    tag: "0.31.1@sha256:381e0cbbcac0e9bba6d3c1c7fe7757d18c09e7afaab912c02f38bdfa481723cd"
   nubusProvisioningDispatcher:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -511,11 +505,9 @@ images:
     # upstreamRepository: "nubus/images/provisioning-events-and-consumer-api"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "14", "0"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-events-and-consumer-api"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-events-and-consumer-api"
-    registry: "artifacts.software-univention.de"
+    tag: "0.28.3@sha256:5b0a2c52d715fde613ecfedb3a3f5e47b9eb73cdcf4c373a9cc58248a919f2bf"
-    repository: "nubus-dev/images/provisioning-events-and-consumer-api"
-    tag: "0.35.0@sha256:044676b9ca7c3aac24193e26fd93ab46601a1d7fada090acb6fbe3512e5a5c6c"
   nubusProvisioningPrefill:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -523,11 +515,9 @@ images:
     # upstreamRepository: "nubus/images/provisioning-prefill"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "14", "0"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-prefill"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-prefill"
-    registry: "artifacts.software-univention.de"
+    tag: "0.28.3@sha256:a98bce46144a6ff943b0432b66277393b7b476b8969b221b9069c708d3380f5d"
-    repository: "nubus-dev/images/provisioning-prefill"
-    tag: "0.35.0@sha256:23d7858847154cdaf49164c8e3a6f92f2fa757b40476be2fdf01bfbfa4387505"
   nubusProvisioningUdmListener:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -535,11 +525,9 @@ images:
     # upstreamRepository: "nubus/images/provisioning-udm-listener"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "14", "0"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-udm-listener"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-udm-listener"
-    registry: "artifacts.software-univention.de"
+    tag: "0.28.3@sha256:b9c452e55e6716f93309bef0af7d401e218cd1e6ea9ad3d2819fb10dd631aecd"
-    repository: "nubus/images/provisioning-udm-listener"
-    tag: "0.34.0@sha256:5f1177f58a189de935b73e905b9dff69b34f508a570244c75d5358cdb1bc2b27"
   nubusProvisioningUdmTransformer:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"
@@ -547,11 +535,9 @@ images:
     # upstreamRepository: "nubus/images/provisioning-udm-transformer"
     # upstreamMirrorTagFilterRegEx: '^(\d+)\.(\d+)\.(\d+)$'
     # upstreamMirrorStartFrom: ["0", "14", "0"]
-    # registry: "registry.opencode.de"
+    registry: "registry.opencode.de"
-    # repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-udm-transformer"
+    repository: "bmi/opendesk/components/supplier/univention/images-mirror/provisioning-udm-transformer"
-    registry: "artifacts.software-univention.de"
+    tag: "0.29.0@sha256:68e27eb9560d2729e9065da3573f28073c5e53fedabac4d19562c4b8c6c1d1f3"
-    repository: "nubus-dev/images/provisioning-udm-transformer"
-    tag: "0.35.0@sha256:1495493affbbf19fdbb77e7ba02522a313dd52dbfb39ce148d48b84874dcd915"
   nubusSelfserviceInvitation:
     # providerCategory: "Supplier"
     # providerResponsible: "Univention"