diff --git a/docs/migrations.md b/docs/migrations.md index 4bc4b7f4..da538089 100644 --- a/docs/migrations.md +++ b/docs/migrations.md @@ -9,6 +9,13 @@ SPDX-License-Identifier: Apache-2.0 * [Disclaimer](#disclaimer) * [Automated migrations - Overview and mandatory upgrade path](#automated-migrations---overview-and-mandatory-upgrade-path) * [Manual checks/actions](#manual-checksactions) + * [v1.6.0+](#v160) + * [Pre-upgrade to v1.6.0+](#pre-upgrade-to-v160) + * [Helmfile new secret: `secrets.minio.openxchangeUser`](#helmfile-new-secret-secretsminioopenxchangeuser) + * [Helmfile new object storage: `objectstores.openxchange.*`](#helmfile-new-object-storage-objectstoresopenxchange) + * [OX App Suite fix-up: Using S3 as storage for non mail attachments (pre-upgrade)](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-pre-upgrade) + * [Post-upgrade to v1.6.0+](#post-upgrade-to-v160) + * [OX App Suite fix-up: Using S3 as storage for non mail attachments (post-upgrade)](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-post-upgrade) * [v1.4.0+](#v140) * [Pre-upgrade to v1.4.0+](#pre-upgrade-to-v140) * [Helmfile new feature: `functional.authentication.ssoFederation`](#helmfile-new-feature-functionalauthenticationssofederation) @@ -101,6 +108,93 @@ If you would like more details about the automated migrations, please read secti # Manual checks/actions +## v1.6.0+ + +### Pre-upgrade to v1.6.0+ + +#### Helmfile new secret: `secrets.minio.openxchangeUser` + +**Target group:** All existing deployments that have OX App Suite enabled and that use externally defined secrets in combination with openDesk provided MinIO object storage. + +For OX App Suite to access the object storage a new secret has been introduced. + +It is declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/secrets.yaml.gotmpl) by the key: `secrets.minio.openxchangeUser`. If you define your own secrets, please ensure that you provide a value for this secret as well, otherwise the aforementioned secret will be derived from the `MASTER_PASSWORD`. + +#### Helmfile new object storage: `objectstores.openxchange.*` + +**Target group:** All deployments that use an external object storage. + +For OX App Suite's newly introduced filestore you have to configure a new object storage (bucket). When you are using +an external object storage you did this already for all the entries in +[`objectstores.yaml.gotmpl`](../helmfile/environments/default/objectstores.yaml.gotmpl). Where we now introduced +`objectstores.openxchange` section that you also need to provide you external configuration for. + +#### OX App Suite fix-up: Using S3 as storage for non mail attachments (pre-upgrade) + +**Target group:** All existing deployments that have OX App Suite enabled. + +With openDesk 1.6.0 OX App Suite persists the attachments on contact, calendar or task objects in object storage. + +To enable the use of this new filestore backend existing deployments must execute the following steps. + +Preparation: +- Ensure your `kubeconfig` is pointing to the cluster that is running your deployment. +- Identify/create a e.g. local temporary directory that can keep the attachments while upgrading openDesk. +- Set some environment variables to prepare running the documented commands: + +```shell +export ATTACHMENT_TEMP_DIR= +export NAMESPACE= +``` + +1. Copy the existing attachments from all `open-xchange-core-mw-default-*` Pods to the identified directory, example for `open-xchange-core-mw-default-0`: +```shell +kubectl cp -n ${NAMESPACE} open-xchange-core-mw-default-0:/opt/open-xchange/ox-filestore ${ATTACHMENT_TEMP_DIR} +``` +2. Run the upgrade. +3. Continue with the [related post-upgrade steps](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-post-upgrade) + +### Post-upgrade to v1.6.0+ + +#### OX App Suite fix-up: Using S3 as storage for non mail attachments (post-upgrade) + +**Target group:** All existing deployments having OX App Suite enabled. + +Continued from the [related pre-upgrade section](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-pre-upgrade). + +1. Copy the attachments back from your temporary directory into `open-xchange-core-mw-default-0`. +```shell +kubectl cp -n ${NAMESPACE} ${ATTACHMENT_TEMP_DIR}/* open-xchange-core-mw-default-0:/opt/open-xchange/ox-filestore +``` +2. Ideally you verify the files have been copied as expected checking the target directory in the `open-xchange-core-mw-default-0` Pod. All the following commands are for execution within the aforementioned Pod. +3. Get the `id` of the new object storage based OX filestore, using the following command in the first line of the following block. In the shown example output the `id` for the new filestore would be `10` as the filestore can be identified by its path value `s3://ox-filestore-s3`, the `id` of the existing filestore would be `3` identified by the corresponding path `/opt/open-xchange/ox-filestore`: +```shell +/opt/open-xchange/sbin/listfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW +id path size reserved used max-entities cur-entities + 3 /opt/open-xchange/ox-filestore 100000 200 5 5000 1 +10 s3://ox-filestore-s3 100000 0 0 5000 0 +``` +4. Get the list of your OX contexts IDs (`cid` column in the output of the `listcontext` command), as the next step needs to be executed per OX context. Most installation will just have a single OX context (`1`). +```shell +/opt/open-xchange/sbin/listcontext -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW +cid fid fname enabled qmax qused name lmappings + 1 3 1_ctx_store true 5 1 1,context1 +``` +5. For each of your OX contexts IDs run the final filestore migration command and you will get output like this: `context 1 to filestore 10 scheduled as job 1`: +```shell +/opt/open-xchange/sbin/movecontextfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -f -c +``` +6. Depending on the size of your filestore, moving the contexts will take some time. You can check the status of a context's jobs with the command below. When the job status is `Done` you can also doublecheck that everything worked as expected by running the `listfilestore` command from step #3 and should see that the filestore is no longer used. +```shell +/opt/open-xchange/sbin/jobcontrol -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -c -l +ID Type of Job Status Further Information +1 movefilestore Done move context 1 to filestore 10 +``` +7. Finally you can unregister the old filestore: +```shell +/opt/open-xchange/sbin/unregisterfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -i +``` + ## v1.4.0+ ### Pre-upgrade to v1.4.0+ @@ -280,7 +374,7 @@ persistence: #### Helmfile new secret: `secrets.nubus.masterpassword` -A not yet templated secret was discovered in the Nubus deployment. It is now declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/theme.yaml.gotmpl) and can be defined using: `secrets.nubus.masterpassword`. If you define your own secrets, please be sure this new secret is set to the same value as the `MASTER_PASSWORD` environment variable used in your deployment. +A not yet templated secret was discovered in the Nubus deployment. It is now declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/secrets.yaml.gotmpl) and can be defined using: `secrets.nubus.masterpassword`. If you define your own secrets, please be sure this new secret is set to the same value as the `MASTER_PASSWORD` environment variable used in your deployment. ## v1.1.0+ diff --git a/helmfile/apps/open-xchange/values-openxchange-bootstrap.yaml.gotmpl b/helmfile/apps/open-xchange/values-openxchange-bootstrap.yaml.gotmpl index 39f9ff00..36051d71 100644 --- a/helmfile/apps/open-xchange/values-openxchange-bootstrap.yaml.gotmpl +++ b/helmfile/apps/open-xchange/values-openxchange-bootstrap.yaml.gotmpl @@ -42,4 +42,8 @@ serviceAccount: annotations: {{ .Values.annotations.openxchangeBootstrap.serviceAccount | toYaml | nindent 4 }} +filestore: + # identifier must match identifier in /opt/open-xchange/etc/filestore-s3.properties + identifier: "ox-filestore-s3" + size: {{ .Values.objectstores.openxchange.maxSize }} ... diff --git a/helmfile/apps/open-xchange/values-openxchange.yaml.gotmpl b/helmfile/apps/open-xchange/values-openxchange.yaml.gotmpl index 19549ef8..1599abb5 100644 --- a/helmfile/apps/open-xchange/values-openxchange.yaml.gotmpl +++ b/helmfile/apps/open-xchange/values-openxchange.yaml.gotmpl @@ -295,6 +295,8 @@ appsuite: open-xchange-oauth-provider: "enabled" # Needed to set com.openexchange.hostname open-xchange-hostname-config-cascade: "enabled" + # Enable s3 storage + open-xchange-filestore-s3: "enabled" properties: com.openexchange.hostname: {{ printf "%s.%s" .Values.global.hosts.openxchange .Values.global.domain }} com.openexchange.UIWebPath: "/appsuite/" @@ -479,6 +481,11 @@ appsuite: com.openexchange.antivirus.port: "1344" {{- end }} com.openexchange.antivirus.maxFileSize: "1024" + /opt/open-xchange/etc/filestore-s3.properties: + com.openexchange.filestore.s3.ox-filestore-s3.endpoint: {{ .Values.objectstores.openxchange.endpoint | default (printf "https://%s.%s" .Values.global.hosts.minioApi .Values.global.domain) | quote }} + com.openexchange.filestore.s3.ox-filestore-s3.bucketName: {{ .Values.objectstores.openxchange.bucket | quote }} + com.openexchange.filestore.s3.ox-filestore-s3.accessKey: {{ .Values.objectstores.openxchange.username | quote }} + com.openexchange.filestore.s3.ox-filestore-s3.secretKey: {{ .Values.objectstores.openxchange.secretKey | default .Values.secrets.minio.openxchangeUser | quote }} uiSettings: io.ox.nextcloud//server: "https://{{ .Values.global.hosts.intercomService }}.{{ .Values.global.domain }}/fs/" io.ox.public-sector//ics/url: "https://{{ .Values.global.hosts.intercomService }}.{{ .Values.global.domain }}/" diff --git a/helmfile/apps/services-external/values-minio.yaml.gotmpl b/helmfile/apps/services-external/values-minio.yaml.gotmpl index 05ef8622..c816d4aa 100644 --- a/helmfile/apps/services-external/values-minio.yaml.gotmpl +++ b/helmfile/apps/services-external/values-minio.yaml.gotmpl @@ -124,6 +124,9 @@ provisioning: - name: {{ .Values.objectstores.openproject.bucket | quote }} versioning: "Suspended" withLock: false + - name: {{ .Values.objectstores.openxchange.bucket | quote }} + versioning: "Suspended" + withLock: false - name: {{ .Values.objectstores.nubus.bucket | quote }} versioning: "Suspended" withLock: false @@ -183,6 +186,18 @@ provisioning: effect: "Allow" actions: - "s3:*" + - name: "openxchange-bucket-policy" + statements: + - resources: + - "arn:aws:s3:::openxchange" + effect: "Allow" + actions: + - "s3:*" + - resources: + - "arn:aws:s3:::openxchange/*" + effect: "Allow" + actions: + - "s3:*" - name: "ums-bucket-policy" statements: - resources: @@ -234,6 +249,12 @@ provisioning: policies: - "openproject-bucket-policy" setPolicies: true + - username: {{ .Values.objectstores.openxchange.username | quote }} + password: {{ .Values.secrets.minio.openxchangeUser | quote }} + disabled: false + policies: + - "openxchange-bucket-policy" + setPolicies: true - username: {{ .Values.objectstores.nubus.username | quote }} password: {{ .Values.secrets.minio.umsUser | quote }} disabled: false diff --git a/helmfile/environments/default/charts.yaml.gotmpl b/helmfile/environments/default/charts.yaml.gotmpl index a1ec59da..eefdd255 100644 --- a/helmfile/environments/default/charts.yaml.gotmpl +++ b/helmfile/environments/default/charts.yaml.gotmpl @@ -397,7 +397,7 @@ charts: registry: "registry.opencode.de" repository: "bmi/opendesk/components/platform-development/charts/opendesk-open-xchange-bootstrap" name: "opendesk-open-xchange-bootstrap" - version: "2.1.2" + version: "3.0.1" verify: true oxConnector: # providerCategory: "Supplier" diff --git a/helmfile/environments/default/objectstores.yaml.gotmpl b/helmfile/environments/default/objectstores.yaml.gotmpl index 0cd8a146..cfad9fa7 100644 --- a/helmfile/environments/default/objectstores.yaml.gotmpl +++ b/helmfile/environments/default/objectstores.yaml.gotmpl @@ -55,4 +55,14 @@ objectstores: username: "openproject_user" pathStyle: true useIamProfile: false + openxchange: + bucket: "openxchange" + endpoint: "" + # Size in MB + maxSize: 100000 + region: "" + secretKey: "" + storageClass: "STANDARD" + username: "openxchange_user" + useSSL: true ... diff --git a/helmfile/environments/default/secrets.yaml.gotmpl b/helmfile/environments/default/secrets.yaml.gotmpl index a71ef7d4..e60801b3 100644 --- a/helmfile/environments/default/secrets.yaml.gotmpl +++ b/helmfile/environments/default/secrets.yaml.gotmpl @@ -7,8 +7,8 @@ SPDX-License-Identifier: Apache-2.0 secrets: cassandra: rootPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "root_password" | sha1sum | quote }} - dovecotDictmapUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_dictmap_user" | sha1sum | quote }} dovecotACLUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_acl_user" | sha1sum | quote }} + dovecotDictmapUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_dictmap_user" | sha1sum | quote }} oxAppSuite: adminPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "ox_appsuite" "admin_password" | sha1sum | quote }} basicAuthPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "ox_appsuite" "basic_auth_password" | sha1sum | quote }} @@ -82,6 +82,7 @@ secrets: nextcloudUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "nextcloud_user" | sha1sum | quote) }} notesUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "notes_user" | sha1sum | quote) }} openprojectUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "openproject_user" | sha1sum | quote) }} + openxchangeUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "openxchange_user" | sha1sum | quote) }} umsUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "ums_user" | sha1sum | quote) }} keycloak: adminPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "keycloak" "adminPassword" | sha1sum | quote }}