mirror of
https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk.git
synced 2025-12-06 07:21:36 +01:00
feat(open-xchange): Store attachments for calendar, contact and task objects in object storage; review migrations.md for required upgrade steps
This commit is contained in:
Thorsten Roßner
@@ -9,6 +9,13 @@ SPDX-License-Identifier: Apache-2.0
|
||||
* [Disclaimer](#disclaimer)
|
||||
* [Automated migrations - Overview and mandatory upgrade path](#automated-migrations---overview-and-mandatory-upgrade-path)
|
||||
* [Manual checks/actions](#manual-checksactions)
|
||||
* [v1.6.0+](#v160)
|
||||
* [Pre-upgrade to v1.6.0+](#pre-upgrade-to-v160)
|
||||
* [Helmfile new secret: `secrets.minio.openxchangeUser`](#helmfile-new-secret-secretsminioopenxchangeuser)
|
||||
* [Helmfile new object storage: `objectstores.openxchange.*`](#helmfile-new-object-storage-objectstoresopenxchange)
|
||||
* [OX App Suite fix-up: Using S3 as storage for non mail attachments (pre-upgrade)](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-pre-upgrade)
|
||||
* [Post-upgrade to v1.6.0+](#post-upgrade-to-v160)
|
||||
* [OX App Suite fix-up: Using S3 as storage for non mail attachments (post-upgrade)](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-post-upgrade)
|
||||
* [v1.4.0+](#v140)
|
||||
* [Pre-upgrade to v1.4.0+](#pre-upgrade-to-v140)
|
||||
* [Helmfile new feature: `functional.authentication.ssoFederation`](#helmfile-new-feature-functionalauthenticationssofederation)
|
||||
@@ -101,6 +108,93 @@ If you would like more details about the automated migrations, please read secti
|
||||
|
||||
# Manual checks/actions
|
||||
|
||||
## v1.6.0+
|
||||
|
||||
### Pre-upgrade to v1.6.0+
|
||||
|
||||
#### Helmfile new secret: `secrets.minio.openxchangeUser`
|
||||
|
||||
**Target group:** All existing deployments that have OX App Suite enabled and that use externally defined secrets in combination with openDesk provided MinIO object storage.
|
||||
|
||||
For OX App Suite to access the object storage a new secret has been introduced.
|
||||
|
||||
It is declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/secrets.yaml.gotmpl) by the key: `secrets.minio.openxchangeUser`. If you define your own secrets, please ensure that you provide a value for this secret as well, otherwise the aforementioned secret will be derived from the `MASTER_PASSWORD`.
|
||||
|
||||
#### Helmfile new object storage: `objectstores.openxchange.*`
|
||||
|
||||
**Target group:** All deployments that use an external object storage.
|
||||
|
||||
For OX App Suite's newly introduced filestore you have to configure a new object storage (bucket). When you are using
|
||||
an external object storage you did this already for all the entries in
|
||||
[`objectstores.yaml.gotmpl`](../helmfile/environments/default/objectstores.yaml.gotmpl). Where we now introduced
|
||||
`objectstores.openxchange` section that you also need to provide you external configuration for.
|
||||
|
||||
#### OX App Suite fix-up: Using S3 as storage for non mail attachments (pre-upgrade)
|
||||
|
||||
**Target group:** All existing deployments that have OX App Suite enabled.
|
||||
|
||||
With openDesk 1.6.0 OX App Suite persists the attachments on contact, calendar or task objects in object storage.
|
||||
|
||||
To enable the use of this new filestore backend existing deployments must execute the following steps.
|
||||
|
||||
Preparation:
|
||||
- Ensure your `kubeconfig` is pointing to the cluster that is running your deployment.
|
||||
- Identify/create a e.g. local temporary directory that can keep the attachments while upgrading openDesk.
|
||||
- Set some environment variables to prepare running the documented commands:
|
||||
|
||||
```shell
|
||||
export ATTACHMENT_TEMP_DIR=<your_temporary_directory_for_the_attachments>
|
||||
export NAMESPACE=<your_namespace>
|
||||
```
|
||||
|
||||
1. Copy the existing attachments from all `open-xchange-core-mw-default-*` Pods to the identified directory, example for `open-xchange-core-mw-default-0`:
|
||||
```shell
|
||||
kubectl cp -n ${NAMESPACE} open-xchange-core-mw-default-0:/opt/open-xchange/ox-filestore ${ATTACHMENT_TEMP_DIR}
|
||||
```
|
||||
2. Run the upgrade.
|
||||
3. Continue with the [related post-upgrade steps](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-post-upgrade)
|
||||
|
||||
### Post-upgrade to v1.6.0+
|
||||
|
||||
#### OX App Suite fix-up: Using S3 as storage for non mail attachments (post-upgrade)
|
||||
|
||||
**Target group:** All existing deployments having OX App Suite enabled.
|
||||
|
||||
Continued from the [related pre-upgrade section](#ox-app-suite-fix-up-using-s3-as-storage-for-non-mail-attachments-pre-upgrade).
|
||||
|
||||
1. Copy the attachments back from your temporary directory into `open-xchange-core-mw-default-0`.
|
||||
```shell
|
||||
kubectl cp -n ${NAMESPACE} ${ATTACHMENT_TEMP_DIR}/* open-xchange-core-mw-default-0:/opt/open-xchange/ox-filestore
|
||||
```
|
||||
2. Ideally you verify the files have been copied as expected checking the target directory in the `open-xchange-core-mw-default-0` Pod. All the following commands are for execution within the aforementioned Pod.
|
||||
3. Get the `id` of the new object storage based OX filestore, using the following command in the first line of the following block. In the shown example output the `id` for the new filestore would be `10` as the filestore can be identified by its path value `s3://ox-filestore-s3`, the `id` of the existing filestore would be `3` identified by the corresponding path `/opt/open-xchange/ox-filestore`:
|
||||
```shell
|
||||
/opt/open-xchange/sbin/listfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW
|
||||
id path size reserved used max-entities cur-entities
|
||||
3 /opt/open-xchange/ox-filestore 100000 200 5 5000 1
|
||||
10 s3://ox-filestore-s3 100000 0 0 5000 0
|
||||
```
|
||||
4. Get the list of your OX contexts IDs (`cid` column in the output of the `listcontext` command), as the next step needs to be executed per OX context. Most installation will just have a single OX context (`1`).
|
||||
```shell
|
||||
/opt/open-xchange/sbin/listcontext -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW
|
||||
cid fid fname enabled qmax qused name lmappings
|
||||
1 3 1_ctx_store true 5 1 1,context1
|
||||
```
|
||||
5. For each of your OX contexts IDs run the final filestore migration command and you will get output like this: `context 1 to filestore 10 scheduled as job 1`:
|
||||
```shell
|
||||
/opt/open-xchange/sbin/movecontextfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -f <your_s3_filestore_id_from_step_3> -c <your_context_id_from_step_4>
|
||||
```
|
||||
6. Depending on the size of your filestore, moving the contexts will take some time. You can check the status of a context's jobs with the command below. When the job status is `Done` you can also doublecheck that everything worked as expected by running the `listfilestore` command from step #3 and should see that the filestore is no longer used.
|
||||
```shell
|
||||
/opt/open-xchange/sbin/jobcontrol -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -c <your_context_id_from_step_4> -l
|
||||
ID Type of Job Status Further Information
|
||||
1 movefilestore Done move context 1 to filestore 10
|
||||
```
|
||||
7. Finally you can unregister the old filestore:
|
||||
```shell
|
||||
/opt/open-xchange/sbin/unregisterfilestore -A $MASTER_ADMIN_USER -P $MASTER_ADMIN_PW -i <your_old_filestore_id_from_step_3>
|
||||
```
|
||||
|
||||
## v1.4.0+
|
||||
|
||||
### Pre-upgrade to v1.4.0+
|
||||
@@ -280,7 +374,7 @@ persistence:
|
||||
|
||||
#### Helmfile new secret: `secrets.nubus.masterpassword`
|
||||
|
||||
A not yet templated secret was discovered in the Nubus deployment. It is now declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/theme.yaml.gotmpl) and can be defined using: `secrets.nubus.masterpassword`. If you define your own secrets, please be sure this new secret is set to the same value as the `MASTER_PASSWORD` environment variable used in your deployment.
|
||||
A not yet templated secret was discovered in the Nubus deployment. It is now declared in [`secrets.yaml.gotmpl`](../helmfile/environments/default/secrets.yaml.gotmpl) and can be defined using: `secrets.nubus.masterpassword`. If you define your own secrets, please be sure this new secret is set to the same value as the `MASTER_PASSWORD` environment variable used in your deployment.
|
||||
|
||||
## v1.1.0+
|
||||
|
||||
|
||||
@@ -42,4 +42,8 @@ serviceAccount:
|
||||
annotations:
|
||||
{{ .Values.annotations.openxchangeBootstrap.serviceAccount | toYaml | nindent 4 }}
|
||||
|
||||
filestore:
|
||||
# identifier must match identifier in /opt/open-xchange/etc/filestore-s3.properties
|
||||
identifier: "ox-filestore-s3"
|
||||
size: {{ .Values.objectstores.openxchange.maxSize }}
|
||||
...
|
||||
|
||||
@@ -295,6 +295,8 @@ appsuite:
|
||||
open-xchange-oauth-provider: "enabled"
|
||||
# Needed to set com.openexchange.hostname
|
||||
open-xchange-hostname-config-cascade: "enabled"
|
||||
# Enable s3 storage
|
||||
open-xchange-filestore-s3: "enabled"
|
||||
properties:
|
||||
com.openexchange.hostname: {{ printf "%s.%s" .Values.global.hosts.openxchange .Values.global.domain }}
|
||||
com.openexchange.UIWebPath: "/appsuite/"
|
||||
@@ -479,6 +481,11 @@ appsuite:
|
||||
com.openexchange.antivirus.port: "1344"
|
||||
{{- end }}
|
||||
com.openexchange.antivirus.maxFileSize: "1024"
|
||||
/opt/open-xchange/etc/filestore-s3.properties:
|
||||
com.openexchange.filestore.s3.ox-filestore-s3.endpoint: {{ .Values.objectstores.openxchange.endpoint | default (printf "https://%s.%s" .Values.global.hosts.minioApi .Values.global.domain) | quote }}
|
||||
com.openexchange.filestore.s3.ox-filestore-s3.bucketName: {{ .Values.objectstores.openxchange.bucket | quote }}
|
||||
com.openexchange.filestore.s3.ox-filestore-s3.accessKey: {{ .Values.objectstores.openxchange.username | quote }}
|
||||
com.openexchange.filestore.s3.ox-filestore-s3.secretKey: {{ .Values.objectstores.openxchange.secretKey | default .Values.secrets.minio.openxchangeUser | quote }}
|
||||
uiSettings:
|
||||
io.ox.nextcloud//server: "https://{{ .Values.global.hosts.intercomService }}.{{ .Values.global.domain }}/fs/"
|
||||
io.ox.public-sector//ics/url: "https://{{ .Values.global.hosts.intercomService }}.{{ .Values.global.domain }}/"
|
||||
|
||||
@@ -124,6 +124,9 @@ provisioning:
|
||||
- name: {{ .Values.objectstores.openproject.bucket | quote }}
|
||||
versioning: "Suspended"
|
||||
withLock: false
|
||||
- name: {{ .Values.objectstores.openxchange.bucket | quote }}
|
||||
versioning: "Suspended"
|
||||
withLock: false
|
||||
- name: {{ .Values.objectstores.nubus.bucket | quote }}
|
||||
versioning: "Suspended"
|
||||
withLock: false
|
||||
@@ -183,6 +186,18 @@ provisioning:
|
||||
effect: "Allow"
|
||||
actions:
|
||||
- "s3:*"
|
||||
- name: "openxchange-bucket-policy"
|
||||
statements:
|
||||
- resources:
|
||||
- "arn:aws:s3:::openxchange"
|
||||
effect: "Allow"
|
||||
actions:
|
||||
- "s3:*"
|
||||
- resources:
|
||||
- "arn:aws:s3:::openxchange/*"
|
||||
effect: "Allow"
|
||||
actions:
|
||||
- "s3:*"
|
||||
- name: "ums-bucket-policy"
|
||||
statements:
|
||||
- resources:
|
||||
@@ -234,6 +249,12 @@ provisioning:
|
||||
policies:
|
||||
- "openproject-bucket-policy"
|
||||
setPolicies: true
|
||||
- username: {{ .Values.objectstores.openxchange.username | quote }}
|
||||
password: {{ .Values.secrets.minio.openxchangeUser | quote }}
|
||||
disabled: false
|
||||
policies:
|
||||
- "openxchange-bucket-policy"
|
||||
setPolicies: true
|
||||
- username: {{ .Values.objectstores.nubus.username | quote }}
|
||||
password: {{ .Values.secrets.minio.umsUser | quote }}
|
||||
disabled: false
|
||||
|
||||
@@ -397,7 +397,7 @@ charts:
|
||||
registry: "registry.opencode.de"
|
||||
repository: "bmi/opendesk/components/platform-development/charts/opendesk-open-xchange-bootstrap"
|
||||
name: "opendesk-open-xchange-bootstrap"
|
||||
version: "2.1.2"
|
||||
version: "3.0.1"
|
||||
verify: true
|
||||
oxConnector:
|
||||
# providerCategory: "Supplier"
|
||||
|
||||
@@ -55,4 +55,14 @@ objectstores:
|
||||
username: "openproject_user"
|
||||
pathStyle: true
|
||||
useIamProfile: false
|
||||
openxchange:
|
||||
bucket: "openxchange"
|
||||
endpoint: ""
|
||||
# Size in MB
|
||||
maxSize: 100000
|
||||
region: ""
|
||||
secretKey: ""
|
||||
storageClass: "STANDARD"
|
||||
username: "openxchange_user"
|
||||
useSSL: true
|
||||
...
|
||||
|
||||
@@ -7,8 +7,8 @@ SPDX-License-Identifier: Apache-2.0
|
||||
secrets:
|
||||
cassandra:
|
||||
rootPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "root_password" | sha1sum | quote }}
|
||||
dovecotDictmapUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_dictmap_user" | sha1sum | quote }}
|
||||
dovecotACLUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_acl_user" | sha1sum | quote }}
|
||||
dovecotDictmapUser: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "cassandra" "dovecot_dictmap_user" | sha1sum | quote }}
|
||||
oxAppSuite:
|
||||
adminPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "ox_appsuite" "admin_password" | sha1sum | quote }}
|
||||
basicAuthPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "ox_appsuite" "basic_auth_password" | sha1sum | quote }}
|
||||
@@ -82,6 +82,7 @@ secrets:
|
||||
nextcloudUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "nextcloud_user" | sha1sum | quote) }}
|
||||
notesUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "notes_user" | sha1sum | quote) }}
|
||||
openprojectUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "openproject_user" | sha1sum | quote) }}
|
||||
openxchangeUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "openxchange_user" | sha1sum | quote) }}
|
||||
umsUser: {{ (derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "minio" "ums_user" | sha1sum | quote) }}
|
||||
keycloak:
|
||||
adminPassword: {{ derivePassword 1 "long" (env "MASTER_PASSWORD" | default "sovereign-workplace") "keycloak" "adminPassword" | sha1sum | quote }}
|
||||
|
||||
Reference in New Issue
Block a user