Merge branch 'trossner/mail_doc' into 'develop'

feat(helmfile): Add templating of `smtp.spamMilterHost`; it is strongly...

See merge request bmi/opendesk/deployment/opendesk!1120

docs/architecture.md

@@ -26,6 +26,9 @@ SPDX-License-Identifier: Apache-2.0
   * [Filepicker](#filepicker)
   * [Newsfeed](#newsfeed)
   * [(OpenProject) File store](#openproject-file-store)
+* [Mail setup](#mail-setup)
+  * [Overview](#overview-1)
+  * [The Postfixes](#the-postfixes)
 * [Applications vs. services](#applications-vs-services)
   * [Collabora (weboffice)](#collabora-weboffice)
   * [CryptPad Online (diagrams)](#cryptpad-online-diagrams)
@@ -348,6 +351,85 @@ The file store must still be enabled per project in OpenProject's project admin
 - [OpenProject's documentation on Nextcloud integration](https://www.openproject.org/docs/system-admin-guide/integrations/nextcloud/)
 - [OpenProject Integration Nextcloud app](https://apps.nextcloud.com/apps/integration_openproject)
 
+# Mail setup
+
+The mail setup depicted in the diagram below shows the design to support multiple application workloads inside openDesk while interoperating with external mail infrastructures and optional mail clients like Thunderbird.
+
+The system is intentionally modular: different applications (Nextcloud, OpenProject, XWiki, Synapse, Notes, etc.) may need to send emails even when no full groupware stack is deployed. In that case the following components are also not being deployed:
+
+* `Dovecot`
+* `Postfix-OX`
+
+Even without these components, the platform remains operational for outbound email because the (Base) Postfix instance provides a simple SMTP submission service using static SASL credentials. This allows all applications in *openDesk* to continue sending system notifications and user emails.
+
+## Overview
+
+```mermaid
+flowchart-elk
+
+extClient[optional Mail Clients]
+extRelay[Mailrelay/MXe]
+extMTA[MTAs]
+
+subgraph extSvc[K8s External Servies]
+    extSvcDC((dovecot-external))
+    extSvcPF((postfix-ox-external))
+end
+
+subgraph openDesk
+    subgraph Apps
+        AppsOther[Nubus<br>Nextcloud<br>OpenProject<br>Synapse<br>XWiki<br>Notes]
+        AppsOXAS[OX App Suite]
+    end
+    subgraph Postfix
+        PostfixBase[#40;Base#41; Postfix]
+        PostfixOX[Postfix-OX]
+    end
+    Dovecot[Dovecot<br>authenticates using<br>SASL using LDAP & OAuth]
+    Dovecot -->|Sieve mails<br>without no auth| PostfixBase
+    PostfixOX -->|auth|Dovecot
+end
+
+Postfix -->|lmtps| Dovecot
+Postfix -->|smtp| extRelay
+
+extSvcDC --> Dovecot
+extSvcPF --> PostfixOX
+
+AppsOther -->|auth:<br>static creds.| PostfixBase
+AppsOXAS --> Dovecot
+AppsOXAS -->|auth:<br>OAuth| PostfixOX
+
+extClient --> extSvcDC
+extMTA -->|WARNING: SPF and DKIM validation required| extSvcPF
+extClient -->|auth:<br>LDAP| extSvcPF
+
+classDef postfix fill:#85extMTA9C;
+class PostfixBase postfix;
+classDef postfix-ox fill:#F3E5Dovecot;
+class PostfixOX,extSvcPF postfix-ox;
+classDef dovecot fill:#BECBD6;
+class Dovecot,extSvcDC dovecot;
+```
+
+## The Postfixes
+
+* Common for both Postfix
+  * Deliver internal mails to Dovecot using lmtps
+  * Deliver non-internal mails directly to a configured mail relay or to the recipients MX
+
+* (Base) Postfix specific
+   * SMTP submission from applications using static credentials
+   * SMTP submission without authentication for Dovecot generated mails by Sieve filters, e.g. out-of-office replys, as Dovecot does not support authentication in this flow
+   * Available even if OX App Suite is not installed
+
+* Postfix-OX specific
+   * External mails are relayed for internal maildomains unauthenticated
+   * Requires Dovecot for SASL authentication on
+     * mails sent from OX App Suite's Web UI using OAuth
+     * mails sent from mail clients using LDAP Auth
+   * Used exclusively when OX App Suite is deployed
+
 # Applications vs. services
 
 openDesk consists of a variety of open-source projects, please find an overview below:

docs/getting-started.md

@@ -65,7 +65,7 @@ For your convenience, we recommend creating a `*.domain.tld` A-Record for your c
 | Record name                   | Type | Value                                              | Additional information                                            |
 |-------------------------------|------|----------------------------------------------------|-------------------------------------------------------------------|
 | *.domain.tld                  | A    | IPv4 address of your Ingress Controller            |                                                                   |
-| *.domain.tld                  | AAAA | IPv6 address of your Ingress Controller            |                                                                   |
+| *.domain.tld                  | AAAA | IPv6 address of your Ingress Controller            | Optional                                                          |
 | mail.domain.tld               | A    | IPv4 address of your postfix NodePort/LoadBalancer | Optional, mail should directly be delivered to openDesk's Postfix |
 | mail.domain.tld               | AAAA | IPv6 address of your postfix NodePort/LoadBalancer | Optional, mail should directly be delivered to openDesk's Postfix |
 | domain.tld                    | MX   | `10 mail.domain.tld`                               |                                                                   |
@@ -341,6 +341,16 @@ smtp:
   password: "secret"
 ```
 
+It is strongly recommended to configure a milter host for spam filtering (e.g. Rspamd) to get SPF and DKIM
+validation for incoming mails in place. Otherwise external senders could spoof internal sender addresses.
+
+```yaml
+smtp:
+  spamMilter:
+    host: "rspamd.domain.internal"
+    port: "11332"
+```
+
 ### TURN configuration
 
 Some components (Jitsi, Element) use a TURN server for direct communication. You can configure your own TURN server with

docs/monitoring.md

@@ -23,8 +23,7 @@ openDesk includes integration with Prometheus-based monitoring.
 
 Together with [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), you can easily leverage the full potential of the open-source cloud-native observability stack.
 
-Before enabling the following options, you need to install the respective custom resource definitions (CRDs) from the kube-prometheus-stack
-repository or Prometheus operator.
+Before enabling the following options, you need to install the respective custom resource definitions (CRDs) from the kube-prometheus-stack repository which should at least include the Prometheus Operator.
 
 # Defaults
 
@@ -33,14 +32,16 @@ All configurable options and their defaults can be found in
 
 # Metrics
 
-To deploy `podMonitor` and `serviceMonitor` custom resources, enable it by:
+To deploy `podMonitor` and `serviceMonitor` custom resources, enable them by:
 
 ```yaml
-prometheus:
-  serviceMonitors:
-    enabled: true
-  podMonitors:
-    enabled: true
+monitoring:
+  prometheus:
+    serviceMonitors:
+      enabled: true
+    podMonitors:
+      enabled: true
+```
 ```
 
 # Alerts
@@ -51,19 +52,23 @@ Some of these are created by our partners while others are defined in [opendesk-
 All alert rules are deployed as [PrometheusRule](https://prometheus-operator.dev/docs/api-reference/api/#monitoring.coreos.com/v1.PrometheusRule) and can be enabled like this:
 
 ```yaml
-prometheus:
-  prometheusRules:
-    enabled: true
+monitoring:
+  prometheus:
+    prometheusRules:
+      enabled: true
 ```
 
 # Dashboards for Grafana
 
-To deploy optional Grafana dashboards with ConfigMaps, enable the functionality with:
+If your Grafana instance is deployed via kube-prometheus-stack, or you have deployed the [Sidecar for datasources](https://github.com/grafana/helm-charts/blob/main/charts/grafana/README.md#sidecar-for-datasources), openDesk can make dashboards available via ConfigMap resources. 
+
+Enable the functionality with the following snippet:
 
 ```yaml
-grafana:
-  dashboards:
-    enabled: true
+monitoring:
+  grafana:
+    dashboards:
+      enabled: true
 ```
 
 Please find further details in the [related Helm chart](https://gitlab.opencode.de/bmi/opendesk/components/platform-development/charts/opendesk-dashboards).

helmfile/apps/open-xchange/values-postfix.yaml.gotmpl

@@ -47,12 +47,29 @@ postfix:
   inetProtocols: "ipv4"
   messageSizeLimit: {{ mul .Values.functional.groupware.mail.maxSize 1024 1024 | int | printf "%d" | quote }}
   milterDefaultAction: "tempfail"
-  {{- if .Values.apps.dkimpy.enabled }}
-  dkimpyHost: "opendesk-dkimpy-milter.{{ .Release.Namespace }}.svc.{{.Values.cluster.networking.domain }}:8892"
-  {{- end }}
+  smtpdMilters:
+    {{- if .Values.apps.dkimpy.enabled }}
+    - host: "opendesk-dkimpy-milter.{{ .Release.Namespace }}.svc.{{.Values.cluster.networking.domain }}"
+      port: 8892
+    {{- end }}
+    {{- if .Values.smtp.spamMilter.host }}
+    - host: {{ .Values.smtp.spamMilter.host | quote }}
+      port: {{ .Values.smtp.spamMilter.port }}
+    {{- end }}
+    {{- if .Values.antivirus.milter.host }}
+    - host: {{ .Values.antivirus.milter.host | quote }}
+      port: {{ .Values.antivirus.milter.port }}
+    {{- else }}
+    {{- if .Values.apps.clamavDistributed.enabled }}
+    - host: "clamav-milter"
+      port:7357
+    {{- else if .Values.apps.clamavSimple.enabled }}
+    - host: "clamav-simple"
+      port: 7357
+    {{- end }}
+    {{- end }}
   minTLSVersion: "TLSv1.2"
   smtpdTLSMandatoryCiphers: "medium"
-  rspamdHost: ""
   {{- if .Values.smtp.host }}
   relayHost:
     enabled: true
@@ -100,15 +117,6 @@ postfix:
       # -- return the following attribute from all found leaves when a recursive search is done
       leafResultAttribute: "mailPrimaryAddress"
 
-  {{- if .Values.antivirus.milter.host }}
-  smtpdMilters: "inet:{{ .Values.antivirus.milter.host }}:{{ .Values.antivirus.milter.port }}"
-  {{- else }}
-  {{- if .Values.apps.clamavDistributed.enabled }}
-  smtpdMilters: "inet:clamav-milter:7357"
-  {{- else if .Values.apps.clamavSimple.enabled }}
-  smtpdMilters: "inet:clamav-simple:7357"
-  {{- end }}
-  {{- end }}
   virtualMailboxDomains: {{ toYaml (prepend .Values.global.additionalMailDomains (.Values.global.mailDomain | default .Values.global.domain) | uniq) | nindent 4 }}
   virtualTransport: "lmtps:dovecot:24"
 

helmfile/apps/services-external/values-postfix.yaml.gotmpl

@@ -57,10 +57,23 @@ postfix:
   hostname: "postfix"
   inetProtocols: "ipv4"
   milterDefaultAction: "accept"
-  {{- if .Values.apps.dkimpy.enabled }}
-  dkimpyHost: "opendesk-dkimpy-milter.{{ .Release.Namespace }}.svc.{{.Values.cluster.networking.domain }}:8892"
-  {{- end }}
-  rspamdHost: ""
+  smtpdMilters:
+    {{- if .Values.apps.dkimpy.enabled }}
+    - host: "opendesk-dkimpy-milter.{{ .Release.Namespace }}.svc.{{.Values.cluster.networking.domain }}"
+      port: 8892
+    {{- end }}
+    {{- if .Values.antivirus.milter.host }}
+    - host: {{ .Values.antivirus.milter.host | quote }}
+      port: {{ .Values.antivirus.milter.port }}
+    {{- else }}
+    {{- if .Values.apps.clamavDistributed.enabled }}
+    - host: "clamav-milter"
+      port: 7357
+    {{- else if .Values.apps.clamavSimple.enabled }}
+    - host: "clamav-simple"
+      port: 7357
+    {{- end }}
+    {{- end }}
   {{- if .Values.smtp.host }}
   relayHost:
     enabled: true
@@ -116,15 +129,6 @@ postfix:
       # -- return the following attribute from all found leaves when a recursive search is done
       leafResultAttribute: "mailPrimaryAddress"
 
-  {{- if .Values.antivirus.milter.host }}
-  smtpdMilters: "inet:{{ .Values.antivirus.milter.host }}:{{ .Values.antivirus.milter.port }}"
-  {{- else }}
-  {{- if .Values.apps.clamavDistributed.enabled }}
-  smtpdMilters: "inet:clamav-milter:7357"
-  {{- else if .Values.apps.clamavSimple.enabled }}
-  smtpdMilters: "inet:clamav-simple:7357"
-  {{- end }}
-  {{- end }}
   # Only deliver mail to Dovecot, if it is available
   {{- if .Values.apps.oxAppSuite.enabled }}
   virtualMailboxDomains: {{ toYaml (prepend .Values.global.additionalMailDomains (.Values.global.mailDomain | default .Values.global.domain) | uniq) | nindent 4 }}

helmfile/environments/default/charts.yaml.gotmpl

@@ -437,7 +437,7 @@ charts:
     registry: "registry.opencode.de"
     repository: "bmi/opendesk/components/platform-development/charts/opendesk-postfix"
     name: "postfix"
-    version: "5.1.0"
+    version: "5.1.1"
     verify: true
   postgresql:
     # providerCategory: "Platform"

helmfile/environments/default/smtp.yaml.gotmpl

@@ -7,6 +7,9 @@ smtp:
   username: ""
   password: {{ env "SMTP_PASSWORD" | quote }}
   localpartNoReply: "no-reply"
+  spamMilter:
+    host: ""
+    port: 11332
 
   # For the following settings to have effect `apps.dkimpy.enabled` must be `true`.
   dkim: