sheppy/opendesk
1
0
Fork 0
mirror of https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk.git synced 2025-12-05 23:11:40 +01:00
Code Issues Packages Projects Releases Wiki Activity

docs(testing.md): Add link to load test repo and information about nightly tests

Browse Source
This commit is contained in:
René Fischer
2025-04-02 19:17:11 +00:00
committed by Thorsten Roßner
parent 79975a58e3
commit e25cb5e0f3
2 changed files with 60 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/baseline-requirements.md
View File

@@ -119,7 +119,7 @@ openDesk should be compliant with the "Deutsche Verwaltungscloud Strategie" (DVS
## IT-Grundschutz
The IT-Grundschutz catalog knowns a lot of modules ("Bausteine"), but not all of them apply to all components, as there are some related to hardware or some just relevant for the operator while openDesk is "just" the software platform. The first step for an IT-Grundschutz evaluation of a component (or the platform as a whole) requires defining which modules are applicable. Other modules apply to all components e.g. [APP.4.4 Kubernetes](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/06_APP_Anwendungen/APP_4_4_Kubernetes_Edition_2023.pdf), [SYS.1.6 Containerisierung](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/07_SYS_IT_Systeme/SYS_1_6_Containerisierung_Edition_2023.pdf) and [CON 2 Datenschutz](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/03_CON_Konzepte_und_Vorgehensweisen/CON_2_Datenschutz_Edition_2023.pdf).
The IT-Grundschutz catalog knowns a lot of modules ("Bausteine"), but not all of them apply to all components, as there are some related to hardware or some just relevant for the operator while openDesk is "just" the software platform. The first step for an IT-Grundschutz evaluation of a component (or the platform as a whole) requires defining which modules are applicable. Other modules apply to all components e.g. [APP.4.4 Kubernetes](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/06_APP_Anwendungen/APP_4_4_Kubernetes_Edition_2023.pdf), [SYS.1.6 Containerisierung](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/07_SYS_IT_Systeme/SYS_1_6_Containerisierung_Edition_2023.pdf) and [CON.2 Datenschutz](https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/IT-GS-Kompendium_Einzel_PDFs_2023/03_CON_Konzepte_und_Vorgehensweisen/CON_2_Datenschutz_Edition_2023.pdf).
Within each module are multiple requirements ("Anforderungen") that are usually composed of multiple partial requirements ("Teilanforderungen"). Each requirement has a given category:
- B for basic ("Basis") - the requirement must be fulfilled.
@@ -133,7 +133,7 @@ Different requirements address different roles in IT-Grundschutz.
- Operator: processes & product (service)
- Customer: processes.
As a supplier of an openDesk component, you will focus on the "Supplier" requirements, while the outcome (your product) must enable the Provider to fulfill the requirements that lay with its responsibility for the openDesk platform. Operators use openDesk to provide a service, therefore the openDesk platform must enable an Operator to fulfill the related requirements. Finally, the service must enable the customer to align with the scope of the IT Grundschutz catalog. So it will happen that a requirement from e.g. the customer level needs a specific capability by the product (Supplier's responsibility), a defined core configuration from the platform (Provider's responsibility), or a certain service setup from the Operator.
As a supplier of an openDesk component, you will focus on the "Supplier" requirements, while the outcome (your product) must enable the Provider to fulfill the requirements that lay with its responsibility for the openDesk platform. Operators use openDesk to provide a service, therefore the openDesk platform must enable an Operator to fulfill the related requirements. Finally, the service must enable the customer to align with the scope of the IT-Grundschutz catalog. So it will happen that a requirement from e.g. the customer level needs a specific capability by the product (Supplier's responsibility), a defined core configuration from the platform (Provider's responsibility), or a certain service setup from the Operator.
We are aware that IT-Grundschutz is a complex topic and are working towards a streamlined process to reduce overhead as much as possible and ensure to maximize the use of synergies.
@@ -167,7 +167,7 @@ Each component must be able to operate according to the [EU's General Data Prote
While this can be answered by each component that will be in the spotlight for the suppliers, we also need an aligned overall picture for openDesk that at least has the platform-specific user lifecycle and cross-application interfaces in focus.
Note: The topics of availability, integrity, and confidentiality of personal data are also being addressed by the IT-Grundschutz module "CON 2". It has to be ensured that it is not in contradiction to what is being done in the general area of data protection.
Note: The topics of availability, integrity, and confidentiality of personal data are also being addressed by the IT-Grundschutz module "CON.2". It has to be ensured that it is not in contradiction to what is being done in the general area of data protection.
**Reference:** https://gitlab.opencode.de/bmi/opendesk/documentation/datenschutz

99
docs/testing.md
View File

@@ -9,14 +9,14 @@ SPDX-License-Identifier: Apache-2.0
* [Overview](#overview)
* [Test concept](#test-concept)
* [Rely on upstream applications QA](#rely-on-upstream-applications-qa)
* [Run minimal functional QA (end-to-end tests)](#run-minimal-functional-qa-end-to-end-tests)
* [Run extensive load and performance tests](#run-extensive-load-and-performance-tests)
* [Functional QA (end-to-end tests)](#functional-qa-end-to-end-tests)
* [Nightly testing](#nightly-testing)
* [Reporting test results](#reporting-test-results)
* [Load- and performance testing](#load--and-performance-testing)
* [Base performance testing](#base-performance-testing)
* [Load testing to saturation point](#load-testing-to-saturation-point)
* [Load testing up to a defined user count](#load-testing-up-to-a-defined-user-count)
* [Overload/recovery tests](#overloadrecovery-tests)
* [Reporting and test results](#reporting-and-test-results)
* [Allure TestOps](#allure-testops)
<!-- TOC -->
# Overview
@@ -42,24 +42,65 @@ We receive the release notes early before a new application release is integrate
we are able to check for the existence of a sufficient set of test cases.
The suppliers create a set of test cases for each new function.
## Run minimal functional QA (end-to-end tests)
## Functional QA (end-to-end tests)
To ensure the function of all applications, we run a minimal set of testcases to check the
basic functionality of openDesk along with the integrated applications.
We develop and maintain a [set of end-to-end tests](https://gitlab.opencode.de/bmi/opendesk/deployment/e2e-tests) focussing on:
Furthermore, we analyze all features and use cases which are implemented by a set of more than one
application.
Not all of these features are testable by the suppliers, so we develop testcases
for such features.
- use cases that are spanning more than a single application, e.g.
- the filepicker in OX App Suite for selecting files from Nextcloud or
- the central navigation that is part of the top bar of most applications.
- openDesk specific configurations/supported settings that can be found in the `functional.yaml.gotmpl`, e.g.
- SSO federation or
- sharing settings for Nextcloud.
- bugs identified in the past, e.g.
- creating a folder in OX or
- enforcement of an account's password renewal.
The openDesk application owners prioritize this list of end-to-end-testcases, and we
implement these testcases in the [test automation framework](https://gitlab.opencode.de/bmi/opendesk/deployment/e2e-tests).
We execute the tests using English and German as language profile.
## Run extensive load and performance tests
The development team utilizes the test automation described above for QA'ing their feature branches.
### Nightly testing
We use the functional e2e-tests in nightly testruns on a matrix of deployments addressing different application profiles to ensure the quality of the development branch's current state.
The following naming scheme is applied for the deployment matrix:
- `<edition>-<type>-<profile>` resulting e.g. in `ce-init-default` or `ee-upgr-extsrv`
**`<edition>`**
- `ce`: openDesk Community Edition
- `ee`: openDesk Enterprise Edition
**`<type>`**
- `init`: Initial / fresh / from the scratch deployment of `develop` branch into an empty namespace.
- `upgr`: Deploy latest migration release (needs to be pinned manually) into an empty namespace, afterwards run upgrade deployment with current state of `develop` branch.
- `upd`: Deploy latest release (`main` branch) into an empty namespace, afterwards run upgrade deployment with current state of `develop` branch.
**`<profile>`**: The following profiles are defined
- `default`: With
- *`functional.yaml`*: No changes beside specific `2FA testing` group and enabled UDM REST API (required for user import).
- *Services*: Internal services deployed with openDesk are used.
- *Secrets*: Master password based secrets based on `secrets.yaml.gotmpl`
- *Certificates*: Letsencrypt-prod certificates are used.
- *Deployment*: GitLab CI based deployment.
- `funct1`: Different configuration of `functional.yaml`, self-signed-certs [and when available external secrets].
- `extsrv`: External services (where possible).
- `gitops`: Argo CD based deployment.
### Reporting test results
All executions of the end-to-end tests are tracked in a central platform running [Allure TestOps](https://docs.qameta.io/allure-testops/).
As the TestOps tool contains infrastructure details of our development and test clusters it is currently only accessible for to project members.
## Load- and performance testing
Our goal is to deliver openDesk as application-grade software with the ability to serve large user bases.
We create and perform extensive load and performance tests for each release of openDesk.
We create and perform [load- and performance tests](https://gitlab.opencode.de/bmi/opendesk/deployment/load-tests) for each release of openDesk.
Our approach consists of different layers of load testing.
@@ -109,30 +150,4 @@ If necessary, we perform overload tests, which will saturate the system with mul
test cases until no further increase in throughput is visible. Then we add even more load
until the first HTTP requests run into timeouts or errors.
After a few minutes, we reduce the load below the saturation point.
Then we check if the system is able to recover from the overload status.
# Reporting and test results
We perform test runs every night, on all of our environments.
For each environment, we define so-called profiles, these contain the features enabled
per environment.
For example: Testing the email features in an environment without deployment of Open-Xchange makes no sense at all.
Also, we test the whole system via a browser with `language=DE` and another browser with `language=EN`.
The test results are saved in an [Allure TestOps](https://qameta.io/) server, so interested persons
are able to view the test results later in detail.
## Allure TestOps
The Allure TestOps [server](https://testops.opendesk.run/) is currently only accessible to project members.
The relevant project is called *opendesk*.
To get an overview, click in the left symbol list onto the symbol "Rocket" to
check all relevant launches.
Now you can see the launch #xxxx, and directly check for the success
of this launch.
Now we can check if the system is able to recover from the overload status.