IT & integrations

Reference architecture

This page describes the reference deployment topology for MANSOQ: a self-contained, two-node Kubernetes cluster that runs the MANSOQ backend and frontend alongside every supporting datastore and platform service. It is the architecture…

Current source · source documentation · 8 min read

This page describes the reference deployment topology for MANSOQ: a self-contained, two-node Kubernetes cluster that runs the MANSOQ backend and frontend alongside every supporting datastore and platform service. It is the architecture implemented by the deployment-k8 repository, where the infrastructure is provisioned with OpenTofu/Terraform and Helm.

The design targets on-premise or single-tenant cloud installations: everything — database, object storage, search, container registry, terminology server, monitoring — lives inside the cluster, so the deployment has no hard dependency on managed cloud services beyond an optional S3-compatible target for off-site backups.

This is a reference design, not a turnkey product. Several pieces are still hardening or evolving — see Planned and evolving below, and treat the day-2 operations page as the operational companion to this one.

How the repository maps to the architecture

The component inventory below is grounded in three directories of the deployment-k8 repository. Reading these is the fastest way to see exactly what gets deployed:

Each infra/<module> is applied with the repository's deployer.py helper (apply-tofu to run the OpenTofu module, apply-k8 to apply any generated Kubernetes manifests). See Deploying MANSOQ onto the cluster for the exact order.

Cluster topology

The reference cluster is two nodes on the same network, provisioned with kubeadm:

• A primary node (labelled ohn/role=primary) that runs the control plane and hosts the stateful, latency-sensitive workloads — PostgreSQL, the RustFS object store, the container registry, and search.
• A secondary / failover node (labelled ohn/role=secondary) that hosts the public-facing ingress and the observability stack, and acts as the failover target.

The primary control-plane node is made schedulable (its control-plane taint is removed) so it can also run application workloads. Node labels drive placement: Helm values pin pods to the correct node via nodeSelector on ohn/role. Data is replicated across both nodes at the storage layer (see Storage) so the cluster can tolerate the loss of a node and be rebuilt from replicated volumes plus off-site backups.

The cluster is built directly with kubeadm against a stable Kubernetes release line (the install notes target the v1.34 package channel). The full node-bootstrap procedure — installing containerd, kubelet/kubeadm/kubectl, joining the worker, removing taints, and applying node labels — is covered in Provisioning the cluster.

Networking

• CNI: Cilium. Pod networking and service routing are handled by Cilium, installed in kubernetes IPAM mode and pointed at the primary node's API server.
• Load balancing: NodePort / node external IPs. Ingress controllers are exposed on the nodes rather than via a cloud load balancer. The public ingress binds the secondary node's IP via externalIPs; other services (for example the registry) are reached on fixed NodePorts. MetalLB is the intended direction for cleaner L2/L3 load balancing — see Planned and evolving.
• Ingress + WAF. Two ingress controllers run side by side, split into a private ingress class (private-nginx, for internal/admin surfaces such as Grafana, the object store console, Odoo, and Metabase) and a public ingress class (public-nginx, for the MANSOQ frontend, MANSOQ API, and other internet-facing hosts). The public ingress runs with a web application firewall enabled — ModSecurity with the OWASP Core Rule Set — to filter inbound traffic. The longer-term target is an Apache APISIX-based ingress with an integrated WAF/firewall.
• TLS termination at the ingress. Certificates terminate at the ingress controller. The reference setup uses Let's Encrypt, ideally a pre-fetched wildcard certificate issued via DNS validation and loaded into each application namespace as a TLS secret (lets-encrypt-cert). Internal pod-to-pod TLS is on the roadmap, not yet in place.

Routing host names, node IPs, and domains are environment-specific and supplied through Terraform variables (for example mansoq_public_domain, mansoq_api_public_domain, primary_node_ip, secondary_node_ip). Do not hardcode site-specific host names into shared manifests — keep them in your environment's variable file.

Storage

• Block storage: Longhorn. Replicated block volumes are provisioned by Longhorn, with dedicated StorageClasses per workload (longhorn-database, longhorn-object, longhorn-registry, longhorn-opensearch, longhorn-prom, longhorn-loki, longhorn-grafana-dashboard). Disks are tagged (ssd, hdd) so latency-sensitive data lands on SSD and bulkier data on HDD, and a Longhorn BackupTarget points at an S3-compatible bucket for volume snapshots.
• Object storage: RustFS. A MinIO-compatible object store (RustFS) provides S3-style buckets for MANSOQ's file uploads and facility data, plus log/audit storage. It runs in standalone mode on the primary node, backed by a Longhorn volume, and is fronted by its own ingress for the management console.

The buckets MANSOQ and the platform expect (for example mansoq-facility, mansoq-fileupload, and the log/ruler buckets used by Loki) are created in RustFS as part of bring-up.

Components

The MANSOQ backend is pluggable: integrations such as mansoq_odoo are installed as additional plugs at deploy time rather than baked into the core image. See the contributor guide on MANSOQ pluggable apps for how plugs are structured.

Observability

Monitoring is a self-hosted Grafana stack, installed by the monitoring module:

• Prometheus + Grafana via the kube-prometheus-stack chart, for metrics and dashboards. Grafana is exposed on the private ingress.
• Loki for log aggregation, with its chunk/ruler storage backed by the in-cluster RustFS object store.
• Alloy runs as the log shipper: it discovers pods, relabels Kubernetes metadata onto log streams, and pushes them to Loki.

Prometheus, Loki, and Grafana persist to Longhorn volumes pinned to the secondary node.

Request and data flow

The high-level path from a client to a datastore:

                         Clients (browsers, integrations)
                                      |
                                      v
              +-------------------------------------------------+
              |   Ingress controllers (nginx + WAF / OWASP CRS) |
              |   TLS termination (Let's Encrypt wildcard)      |
              |   public-nginx  |  private-nginx                |
              +-------------------------------------------------+
                       |                         |
                       v                         v
              +----------------+        +------------------+
              |   MANSOQ FE      |        |   MANSOQ BE        |
              |   (nginx)      |  --->  |  web + celery    |
              +----------------+        |  + beat          |
                                        +------------------+
                                               |
            +----------------+----------------+----------------+
            v                v                v                v
     +-------------+  +-------------+  +-------------+  +-------------+
     | PostgreSQL  |  | Valkey/     |  | OpenSearch  |  |  RustFS     |
     | (CNPG)      |  | Redis       |  | (search)    |  | (S3 object  |
     | care DB     |  | cache/broker|  |             |  |  storage)   |
     +-------------+  +-------------+  +-------------+  +-------------+

Supporting services — Odoo, Metabase, Snowstorm, the zot registry, and the Prometheus/Grafana/Loki stack — sit alongside this path. Odoo and Metabase share the same CNPG-managed PostgreSQL cluster (separate databases), MANSOQ reaches Odoo over the internal service network, and the registry serves the MANSOQ images that the backend and frontend deployments pull.

Planned and evolving

Several aspects of this architecture are explicitly works in progress. Treat the items below as roadmap direction drawn from the design's own backlog, not as current guarantees:

• PostgreSQL fully on a cluster operator. PostgreSQL is being consolidated onto the CloudNativePG operator (moving off hand-written YAML and onto operator-managed clusters) for both MANSOQ and the supporting databases.
• Ingress and load balancing. Migrating toward an APISIX-based ingress with an integrated WAF/firewall, and toward MetalLB for load balancing in place of plain NodePort/external-IP exposure.
• Terminology server. The Snowstorm-based terminology service is still being wired into the standard deployment flow.
• Secrets and automation. Generating credentials through Terraform and auto-wiring them into dependent services, rather than creating secrets by hand.
• Backups and restore. Extending snapshots/backups to cover every stateful component and documenting a clean restore of a fresh cluster from backups.
• Inter-pod TLS. Adding TLS for database and other pod-to-pod communication.
• Version-locking the charts. Pinning all Helm chart versions so deployments are fully reproducible.

For how these components are stood up in practice, continue to Provisioning the cluster and Deploying MANSOQ onto the cluster.

Source evidence

References are pinned to the private MANSOQ source repository. Repository access is required to open these links.

• repositories/platform/mansoq-documentation/deployment/kubernetes/architecture.md:1 · f6e9f05f