High-Level Architecture

Requirements

Technology stack:

Apache Kafka 3.8+ (https://kafka.apache.org/)
Kubernetes 1.28+ (https://kubernetes.io/)
PostgreSQL 14+ (https://www.postgresql.org/)
Apache Flink Operator 1.13+ (can be deployed together with Prime Detect)

Network and infrastructure:

Outbound internet access allowed:
- to *.socprime.com for API access to the SOC Prime Platform for synchronizing detections
- to *.amazonaws.com, *.apache.org for pulling Docker Images
- to *.hub.docker.com for pulling Docker Images
- to *.github.com for pulling open-source detections
Internal access to deployed resources
Network connectivity between deployed resources
Kubernetes cluster with at least 3 nodes(workers) (min 8 vCPUs, 32 GiB RAM each). For a more detailed calculation, see Appendix A

Delivery options:

An installation package (a set of YAML files) to be downloaded from an online resource
An installation package (a set of YAML files and Docker images) to be copied from a physical drive on-site

Deployment Process

Deployment should be performed by a person with expertise in Kubernetes and admin access to the Kubernetes Cluster provided for this project.

Deployment consists of four parts:

Apache Flink (according to its official deployment recommendations)
PostgreSQL
Apache Kafka
Prime Detect itself

Apache Flink Deployment

Install Cert Manager according to https://cert-manager.io/docs/installation/
Run the following command for creating NameSpace:
```
> kubectl create ns flink
```
Output:
```
namespace/flink created
```
Deploy PostgreSQL for Prime Detect

Run the following command:
```
> kubectl apply -f postgresql.yml -n flink
```
Deploy Apache Kafka (One node) for Prime Detect

Run the following command:
```
> kubectl apply -f kafka-kraft.yml -n flink 
```
Get acquainted with the key concepts of Apache Flink Kubernetes Operator here: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/

Deploy Apache Flink Kubernetes Operator using Helm.

A Helm chart manages the installation of the operator. To install the operator (and also the Helm chart) to a specific namespace, run the following commands:

Command:

> helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.13.0/

Output:

"flink-operator-repo" has been added to your repositories

Command:

> helm repo update

Output:

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "flink-operator-repo" chart repository

...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

Command:

> helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator --namespace flink-operator --create-namespace

Output:

NAME: flink-kubernetes-operator
LAST DEPLOYED: Thu Feb  5 12:07:08 2026
NAMESPACE: flink-operator
STATUS: deployed
REVISION: 1
TEST SUITE: None

Source: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/helm/

Deploy ConfigMap with static configuration for Flink Sigma Detector.

Open the flink-configmap.yaml from the installation package and specify values for the following parameters:
- namespace: flink
For all the other parameters, keep the default values.

Run the following command:
```
> kubectl apply -f flink-configmap.yaml -n flink
```
Output:
```
configmap/flink-config created
```
Deploy Secrets for Flink Sigma Detector.

Open the flink-secret.yaml from the installation package and specify the Base64 values for the following parameters (if needed, either leave by default):
- KAFKA_BOOTSTRAP_SERVERS:
- KAFKA_AUTH_METHOD:
Run the following command:
```
> kubectl apply -f flink-secret.yaml -n flink
```
Output:
```
secret/flink-secret created
```
Deploy PersistentVolumeClaims for Flink State Storage.

Open the flink-pvcs.yaml from the installation package and specify values for the following parameters in each name: block:
- namespace:
- storage:
- storageClassName:
Run the following command:
```
> kubectl apply -f flink-pvcs.yaml -n flink 
```
Output:
```
persistentvolumeclaim/flink-checkpoints-pvc created
persistentvolumeclaim/flink-ha-pvc created
persistentvolumeclaim/flink-savepoints-pvc created
```
Optional for PVC: add a selector/label for nodes by running the following command:
```
> kubectl label nodes NAME spec-node-ns-pod-disk=pvc-security
```

Deploy RBAC Configuration for Flink ServiceAccount. It grants Flink permission to create and manage TaskManager pods.

Open the flink-rbac.yaml from the installation package and specify a value for the following parameter:

Namespace: flink

Run the following command:

> kubectl apply -f flink-rbac.yaml -n flink

Output:

serviceaccount/flink created
role.rbac.authorization.k8s.io/flink created
rolebinding.rbac.authorization.k8s.io/flink-role-binding created
role.rbac.authorization.k8s.io/event-reader created
rolebinding.rbac.authorization.k8s.io/grant-event-reader created

Docker Images Pulling

Some of the images are stored at the Private SOC Prime Docker Image Registry Repo at Docker HUB.

SOC Prime will provide login credentials for Private Docker HUB.

Set environments:

export DOCKERHUB_USER="your_username" 
export DOCKERHUB_TOKEN="your_token"

Check environment settings:

echo $DOCKERHUB_USER
echo $DOCKERHUB_TOKEN

Set credentials for namespace flink

kubectl create secret docker-registry dockerhub-creds \
    --docker-server=https://index.docker.io/v1/ \
    --docker-username="${DOCKERHUB_USER}" \
    --docker-password="${DOCKERHUB_TOKEN}" \
    -n flink \
    --dry-run=client -o yaml | kubectl apply -f -

Prime Detect Deployment

Prime Detect consists of two parts:

Admin Panel backend
Admin Panel UI

Admin Panel Backend Deployment

Deploy ConfigMap with static configuration for Admin Panel Backend.

Open admin-panel-be-configmap.yaml from the installation package and specify a value for the following parameter:
- KAFKA_BOOTSTRAP_SERVERS:
  
  Set IP/Hostname and port of the Kafka Server
Run the following command:
```
> kubectl apply -f admin-panel-be-configmap.yaml -n flink
```
Output:
```
configmap/admin-panel-be-config created
```
Deploy a Secret with sensitive data configuration for Admin Panel Backend.

Open admin-panel-be-secret.yaml from the installation package and specify a value for the following parameter:

# DATABASE_URL has the following format:
```
postgresql+asyncpg://user:password@postgres_url:postgres_port/db_name
```
It should be converted to Base64
- DATABASE_URL:
Run the following command:
```
> kubectl apply -f admin-panel-be-secret.yaml -n flink
```
Output:
```
secret/admin-panel-be-secret created
```
Deploy PersistentVolumeClaims for Admin Panel Backend using admin-panel-be-pvc.yaml from the installation package.

Run the following command:
```
> kubectl apply -f admin-panel-be-pvc.yaml -n flink
```
Output:
```
persistentvolumeclaim/admin-panel-metrics-pvc created
```
Create a Deployment resource for Admin Panel Backend using admin-panel-be-deployment.yaml from the installation package.

Run the following command:
```
> kubectl apply -f admin-panel-be-deployment.yaml -n flink
```
Output:
```
deployment.apps/admin-panel-be created
```
Deploy Network Service resource for Admin Panel Backend using admin-panel-be-service.yaml from the installation package.

Run the following command:
```
> kubectl apply -f admin-panel-be-service.yaml -n flink
```
Output:
```
service/admin-panel-be created
```
Check that Admin-Panel-Backend-API is running and accessible by following the link:
- http://network_service_ip:8000

Admin Panel UI Deployment

Deploy ConfigMap with a static configuration for Admin Panel UI.

Open admin-panel-ui-configmap.yaml from the installation package and specify a value for the following parameter:

Note: NAME.NAMESPACE.CLUSTER_DOMAIN_NAME (default: svc.cluster.local)
- VITE_PREVIEW_ALLOWED_HOSTS:
Set internal hostname of admin-panel-be, for example: admin-panel-be.flink.svc.cluster.local

Run the following command:
```
> kubectl apply -f admin-panel-ui-configmap.yaml -n flink
```
Output:
```
configmap/admin-panel-ui-config created
```
Create a Deployment resource for Admin Panel UI using admin-panel-ui-deployment.yaml from the installation package.

Run the following command:
```
> kubectl apply -f admin-panel-ui-deployment.yaml -n flink
```
Output:
```
deployment.apps/admin-panel-ui created
```
Deploy Network Service resource for Admin Panel UI using admin-panel-ui-service.yaml from the installation package.

Run the following command:
```
> kubectl apply -f admin-panel-ui-service.yaml -n flink
```
Output:
```
service/admin-panel-ui created
```
Check that Admin-Panel-UI is running and accessible by following the next link:
- http://network-service-ip:4173

Default:

Email: admin@soc.local

Password: admin

New Resources: Prime Detect Agent

v2.1.0/v2.0.0 introduces two new manifests for the LLM-based Prime Detect agent — a ConfigMap for runtime settings and a Secret for the API key.

detectflow-agent-config-configmap.yaml

ConfigMap detectflow-agent-config — non-sensitive runtime settings for the LLM agent.

apiVersion: v1
kind: ConfigMap
metadata:
 name: detectflow-agent-config
 namespace: flink
data:
 LLM_BASE_URL: http://localhost:8000/v1/   # LLM endpoint the agent calls
 LLM_CALLS_PER_MINUTE: "60"                 # rate limit — max requests/min
 LLM_MAX_RETRIES: "3"                       	# retry attempts on a failed call
 LLM_MAX_TOKENS: "2048"                     	# max tokens per completion
 LLM_MODEL: model                           	# model name/identifier to request
 LLM_PROMPT_CHAR_BUDGET: "24000"            	# max prompt size in characters
 MAX_CONCURRENT_LLM_REQUESTS: "1"           	# concurrency cap for in-flight calls

Note: LLM_BASE_URL points at localhost:8000 — the agent expects the LLM endpoint reachable as a sidecar/local proxy in its own pod, not the cluster-wide admin-panel-be service. Confirm this matches the actual agent deployment topology before rollout.

detectflow-agent-secret.yaml

Secret detectflow-agent-secrets — holds the LLM API key. Ships empty in the manifest — must be populated with a base64-encoded value before applying, otherwise the agent will fail to authenticate to the LLM endpoint.

apiVersion: v1
kind: Secret
metadata:
  name: detectflow-agent-secrets
  namespace: flink
data:
  LLM_API_KEY: ""   # base64-encoded API key — empty by default, fill before apply
type: Opaque

To populate and apply without hand-encoding base64:

kubectl -n flink create secret generic detectflow-agent-secrets \
  --from-literal=LLM_API_KEY='<your-api-key>' \
  --dry-run=client -o yaml | kubectl apply -f -

Upgrade Procedure

All five images live in the same Docker Hub repo (socprimecom/detectflow), tagged by component.

New Images in v2.1.0

Component	Image
Admin Panel Backend	socprimecom/detectflow:admin-panel-be.v2.1.0
Admin Panel UI	socprimecom/detectflow:admin-panel-ui.v2.1.0
Flink Sigma Detector	socprimecom/detectflow:flink-sigma-detector.v2.1.0
DF Agent job	socprimecom/detectflow:df-agent.v2.1.0
DF Correlation job	socprimecom/detectflow:df-correlation.v2.1.0

Upgrade Procedure (v2.0.0 → v2.1.0): Targeted Field Patch + Restart

Image: admin-panel-be deployment

kubectl -n flink set image deployment/admin-panel-be \
  admin-panel-be=socprimecom/detectflow:admin-panel-be.v2.1.0

set image immediately triggers a rollout — no separate restart needed.

Image: admin-panel-ui deployment

kubectl -n flink set image deployment/admin-panel-ui \
  admin-panel-ui=socprimecom/detectflow:admin-panel-ui.v2.1.0

Fields in ConfigMap admin-panel-be-config
Three keys (FLINK_IMAGE, AGENT_JOB_IMAGE, CORRELATION_JOB_IMAGE) feed the Flink jobs launched by admin-panel-be, and are also consumed by the admin-panel-be deployment itself via envFrom: configMapRef (not injected per-key), so the change is only picked up by a new pod after restart.
```
kubectl -n flink patch configmap admin-panel-be-config --type merge -p '
data:
  FLINK_IMAGE: "socprimecom/detectflow:flink-sigma-detector.v2.1.0"
  AGENT_JOB_IMAGE: "socprimecom/detectflow:df-agent.v2.1.0"
  CORRELATION_JOB_IMAGE: "socprimecom/detectflow:df-correlation.v2.1.0"
'
```
ConfigMap changes do not auto-trigger a rollout — restart explicitly:
```
kubectl -n flink rollout restart deployment/admin-panel-be
```
Step 1 already restarts this same deployment. If applying both changes together, one rollout restart after the patch is enough; running it twice is harmless.

Verification

kubectl -n flink rollout status deployment/admin-panel-be

kubectl -n flink rollout status deployment/admin-panel-ui

kubectl -n flink get deployment admin-panel-be -o jsonpath='{.spec.template.spec.containers[0].image}'

kubectl -n flink get deployment admin-panel-ui -o jsonpath='{.spec.template.spec.containers[0].image}'

Rollback
```
kubectl -n flink rollout undo deployment/admin-panel-be

kubectl -n flink rollout undo deployment/admin-panel-ui
```
ConfigMap rollback must be done manually (patch back to the .v2.0.0 values + rollout restart) — rollout undo does not version ConfigMaps.

Appendix A

In terms of calculating required resources, the following components of Prime Detect should be considered, with individual requirements for each component summed up to estimate the total amount:

Admin Panel is the control center to manage event matching processes. This component requires at least 2 CPUs and 4 GB RAM
Task Manager, the default component to control the operation of pipelines (based on match nodes, the Apache Flink technology). This component requires at least 1 CPU and 4 GB RAM
Match Node, a pipeline controlled via Admin Panel that matches detection rules to Apache Kafka topic events. The number of Match Nodes (pipelines) is custom, and you should estimate the required Kubernetes cluster resources based on the number of pipelines you're going to create, the data you're processing (EPS), and how many Sigma rules you'll apply. Reach out to SOC Prime for a detailed resource calculation.

Prime Detect Requirements and Deployment Guide

High-Level Architecture

Requirements

Deployment Process

Apache Flink Deployment

Docker Images Pulling

Prime Detect Deployment

Admin Panel Backend Deployment

Admin Panel UI Deployment

New Resources: Prime Detect Agent

Upgrade Procedure

Appendix A