About this edition
This guide targets a local proof-of-value (PoV) deployment of Prime Detect on Ubuntu using Minikube. It is not a production / multi-node Kubernetes guide.
The deployment script and all configuration files in the provided package contain placeholder data (mock secrets) and are intended for demonstration purposes only.
How to Use This Guide
This guide describes how to deploy Prime Detect for a proof of value on a single Ubuntu machine using Minikube (a local, single-node Kubernetes cluster). Everything is driven by one script and one configuration file.
We learned from real client deployments that the same document has to serve two very different readers, so it is split into two tracks. Read the one that fits you — you do not need both.
Track | Who it is for | What you get |
A — Fast Install | You are comfortable with Linux, Docker, and Kubernetes and just want the commands. | A one-page command sequence per scenario. Minimal explanation. |
B — Guided Walkthrough | You are deploying this for the first time, or you want to be “led by the hand.” | Every step explained, what you will see after each one, time estimates, common mistakes, and the path all the way to your first detection. |
Jump to Track A — Fast Install ·
Jump to Track B — Guided Walkthrough
What you are going to build
One Ubuntu host running a local Minikube cluster with the full Prime Detect stack: a web UI and REST API, a PostgreSQL database, Apache Kafka (with a Kafka UI), a Logstash log-ingestion layer, and an Apache Flink stream-processing engine that applies Sigma rules to incoming events. When everything is up, you log in to the UI, point a log source at it, and watch detections appear — sub-second, in real time.
End state — the finish line
All pods report Running (you will see a list like 1/1 Running for each component).
The script prints a summary with access URLs, login credentials, and port-forward PIDs.
You can open the Prime Detect UI in a browser, log in, send a test event, and see it flow through Kafka into a detection.
How long it takes
Times depend heavily on your hardware and your internet speed (the container images are pulled from Docker Hub). The numbers below are rough estimates for the recommended server.
Phase | What happens | Typical time |
Preparation | Unpack the zip, edit | 10–20 min |
First script run | Installs Docker, adds you to the | 2–5 min |
Re-login | Activate group membership and re-run the script | 1 min |
Second script run | Starts Minikube, pulls images, deploys all 15 components | 15–25 min |
First detection | Log in, tune runtime settings, send a test event, see a match | 10–15 min |
Read these three things before you start
SERVER_HOST accepts an IP address or a domain name (FQDN). Older templates phrased the placeholder as “replace with server IP,” but a hostname works just as well — and is required if you want warning-free HTTPS with your own certificate.
The image names in .env must match exactly what SOC Prime sent you. A single typo in IMAGE_UI, IMAGE_BACKEND or IMAGE_FLINK does not fail fast — the deploy hangs for many minutes trying to pull an image that does not exist. See Step 1 and Troubleshooting.
The very first script run stops on purpose after installing Docker. It is not broken. You log out / back in (or run newgrp docker) and run it once more. See Step 3.
Part 1 — Before You Begin
What Prime Detect is
Prime Detect is a real-time SIEM detection pipeline platform by SOC Prime. It applies Sigma rules to streaming events from Apache Kafka using Apache Flink, providing sub-second threat detection. Events arrive (via Logstash or directly into Kafka), Flink pipelines evaluate Sigma rules against them, and matches surface in the UI.
Tech stack
You do not install these by hand — the script does. The table is here so you recognize the names when you see them scroll past during deployment.
Dependency | Version | Purpose | Required by |
Kubernetes | 1.28+ | Container orchestration | All components |
PostgreSQL | 14+ | Backend database (rules, pipelines, users) | Backend |
Apache Kafka | 3.8+ | Event streaming & rule distribution | Backend, Flink Sigma Detector |
Apache Flink | 2.2 | Stream-processing engine | Flink Sigma Detector |
Flink Kubernetes Operator | 1.13.0 | Manages Flink deployments as CRDs | Flink Sigma Detector lifecycle |
cert-manager | 1.17.2 | TLS certificate management (Flink operator; app TLS in selfsigned mode) | Flink Operator, optional TLS |
Traefik | chart (set in | Per-port TLS termination for optional HTTPS (UI, API, Kafka UI, dashboard) | Optional TLS ( |
The deploy script also installs these command-line tools if they are missing (check-then-install):
Tool | Purpose | Installed from |
docker | Container runtime + driver | Official apt repo ( |
minikube | Local Kubernetes cluster | Official binary → |
kubectl | Kubernetes CLI | Official binary from |
helm | Kubernetes package manager | Official |
kcat | Kafka CLI producer/consumer |
|
envsubst | Render |
|
Prerequisites checklist
Confirm each item before you touch the script:
A clean Ubuntu host — 22.04, 24.04 or 26.04.
Resources. Tuned for a 32 CPU / 64 GB RAM / 128 GB SSD server. Minimum ~16 CPU / 24 GB RAM / 64 GB disk. Less than this and pods will not fit (see Troubleshooting → out of resources).
Docker Hub credentials — a username and an access token that has permission to pull the private Prime Detect images. SOC Prime grants this access; without it nothing pulls.
The Prime Detect images. SOC Prime makes the private images available to your Docker Hub account. Confirm you can see them before deploying.
Network reachability. Decide how clients will reach this host — by IP or by hostname/FQDN. That value goes into
SERVER_HOST.
Do I need to be root?
No — the recommended path is a normal user in the docker group. Running as root is supported but Minikube's docker driver then needs an extra flag (MINIKUBE_FORCE=true). Both paths are covered below.
Choose your deployment profile
There are three common ways to run the PoV. They differ only in who runs the script and whether/how TLS (HTTPS) is enabled. Pick one now; each track tells you exactly what changes for your choice.
Profile | Run as | Access by | TLS | Browser warning? |
1 - Basic | normal user | IP address | Off (HTTP) | n/a |
2 - Self-signed | root ( | IP address | On, | Yes, until you import the CA |
3 - Provided cert | normal user | FQDN | On, | No (valid cert) |
Not sure? Start with Profile 1.
Profile 1 (normal user, IP, plain HTTP) is the simplest and fastest. You can switch TLS on later by editing .env and re-running — no need to start over.
Part 2 — Track A: Fast Install
For readers who know Docker and Kubernetes. If any command surprises you, switch to Track B — Guided Walkthrough, which explains every step.
Two gotchas even fast installers hit
IMAGE_* values in .env must match exactly what SOC Prime provided, or the deploy hangs at the image-pull step (not an instant failure).
On a host where your user is not yet in the docker group, the first run installs Docker and exits. Re-login and run again.
Common steps (all profiles)
# 1) Get the code
# Unpack files provided in the detectflow-oss-private-ubuntu-based.zip, then cd into the folder
# 2) Configure
cp .env.example .env
# Edit at least: SERVER_HOST (IP or FQDN) and IMAGE_* (exact tags from SOC Prime)
chmod +x deploy-detectflow-minikube.sh
# 3) (Optional) provide Docker Hub creds up front to avoid the prompt
export DOCKERHUB_USER=youruser DOCKERHUB_TOKEN=dckr_pat_xxx
Profile 1 - normal user, IP, HTTP
./deploy-detectflow-minikube.sh
# First run installs Docker + adds you to the 'docker' group, then stops:
newgrp docker # or log out / back in
./deploy-detectflow-minikube.sh # run again; proceeds to full deploy
Access (default ports): UI http://SERVER_HOST:4173, API:8000, Kafka UI:8080, K8s Dashboard:8443, Kafka broker SERVER_HOST:9094, Logstash:10501/:10510/:10520.
Profile 2 - root, IP, self-signed TLS
Set in .env: TLS_ENABLED=true, TLS_MODE=selfsigned. Then:
MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh
# (root skips the docker-group dance; MINIKUBE_FORCE allows the docker driver as root)
UI/API/Kafka-UI/Dashboard now serve HTTPS on the same ports (https://SERVER_HOST:4173, etc.). Browsers warn until you import the CA:
kubectl get secret detectflow-ca-keypair -n cert-manager \
-o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crt
# import detectflow-ca.crt into your OS/browser trust store
Profile 3 — normal user, FQDN, provided wildcard cert
Put your PEM cert/key under certs/ (gitignored). Set in .env:
TLS_ENABLED="true"
TLS_MODE="provided"
TLS_CERT_FILE="/path/to/your/cert" # full chain (leaf + intermediates)
TLS_KEY_FILE="/path/to/your/key" # unencrypted PEM
SERVER_HOST="df-005.cloud.example.com" # a name your cert covers (wildcards: use a sub-host, not the apex)
./deploy-detectflow-minikube.sh
newgrp docker && ./deploy-detectflow-minikube.sh # if first run installed Docker
HTTPS on the FQDN with no browser warning (valid cert).
Verify (any profile)
kubectl get pods -n flink # all should be 1/1 Running
# send a quick test event into Logstash (syslog input) and read it back from Kafka:
bash -c 'echo "<13>Feb 24 19:02:11 HOST sshd[1]: test" > /dev/tcp/SERVER_HOST/10501'
kcat -C -b SERVER_HOST:9094 -t logstash-syslog -o beginning -e
Then open the UI, log in (default admin@soc.local / admin), tune Pipeline Runtime settings, and create your first pipeline — see Step 9: From deploy to first detection.
Part 3 — Track B: Guided Walkthrough
This track assumes no prior Kubernetes or Kafka experience. It walks the whole deployment, tells you what each command does, what you should see afterwards, how long things take, and what to do when something looks wrong. Follow the steps in order.
How the steps map to your profile
The walkthrough is written for Profile 1 (normal user, IP, HTTP) because it is the simplest. Where Profile 2 (self-signed TLS) or Profile 3 (FQDN + your cert) differ, a coloured box calls it out. Decide your profile in
Step 0 — Gather what you need
Before opening a terminal, have these ready. Missing one is the most common reason a deploy stalls later.
Access to the Ubuntu host (SSH or a local session) on 22.04 / 24.04 / 26.04.
Docker Hub username + access token with pull permission for the Prime Detect images (from SOC Prime).
The exact image names/tags SOC Prime gave you for
IMAGE_BACKEND,IMAGE_UI,IMAGE_FLINK,IMAGE_AGENT,IMAGE_CORRELATION. Copy-paste them; do not retype.Your
SERVER_HOSTvalue — the IP or hostname clients will use to reach this machine.(Profile 3 only) Your TLS certificate and private key files (PEM).
What success looks like at the end of Step 0
You have a terminal on the host, your Docker Hub token in hand, and the five image names on a sticky note (or clipboard).
Step 1 — Get the code and create your .env
Unpack the files SOC Prime provided to you (detectflow-oss-private-ubuntu-based.zip) and cd into the resulting folder:
unzip detectflow-oss-private-ubuntu-based.zip
cd detectflow-oss-private-ubuntu-based
All settings live in a file called .env. Copy the template, then open it in an editor (nano .env is fine):
cp .env.example .env
nano .env
You only need to touch a few values. The two that matter most for a first run are SERVER_HOST and the IMAGE_* tags.
All five images live in the same Docker Hub repo (socprimecom/detectflow), tagged by component.
Component | Image |
Admin Panel Backend | socprimecom/detectflow:admin-panel-be.v2.1.0 |
Admin Panel UI | socprimecom/detectflow:admin-panel-ui.v2.1.0 |
Flink Sigma Detector | socprimecom/detectflow:flink-sigma-detector.v2.1.0 |
DF Agent job | socprimecom/detectflow:df-agent.v2.1.0 |
DF Correlation job | socprimecom/detectflow:df-correlation.v2.1.0 |
Variable | What to set it to |
| This server's IP address or hostname/FQDN. Used for the Kafka external listener and the printed URLs. For remote clients, it must not be If you plan to use your own certificate (Profile 3), set this to a name your certificate covers. |
| The exact Docker Hub image tags SOC Prime sent you. Paste them — a typo here causes a long hang later (see the warning below). All five are required — the script fails fast if any is empty. IMAGE_AGENT (df-agent) and IMAGE_CORRELATION (df-correlation) are the job images the backend launches. |
| Port-forward bind address. |
| Cluster sizing (default 24 / 49152 MB / 100g). Lower these if your host is smaller than the recommended spec. |
| PostgreSQL and Kafka resources/tuning. Defaults are fine for a PoV. |
Stopper #1 — SERVER_HOST is not IP-only
Some older templates wrote the placeholder as REPLACE_WITH_SERVER_IP, which reads as “IP only.” That is misleading. SERVER_HOST happily takes a domain / FQDN too (for example df-005.cloud.example.com). A hostname is in fact required for a warning-free certificate in Profile 3.
Stopper #2 — a wrong IMAGE_* name hangs the deploy
If IMAGE_UI (or any image tag) is mistyped, the deploy does not stop immediately. Kubernetes keeps retrying to pull an image that does not exist, so the script appears to “freeze” — in one client case for ~15 minutes around the image-pull step — before the pod lands in ImagePullBackOff.
Prevention: copy-paste the tags exactly. Detection: if a step seems stuck, open a second terminal and run kubectl get pods -n flink; a ImagePullBackOff or ErrImagePull status points straight at the bad image name. Fix it in .env and re-run.
Secrets and .env
.env is gitignored. Your Docker Hub credentials are never written to .env — you provide them at run time (Step 4). Keep certificate files out of git too (put them under certs/).
Step 2 — Set your TLS profile (optional)
TLS is off by default. If you are doing Profile 1, skip this step entirely — everything will serve plain HTTP. Come back and switch TLS on later if you want; you only edit .env and re-run.
When TLS is enabled, Traefik terminates TLS and the UI, API, Kafka UI and K8s dashboard serve HTTPS on the same port numbers (so http://…:4173 becomes https://…:4173). The Kafka broker and Logstash always stay plaintext.
Profile 2 — self-signed certificate
Set in .env:
TLS_ENABLED=true
TLS_MODE=selfsigned
cert-manager issues a certificate from a self-signed CA, using SERVER_HOST as the certificate name (SAN). Browsers will warn until you import that CA (covered in Step 7). Good for a demo on a LAN by IP.
Profile 3 — your own (wildcard) certificate
Use this when you have a valid certificate and want no browser warnings. Put the PEM files somewhere safe (for example under certs/) and set in .env:
# --- TLS (optional, per-port) ---
TLS_ENABLED="true"
TLS_MODE="provided" # your own cert, not self-signed
TRAEFIK_VERSION="34.1.0" # verify: helm search repo traefik/traefik --versions
TLS_CERT_FILE="/path/to/your/cert" # full chain (leaf + intermediates), PEM
TLS_KEY_FILE="/path/to/your/key" # unencrypted PEM key
SERVER_HOSTmust be a name the certificate covers. For a wildcard such as*.cloud.example.com, use a sub-host likedf-005.cloud.example.com— wildcards do not cover the apexcloud.example.com.TLS_CERT_FILEshould be the full chain (leaf + intermediates). Check withgrep -c 'BEGIN CERTIFICATE' cert.pem(1= leaf only; append intermediates if clients report an incomplete chain).TLS_KEY_FILEmust be an unencrypted PEM key (head -1 key.pemmust not sayENCRYPTED; decrypt withopenssl rsa -in key.pem -out key.dec.pem).The secret lives in the files, not in
.env.
Full TLS reference
Switching modes, trusting the CA, and verifying what is served on the wire are all in Part 5 → HTTPS / TLS.
Step 3 — First run: Docker installs, then the script stops
Make the script executable and run it as your normal user:
chmod +x deploy-detectflow-minikube.sh
./deploy-detectflow-minikube.sh
If Docker was not already installed (or your user was not yet in the docker group), the script installs Docker, adds you to the group, and stops on purpose. You will see something like:
[✓] Docker Engine installed
[i] Adding 'ansuser' to the 'docker' group...
[!] You were added to the 'docker' group. Log out/in (or run 'newgrp docker')
[!] and re-run this script so the group membership takes effect.
Stopper #3 — this is NOT a failure
Linux only applies a new group at the start of a fresh login session, so your current shell still cannot talk to the Docker socket. The script does the safe thing: it stops and asks you to refresh your session. Nothing is broken.
If you ignore this and just re-run, you will get Cannot talk to Docker daemon — same cause, same fix.
Activate the group membership, then run the script a second time:
newgrp docker # OR fully log out and back in
./deploy-detectflow-minikube.sh # run again — this time it proceeds
What you should see now
On the second run, the script confirms Docker is reachable and moves on to ask for your Docker Hub credentials (Step 4). You will see lines like [✓] Docker is installed and reachable.
Profile 2 (root) skips all of this
If you run the script as root, there is no docker-group step. But Minikube's docker driver refuses to run as root, so you start it with the force flag instead: MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh.
Step 4 — Provide your Docker Hub credentials
The Prime Detect images are private, so the script needs a Docker Hub login to pull them. The credentials are temporary — they are never written to .env. You have two options.
Option A — export before running (no prompt)
export DOCKERHUB_USER=youruser DOCKERHUB_TOKEN=dckr_pat_xxx
./deploy-detectflow-minikube.sh
Option B — let the script prompt you
Just run the script; it asks interactively (the token is hidden as you type):
Please enter your Docker Hub username: ...
Please enter your Docker Hub access token ...
Login Succeeded
[✓] Docker Hub login successful
What you should see now
After a successful login, the script begins the actual installation. From here on it asks no more questions — you can let it run.
Pulls fail with a 401 / no images?
Your token lacks pull permission for the Prime Detect images, or the credentials were mistyped. See Troubleshooting → Image pull errors.
Step 5 — Watch the installation
The script now deploys the full stack as a numbered sequence (you will see headers like ═══ Step 2/15: Starting Minikube Cluster ═══). It installs any missing tools, starts Minikube, then installs each component in order. All Kubernetes manifests live in manifests/ next to the script; ${VAR} placeholders are filled from your .env at apply time. Nothing is fetched from external git repos, and no images are built locally — they are pulled from Docker Hub.
Roughly what happens, and what it is waiting for:
Tools — docker, minikube, kubectl, helm, kcat, envsubst (skipped if already present).
Minikube cluster starts with the CPU/RAM/disk from
.env(1–3 min).cert-manager comes up (its 3 pods must be Running before the Flink operator installs).
Flink Kubernetes Operator, PostgreSQL, Kafka + Kafka UI, Logstash, then the Prime Detect backend and UI.
Port-forwards are created last so you can reach everything from your machine.
What “good” looks like while it runs
Pods appear and move through Pending → ContainerCreating → Running. A pod sitting in Pending for a minute or two while images download is normal.
Expect the whole second run to take roughly 15–25 minutes on the recommended server — mostly image-download time. Slower hardware or a slow connection takes longer.
If a step seems frozen for many minutes
Open a second terminal and look at the pods: kubectl get pods -n flink. A status of ImagePullBackOff / ErrImagePull means a bad or unreachable image name (re-check your IMAGE_* from Step 1). Pending that never clears usually means the host is out of CPU/RAM/disk (see Troubleshooting).
First run ended after only Docker installed (0/15)?
That is the expected Step 3 behaviour, not a crash. Re-login (newgrp docker) and run the script again — the second run does the real 15-step deploy.
Step 6 — Read the final summary
When the deploy finishes, the script prints a summary block: access URLs (with your SERVER_HOST filled in), the default login, the pod list, and the port-forward PIDs. For Profile 1 it looks like this:
DetectFlow Demo - Local Deployment
==================================
All components deployed successfully!
Access URLs (SERVER_HOST=10.11.14.59):
UI: http://10.11.14.59:4173
Backend API: http://10.11.14.59:8000
Kafka UI: http://10.11.14.59:8080
Kafka Broker: 10.11.14.59:9094 (external listener)
K8s Dashboard: http://10.11.14.59:8443
Logstash: 10.11.14.59:10501 syslog, 10510 json, 10520 unstructured
Login Credentials:
Email: admin@soc.local
Password: admin
Pod Status:
NAME READY STATUS RESTARTS AGE
admin-panel-be-7d6bb8cd48-t6spc 1/1 Running 0 82s
admin-panel-ui-7c78d784b4-mvq8z 1/1 Running 0 39s
kafka-868f945d5-bgfh4 1/1 Running 0 3m59s
kafka-ui-7966478868-dsnnd 1/1 Running 0 2m42s
logstash-b54784f8d-sjcmj 1/1 Running 0 3m2s
postgres-7f79948ccd-xpqpc 1/1 Running 0 4m35s
The 10.11.14.59 above is only an example — yours shows whatever you set as SERVER_HOST. Every pod should read 1/1 Running.
With TLS on (Profile 2 / 3), the URLs say https
The summary prints https://… on the UI/API/Kafka-UI/Dashboard ports, plus a TLS line. For self-signed, it adds: “Self-signed CA: browsers warn until you import it.” For provided certs it simply says “TLS enabled (provided).” The Kafka broker and Logstash lines stay plaintext in all cases.
The summary also lists port-forward PIDs, for example:
Port-Forward PIDs:
Backend: 16906
UI: 16907
Kafka UI: 16908
Kafka: 16909
Logstash: 16910
Dashboard: 10581
# (TLS on: a single 'Traefik' PID carries UI/API/Kafka-UI/Dashboard instead)
Those forwards are what make the services reachable from your browser. If they die (for example when your SSH session ends), restart them — see Part 5 → Useful commands. To stop them all at once: kill <PID> <PID> ... using the PIDs from your own run.
Step 7 — Open the UI and log in
In a browser, go to the UI URL exactly as printed in your summary, then sign in with the default credentials:
Field | Value |
| |
Password |
|
Stopper #4 — use the right HTTP vs HTTPS
The single most common login problem is the wrong scheme. If you deployed without TLS (Profile 1), the UI is at http://SERVER_HOST:4173 — typing https:// will fail to connect. If you deployed with TLS (Profile 2/3), it is https://SERVER_HOST:4173 — plain http:// will not work. Always use the URL from your summary (Step 6).
Profile 2: the browser warns about the certificate
With a self-signed certificate, the browser shows a warning until you import the CA. That is expected. To remove the warning, export the CA once and import it into your OS/browser trust store:
Run: kubectl get secret detectflow-ca-keypair -n cert-manager -o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crt, then import detectflow-ca.crt.
What you should see now
The Prime Detect UI loads with a top navigation (Dashboard, Pipelines, Settings) and, under Settings, a left menu (Repositories, Topics, Log Sources, Filters, User Management, Audit Logs, Pipeline Runtime).
Step 8 — Set Pipeline Runtime settings (before your first pipeline)
After the stack is deployed and you log in to the Prime Detect UI, review the Pipeline Runtime settings and tune them to your system resources before you create or run a pipeline.
Settings to review:
Parallelism — how many parallel subtasks Flink uses.
TaskManager CPU (cores) and TaskManager Memory (MB) — resources per Flink TaskManager.
Window Size (sec) — the time window for stateful rule evaluation.
Checkpoint Interval (sec) — how often Flink checkpoints its state.
Enable Autoscaling — automatically adjust parallelism based on workload.
These values significantly affect performance and resource usage. Adjust them as needed for your environment.
Refer to the screenshot below for an example configuration:
Settings → Pipeline Runtime, with example values for a PoV host.
Sensible starting point on the recommended server
Parallelism 2, TaskManager 2 cores / 8192 MB, Window 30 s, Checkpoint 60 s, Autoscaling off. Raise parallelism and memory only if you push high event volumes. Save the settings before moving on.
Step 9 — From deploy to your first detection
Why this section exists
A clean deploy and an open dashboard left earlier readers asking “now what?” This step closes the loop: get one event in, and see one detection out.
9.1 — Send a test event
The fastest way to prove the pipeline end-to-end is to push a sample event into Logstash. The syslog input listens on port 10501 and writes to the Kafka topic logstash-syslog. Replace SERVER_HOST with yours:
# syslog (RFC3164) on 10501
bash -c 'echo "<13>Feb 24 19:02:11 FILESERVER01 sshd[1234]: Accepted password for alice" > /dev/tcp/SERVER_HOST/10501'
Prefer raw Kafka? Produce a structured event straight onto a topic with kcat (see Part 5 for the full example):
echo '{"event_id":4624,"computer":"FILESERVER01","message":"An account was successfully logged on."}' \
| kcat -P -b SERVER_HOST:9094 -t windows-events9.2 — Confirm it reached Kafka
Read the topic back. If your event prints, ingestion works:
kcat -C -b SERVER_HOST:9094 -t logstash-syslog -o beginning -e
You can also watch this visually in the Kafka UI (SERVER_HOST:8080): open the topic and confirm the message count went up. Logstash auto-creates the logstash-* topics on the first event.
9.3 — Create a pipeline and see the match
In the Prime Detect UI, a pipeline is what applies Sigma rules to a topic and produces detections. At a high level:
Confirm a rule repository is available under Settings → Repositories (the Sigma rules your pipeline will evaluate).
Make sure the topic you sent events to (for example
logstash-syslogorwindows-events) is visible under Settings → Topics / Log Sources.Create a Pipeline that reads from that topic and applies the rule set. Use the Pipeline Runtime defaults you set in Step 8 (you can override per pipeline).
Send events that should trigger a rule (re-run the test event, or feed real logs), and watch the pipeline produce detections.
Open the Dashboard / the pipeline's output to view matches as they arrive.
Exact clicks vary by build and loaded content
The available rules, repositories, and log-source connectors depend on what your environment was provisioned with, so the in-product wording may differ slightly from the outline above. The test event → Kafka topic part (9.1–9.2) is the reliable, verifiable signal that the platform is ingesting; from there the UI guides pipeline creation. If a rule does not match, check that the event shape matches the rule's expected fields.
What success looks like
Your test event appears in its Kafka topic, a pipeline is running against that topic, and at least one detection shows up in the UI. That is your first detection — the PoV is live.
From here, see Part 6 — Where to Go Next for tuning, scaling, and day-2 operations.
Part 4 — Troubleshooting
Each entry is written as: what you see, what it means, and how to fix it. Start with the common-mistakes box — it covers most first-deploy surprises.
The five most common first-deploy surprises
“It installed Docker and quit.” Expected. Re-login (newgrp docker) and run again — the second run does the full deploy.
“It froze for ~15 minutes.” Usually a mistyped IMAGE_* in .env (pod stuck in ImagePullBackOff). Fix the tag, re-run.
“Can't reach the UI.” Wrong scheme — use http:// for no-TLS, https:// for TLS. Use the URL from the summary.
“Browser security warning.” Self-signed cert (Profile 2). Import the CA, or proceed past the warning for a demo.
“Everything died after I closed SSH.” The port-forwards ended with the session. Restart them (Part 5 → Useful commands).
Deployment/cluster
First run stops after installing Docker
You see: You were added to the 'docker' group… re-run this script. Meaning: expected behaviour — the new group is not active in the current shell. Fix:
newgrp docker # or log out / back in
./deploy-detectflow-minikube.sh
Cannot connect to the Docker daemon
You see: Cannot talk to Docker daemon. Is it running? Meaning: you were added to the docker group, but the shell predates it (often after skipping the re-login above). Fix: log out/in or newgrp docker, then re-run. Verify with docker info.
Minikube: “docker driver should not be used with root privileges” (DRV_AS_ROOT)
You see: the deploy refuses to start Minikube as root. Meaning: Minikube's docker driver blocks root. Fix — either run as a normal user in the docker group:
sudo usermod -aG docker $USER # then log out/in (or: newgrp docker)
./deploy-detectflow-minikube.sh # no sudo
…or force it (root + docker driver):
MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh
Minikube out of resources
You see: pods stuck in Pending, scheduling/eviction messages, or Minikube failing to start. Meaning: the host does not have enough CPU/RAM/disk for the requested cluster. Fix: lower MINIKUBE_CPUS / MINIKUBE_MEMORY in .env to fit the host, then reset and redeploy:
minikube delete
./deploy-detectflow-minikube.sh
Pods and components
Pods stuck in Pending / CrashLoopBackOff
You see: a pod that never reaches Running. Fix: inspect it — the events and logs almost always say why (no resources, bad image, failed dependency).
kubectl describe pod <pod-name> -n flink
kubectl logs <pod-name> -n flink
Image pull errors (ImagePullBackOff / ErrImagePull)
You see: a pod in ImagePullBackOff, or a long hang during the image-pull step. Meaning: a wrong image name/tag (Stopper #2) or invalid/insufficient Docker Hub credentials. Fix:
Re-check the
IMAGE_*values in.envagainst exactly what SOC Prime sent (a typo is the usual cause).Confirm valid Docker Hub credentials were provided at the start of the script.
Verify the pull secret exists:
kubectl get secret dockerhub-creds -n flink.Confirm your token has pull permission for the Prime Detect images.
To re-create the pull secret, delete it and re-run:
kubectl delete secret dockerhub-creds -n flink
./deploy-detectflow-minikube.sh
Backend can't connect to PostgreSQL
Ensure the PostgreSQL pod is running:
kubectl get pods -n flink -l app=postgres.Check the DATABASE_URL format:
postgresql+asyncpg://detectflow:detectflow@postgres:5432/detectflow.
Backend can't connect to Kafka
Ensure the Kafka pod is running:
kubectl get pods -n flink -l app=kafka.Give Kafka 30–60 seconds to fully start before the backend connects.
The in-cluster bootstrap server is
kafka:9092(not the external:9094).
Flink Operator not starting
kubectl get pods -n flink-operator
kubectl logs -n flink-operator -l app.kubernetes.io/name=flink-kubernetes-operator
cert-manager issues
Wait for all three cert-manager pods to be Running before the Flink operator installs:
kubectl get pods -n cert-manager
HTTPS / TLS (only when TLS_ENABLED=true)
Can't reach the UI at all
Check the scheme first (Stopper #4): http:// for no-TLS, https:// for TLS. Then confirm the port-forward is alive (next item).
Connection refused on the HTTPS port
Meaning: the Traefik port-forward is not running (it can die when the deploy's shell/SSH session ends). Fix:
kubectl port-forward --address 0.0.0.0 -n traefik svc/traefik 4173:4173 8000:8000 8080:8080 8443:8443 &
Browser shows “TRAEFIK DEFAULT CERT” instead of your cert
Meaning: Traefik is serving its built-in default. Fix: confirm the default TLSStore points at df-tls, and the secret holds the right cert, then reload Traefik:
kubectl get tlsstore default -n traefik -o jsonpath='{.spec.defaultCertificate.secretName}'; echo # df-tls
kubectl get secret df-tls -n traefik -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -issuer -ext subjectAltName
kubectl rollout restart deployment/traefik -n traefikProvided cert keeps reverting to the self-signed one
Meaning: a leftover cert-manager Certificate is still reconciling the secret. The deploy deletes it automatically; to do it by hand:
kubectl delete certificate detectflow-tls -n traefik --ignore-not-found
kubectl delete certificate detectflow-tls -n flink --ignore-not-found
Certificate name-mismatch warning
Meaning: the certificate's SAN does not cover the name you are using. Fix: confirm SERVER_HOST matches the cert (openssl … -ext subjectAltName); for a wildcard use a sub-host, not the apex.
“TLSStore … spec.defaultCertificate.secretName: Required value” during install
Meaning: TLS_SECRET_NAME was not rendered into the Traefik values. Fix: make sure you are running the current script (it exports the variable for envsubst).
Part 5 — Reference
Detailed material for both tracks: access URLs, the full TLS story, Kafka and Logstash access, and the commands you will reach for after deploy.
Access URLs & credentials
After deployment, the script sets up port-forwards (bound to PF_ADDRESS, default 0.0.0.0) and prints the URLs. Replace SERVER_HOST with the value from your .env (remote clients use the server IP/hostname; locally, localhost works).
Service | URL (default ports) | Notes |
Prime Detect UI |
| Main admin panel |
Prime Detect Backend API |
| REST API |
Kafka UI |
| Kafka monitoring & management |
Kafka Broker |
| External listener for remote access |
Logstash Ingest |
| Log ingestion → Kafka |
K8s Dashboard |
| Kubernetes cluster monitoring |
Ports and scheme
Ports are configurable in .env (UI_PORT, API_PORT, KAFKA_UI_PORT, KAFKA_EXTERNAL_PORT, DASHBOARD_PORT, LOGSTASH_SYSLOG_PORT, LOGSTASH_JSON_PORT, LOGSTASH_TEXT_PORT).
The UI / API / Kafka UI / dashboard use http by default. With TLS enabled, they serve https on the same port numbers — see HTTPS / TLS below.
Default login:
Field | Value |
| |
Password |
|
HTTPS / TLS (optional, per-port)
TLS is off by default. When enabled, Traefik terminates TLS so the UI, API, Kafka UI and K8s dashboard serve HTTPS on their existing ports (http://…:4173 becomes https://…:4173). The Kafka broker and Logstash stay plaintext. Set in .env:
TLS_ENABLED=true
TLS_MODE=selfsigned # or: provided
# provided mode only:
TLS_CERT_FILE=/path/to/cert.pem
TLS_KEY_FILE=/path/to/key.pem
selfsigned — cert-manager issues a cert from a self-signed CA, with
SERVER_HOSTas the SAN (IP or hostname). Browsers warn until you import the CA. Good for demo / LAN.provided — bring your own cert. Point
TLS_CERT_FILE/TLS_KEY_FILEat your PEM files and setSERVER_HOSTto a name the cert covers (for a wildcard*.example.com, use a sub-host likedf.example.com— wildcards don't cover the apex) for valid, warning-free SSL.TLS_ENABLED=false(default) → all ports stay plain HTTP, exactly as before.
How it works
Traefik is installed via Helm with one TLS entrypoint per service. A single default certificate (df-tls, set as a TLSStore named default in the traefik namespace) is served on every entrypoint — routes match on path only (no Host), so the default cert is what gets served. The deploy port-forwards Traefik on the UI/API/Kafka-UI/dashboard ports.
Provided-cert requirements
TLS_CERT_FILEshould be the full chain (leaf + intermediates). Check withgrep -c 'BEGIN CERTIFICATE' cert.pem(1= leaf only; append intermediates if clients report an incomplete chain).TLS_KEY_FILEmust be an unencrypted PEM key (head -1 key.pemmust not sayENCRYPTED; decrypt withopenssl rsa -in key.pem -out key.dec.pem).The cert must cover
SERVER_HOST. A wildcard*.cloud.socprime.comcoverssrv-005.cloud.socprime.com(one label) but not the apexcloud.socprime.com.Keep cert/key out of git — put them under
certs/(gitignored).
Trusting the self-signed CA
Export the CA once and import it into your OS/browser trust store to remove the warning:
kubectl get secret detectflow-ca-keypair -n cert-manager -o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crtVerify what's served on the wire
echo | openssl s_client -connect SERVER_HOST:4173 -servername SERVER_HOST 2>/dev/null \
| openssl x509 -noout -issuer -ext subjectAltName
Switching selfsigned ↔ provided
Change TLS_MODE (and the cert paths) in .env and re-run. App pods are not restarted; only the df-tls secret swaps. If the browser still shows the old cert, force a reload (restarts Traefik only, ~5s):
kubectl rollout restart deployment/traefik -n traefik
Kafka broker access
The deploy configures Kafka with two listeners:
Listener | Address | Purpose |
PLAINTEXT |
| In-cluster clients (Backend, Flink, MatchNode) |
EXTERNAL |
| Remote / host access via |
The EXTERNAL listener advertises SERVER_HOST:9094 (from .env) and is port-forwarded on 0.0.0.0, so clients on other machines can produce/consume. Example (replace SERVER_HOST with the server IP/hostname):
# Produce events using kcat
cat <<'JSON' | jq -c . | kcat -P -b SERVER_HOST:9094 -t windows-events
{
"event_id": 4624,
"timestamp": "2024-02-24T19:02:11Z",
"channel": "Security",
"computer": "FILESERVER01.domain.local",
"target_user": "DOMAIN\\alice",
"message": "An account was successfully logged on."
}
JSON
# Consume events
kcat -C -b SERVER_HOST:9094 -t windows-events -o beginning
Use 9094, not 9092, from outside the cluster
Port 9092 advertises kafka:9092 in metadata, which is unresolvable outside the cluster. Always connect to SERVER_HOST:9094, and SERVER_HOST must be the server's real IP/hostname (not localhost) for remote clients.
If the port-forward dies, restart it:
kubectl port-forward --address 0.0.0.0 -n flink svc/kafka 9094:9094 &
Logstash ingest
Logstash runs three input listeners (port-forwarded on PF_ADDRESS, default 0.0.0.0). Each tags events with a type and writes them to a per-type Kafka topic logstash-<type>:
Input | Port (.env) | Protocol/codec | Kafka topic |
syslog |
| TCP/UDP, RFC3164 syslog |
|
json |
| TCP, JSON lines |
|
unstructured |
| TCP, raw text |
|
To test each input, send one event to its port (replace SERVER_HOST). Each command opens a TCP connection through bash's built-in /dev/tcp:
# syslog (RFC3164) on 10501
bash -c 'echo "<13>Feb 24 19:02:11 FILESERVER01 sshd[1234]: Accepted password for alice" > /dev/tcp/SERVER_HOST/10501'
# json on 10510
bash -c 'echo "{\"event_id\":4624,\"computer\":\"FILESERVER01\",\"message\":\"logon\"}" > /dev/tcp/SERVER_HOST/10510'
# unstructured text on 10520
bash -c 'echo "raw application log line without structure" > /dev/tcp/SERVER_HOST/10520'
Verify the events landed in Kafka:
kcat -C -b SERVER_HOST:9094 -t logstash-syslog -o beginning -e
kcat -C -b SERVER_HOST:9094 -t logstash-json -o beginning -e
kcat -C -b SERVER_HOST:9094 -t logstash-unstructured -o beginning -e
Logstash auto-creates the logstash-* topics on the first event (Kafka auto.create.topics.enable is on).
Useful commands after deploy
Inspect
kubectl get pods -n flink # all pods
kubectl get svc -n flink # services
kubectl logs -n flink -l app.kubernetes.io/name=admin-panel-be -f # backend logs
kubectl logs -n flink -l app.kubernetes.io/name=admin-panel-ui -f # UI logs
kubectl logs -n flink -l app=postgres -f # PostgreSQL
kubectl logs -n flink -l app=kafka -f # Kafka
kubectl logs -n flink -l app=kafka-ui -f # Kafka UI
kubectl logs -n flink -l app=logstash -f # Logstash
Restart port-forwards (if they die)
kubectl port-forward --address 0.0.0.0 -n flink svc/kafka 9094:9094 &
kubectl port-forward --address 0.0.0.0 -n flink svc/logstash 10501:10501 10510:10510 10520:10520 &
kubectl port-forward --address 0.0.0.0 -n kubernetes-dashboard svc/kubernetes-dashboard 8443:80 &
# UI / API / Kafka-UI -- TLS OFF: forward each service directly (plaintext)
kubectl port-forward --address 0.0.0.0 -n flink svc/admin-panel-be 8000:8000 &
kubectl port-forward --address 0.0.0.0 -n flink svc/admin-panel-ui 4173:4173 &
kubectl port-forward --address 0.0.0.0 -n flink svc/kafka-ui 8080:8080 &
# UI / API / Kafka-UI -- TLS ON: forward Traefik on the same ports instead (do NOT also run the three above)
kubectl port-forward --address 0.0.0.0 -n traefik svc/traefik 4173:4173 8000:8000 8080:8080 8443:8443 &
Stop/reset
minikube stop # stop the cluster, keep data
minikube delete && ./deploy-detectflow-minikube.sh # full reset and redeploy
Part 6 — Where to Go Next
Once you have a first detection, here is how to get more value out of the PoV. These map to the menus you see in the Prime Detect UI.
Connect real log sources. Point your syslog/JSON producers at the Logstash ports, or produce directly to Kafka topics. Manage them under Settings → Log Sources and Topics.
Curate rules. Review the Sigma content under Settings → Repositories, and use Filters to scope what each pipeline evaluates.
Build more pipelines. Create pipelines per log type or use case under Pipelines, overriding the Pipeline Runtime defaults where a pipeline needs more or less throughput.
Tune performance. Revisit Settings → Pipeline Runtime as volume grows — raise parallelism / TaskManager memory, adjust window and checkpoint intervals, or enable autoscaling.
Operate the platform. Use User Management and Audit Logs for access and accountability, the Dashboard for live detection visibility, and the Kafka UI / K8s Dashboard for infrastructure health.
Secure access. For anything beyond a quick LAN demo, enable TLS (Profile 2 or 3) so the UI/API/Kafka-UI/Dashboard are served over HTTPS.
Day-2 housekeeping
minikube stop preserves your data between sessions; minikube delete wipes everything. Remember that port-forwards end when their shell ends — restart them (Part 5 → Useful commands) or run them under a process manager for a longer-lived demo.
Need more detail or hit something not covered here?
The README included in the package tracks the same material in text form and is updated alongside the product. For PoV-specific questions, reach out to your SOC Prime contact.
