Skip to main content

Prime Detect Deployment Guide Minikube

N
Written by Nataliia Pukaliak

About this edition

This guide targets a local proof-of-value (PoV) deployment of Prime Detect on Ubuntu using Minikube. It is not a production / multi-node Kubernetes guide.

The deployment script and all configuration files in the provided package contain placeholder data (mock secrets) and are intended for demonstration purposes only.

How to Use This Guide

This guide describes how to deploy Prime Detect for a proof of value on a single Ubuntu machine using Minikube (a local, single-node Kubernetes cluster). Everything is driven by one script and one configuration file.

We learned from real client deployments that the same document has to serve two very different readers, so it is split into two tracks. Read the one that fits you — you do not need both.

Track

Who it is for

What you get

A — Fast Install

You are comfortable with Linux, Docker, and Kubernetes and just want the commands.

A one-page command sequence per scenario. Minimal explanation.

B — Guided Walkthrough

You are deploying this for the first time, or you want to be “led by the hand.”

Every step explained, what you will see after each one, time estimates, common mistakes, and the path all the way to your first detection.

What you are going to build

One Ubuntu host running a local Minikube cluster with the full Prime Detect stack: a web UI and REST API, a PostgreSQL database, Apache Kafka (with a Kafka UI), a Logstash log-ingestion layer, and an Apache Flink stream-processing engine that applies Sigma rules to incoming events. When everything is up, you log in to the UI, point a log source at it, and watch detections appear — sub-second, in real time.

End state — the finish line

All pods report Running (you will see a list like 1/1 Running for each component).

The script prints a summary with access URLs, login credentials, and port-forward PIDs.

You can open the Prime Detect UI in a browser, log in, send a test event, and see it flow through Kafka into a detection.

How long it takes

Times depend heavily on your hardware and your internet speed (the container images are pulled from Docker Hub). The numbers below are rough estimates for the recommended server.

Phase

What happens

Typical time

Preparation

Unpack the zip, edit .env, gather Docker Hub credentials

10–20 min

First script run

Installs Docker, adds you to the docker group, then stops (this is expected — see Step 3)

2–5 min

Re-login

Activate group membership and re-run the script

1 min

Second script run

Starts Minikube, pulls images, deploys all 15 components

15–25 min

First detection

Log in, tune runtime settings, send a test event, see a match

10–15 min

Read these three things before you start

SERVER_HOST accepts an IP address or a domain name (FQDN). Older templates phrased the placeholder as “replace with server IP,” but a hostname works just as well — and is required if you want warning-free HTTPS with your own certificate.

The image names in .env must match exactly what SOC Prime sent you. A single typo in IMAGE_UI, IMAGE_BACKEND or IMAGE_FLINK does not fail fast — the deploy hangs for many minutes trying to pull an image that does not exist. See Step 1 and Troubleshooting.

The very first script run stops on purpose after installing Docker. It is not broken. You log out / back in (or run newgrp docker) and run it once more. See Step 3.

Part 1 — Before You Begin

What Prime Detect is

Prime Detect is a real-time SIEM detection pipeline platform by SOC Prime. It applies Sigma rules to streaming events from Apache Kafka using Apache Flink, providing sub-second threat detection. Events arrive (via Logstash or directly into Kafka), Flink pipelines evaluate Sigma rules against them, and matches surface in the UI.

Tech stack

You do not install these by hand — the script does. The table is here so you recognize the names when you see them scroll past during deployment.

Dependency

Version

Purpose

Required by

Kubernetes

1.28+

Container orchestration

All components

PostgreSQL

14+

Backend database (rules, pipelines, users)

Backend

Apache Kafka

3.8+

Event streaming & rule distribution

Backend, Flink Sigma Detector

Apache Flink

2.2

Stream-processing engine

Flink Sigma Detector

Flink Kubernetes Operator

1.13.0

Manages Flink deployments as CRDs

Flink Sigma Detector lifecycle

cert-manager

1.17.2

TLS certificate management (Flink operator; app TLS in selfsigned mode)

Flink Operator, optional TLS

Traefik

chart (set in .env)

Per-port TLS termination for optional HTTPS (UI, API, Kafka UI, dashboard)

Optional TLS (TLS_ENABLED=true)

The deploy script also installs these command-line tools if they are missing (check-then-install):

Tool

Purpose

Installed from

docker

Container runtime + driver

Official apt repo (docker-ce, docker-compose-plugin)

minikube

Local Kubernetes cluster

Official binary → /usr/local/bin/minikube

kubectl

Kubernetes CLI

Official binary from dl.k8s.io

helm

Kubernetes package manager

Official get-helm-3 script

kcat

Kafka CLI producer/consumer

apt-get install kcat

envsubst

Render ${VAR} in manifests from .env

apt-get install gettext-base

Prerequisites checklist

Confirm each item before you touch the script:

  • A clean Ubuntu host — 22.04, 24.04 or 26.04.

  • Resources. Tuned for a 32 CPU / 64 GB RAM / 128 GB SSD server. Minimum ~16 CPU / 24 GB RAM / 64 GB disk. Less than this and pods will not fit (see Troubleshooting → out of resources).

  • Docker Hub credentials — a username and an access token that has permission to pull the private Prime Detect images. SOC Prime grants this access; without it nothing pulls.

  • The Prime Detect images. SOC Prime makes the private images available to your Docker Hub account. Confirm you can see them before deploying.

  • Network reachability. Decide how clients will reach this host — by IP or by hostname/FQDN. That value goes into SERVER_HOST.

Do I need to be root?

No — the recommended path is a normal user in the docker group. Running as root is supported but Minikube's docker driver then needs an extra flag (MINIKUBE_FORCE=true). Both paths are covered below.

Choose your deployment profile

There are three common ways to run the PoV. They differ only in who runs the script and whether/how TLS (HTTPS) is enabled. Pick one now; each track tells you exactly what changes for your choice.

Profile

Run as

Access by

TLS

Browser warning?

1 - Basic

normal user

IP address

Off (HTTP)

n/a

2 - Self-signed

root (MINIKUBE_FORCE=true)

IP address

On, selfsigned

Yes, until you import the CA

3 - Provided cert

normal user

FQDN

On, provided (your wildcard cert)

No (valid cert)

Not sure? Start with Profile 1.

Profile 1 (normal user, IP, plain HTTP) is the simplest and fastest. You can switch TLS on later by editing .env and re-running — no need to start over.

Part 2 — Track A: Fast Install

For readers who know Docker and Kubernetes. If any command surprises you, switch to Track B — Guided Walkthrough, which explains every step.

Two gotchas even fast installers hit

IMAGE_* values in .env must match exactly what SOC Prime provided, or the deploy hangs at the image-pull step (not an instant failure).

On a host where your user is not yet in the docker group, the first run installs Docker and exits. Re-login and run again.

Common steps (all profiles)

# 1) Get the code
# Unpack files provided in the detectflow-oss-private-ubuntu-based.zip, then cd into the folder

# 2) Configure
cp .env.example .env
# Edit at least: SERVER_HOST (IP or FQDN) and IMAGE_* (exact tags from SOC Prime)
chmod +x deploy-detectflow-minikube.sh

# 3) (Optional) provide Docker Hub creds up front to avoid the prompt
export DOCKERHUB_USER=youruser DOCKERHUB_TOKEN=dckr_pat_xxx

Profile 1 - normal user, IP, HTTP

./deploy-detectflow-minikube.sh
# First run installs Docker + adds you to the 'docker' group, then stops:
newgrp docker # or log out / back in
./deploy-detectflow-minikube.sh # run again; proceeds to full deploy

Access (default ports): UI http://SERVER_HOST:4173, API:8000, Kafka UI:8080, K8s Dashboard:8443, Kafka broker SERVER_HOST:9094, Logstash:10501/:10510/:10520.

Profile 2 - root, IP, self-signed TLS

Set in .env: TLS_ENABLED=true, TLS_MODE=selfsigned. Then:

MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh
# (root skips the docker-group dance; MINIKUBE_FORCE allows the docker driver as root)

UI/API/Kafka-UI/Dashboard now serve HTTPS on the same ports (https://SERVER_HOST:4173, etc.). Browsers warn until you import the CA:

kubectl get secret detectflow-ca-keypair -n cert-manager \
-o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crt
# import detectflow-ca.crt into your OS/browser trust store

Profile 3 — normal user, FQDN, provided wildcard cert

Put your PEM cert/key under certs/ (gitignored). Set in .env:

TLS_ENABLED="true"
TLS_MODE="provided"
TLS_CERT_FILE="/path/to/your/cert" # full chain (leaf + intermediates)
TLS_KEY_FILE="/path/to/your/key" # unencrypted PEM
SERVER_HOST="df-005.cloud.example.com" # a name your cert covers (wildcards: use a sub-host, not the apex)
./deploy-detectflow-minikube.sh
newgrp docker && ./deploy-detectflow-minikube.sh # if first run installed Docker

HTTPS on the FQDN with no browser warning (valid cert).

Verify (any profile)

kubectl get pods -n flink                 # all should be 1/1 Running
# send a quick test event into Logstash (syslog input) and read it back from Kafka:
bash -c 'echo "<13>Feb 24 19:02:11 HOST sshd[1]: test" > /dev/tcp/SERVER_HOST/10501'
kcat -C -b SERVER_HOST:9094 -t logstash-syslog -o beginning -e

Then open the UI, log in (default admin@soc.local / admin), tune Pipeline Runtime settings, and create your first pipeline — see Step 9: From deploy to first detection.

Part 3 — Track B: Guided Walkthrough

This track assumes no prior Kubernetes or Kafka experience. It walks the whole deployment, tells you what each command does, what you should see afterwards, how long things take, and what to do when something looks wrong. Follow the steps in order.

How the steps map to your profile

The walkthrough is written for Profile 1 (normal user, IP, HTTP) because it is the simplest. Where Profile 2 (self-signed TLS) or Profile 3 (FQDN + your cert) differ, a coloured box calls it out. Decide your profile in

Step 0 — Gather what you need

Before opening a terminal, have these ready. Missing one is the most common reason a deploy stalls later.

  • Access to the Ubuntu host (SSH or a local session) on 22.04 / 24.04 / 26.04.

  • Docker Hub username + access token with pull permission for the Prime Detect images (from SOC Prime).

  • The exact image names/tags SOC Prime gave you for IMAGE_BACKEND, IMAGE_UI, IMAGE_FLINK, IMAGE_AGENT, IMAGE_CORRELATION. Copy-paste them; do not retype.

  • Your SERVER_HOST value — the IP or hostname clients will use to reach this machine.

  • (Profile 3 only) Your TLS certificate and private key files (PEM).

What success looks like at the end of Step 0

You have a terminal on the host, your Docker Hub token in hand, and the five image names on a sticky note (or clipboard).

Step 1 — Get the code and create your .env

Unpack the files SOC Prime provided to you (detectflow-oss-private-ubuntu-based.zip) and cd into the resulting folder:

unzip detectflow-oss-private-ubuntu-based.zip
cd detectflow-oss-private-ubuntu-based

All settings live in a file called .env. Copy the template, then open it in an editor (nano .env is fine):

cp .env.example .env
nano .env

You only need to touch a few values. The two that matter most for a first run are SERVER_HOST and the IMAGE_* tags.

All five images live in the same Docker Hub repo (socprimecom/detectflow), tagged by component.

Component

Image

Admin Panel Backend

socprimecom/detectflow:admin-panel-be.v2.1.0

Admin Panel UI

socprimecom/detectflow:admin-panel-ui.v2.1.0

Flink Sigma Detector

socprimecom/detectflow:flink-sigma-detector.v2.1.0

DF Agent job

socprimecom/detectflow:df-agent.v2.1.0

DF Correlation job

socprimecom/detectflow:df-correlation.v2.1.0

Variable

What to set it to

SERVER_HOST

This server's IP address or hostname/FQDN. Used for the Kafka external listener and the printed URLs. For remote clients, it must not be localhost.

If you plan to use your own certificate (Profile 3), set this to a name your certificate covers.

IMAGE_BACKEND / IMAGE_UI / IMAGE_FLINK / IMAGE_AGENT / IMAGE_CORRELATION

The exact Docker Hub image tags SOC Prime sent you. Paste them — a typo here causes a long hang later (see the warning below). All five are required — the script fails fast if any is empty. IMAGE_AGENT (df-agent) and IMAGE_CORRELATION (df-correlation) are the job images the backend launches.

PF_ADDRESS

Port-forward bind address. 0.0.0.0 (default) makes services reachable from the network. Leave as-is for a shared host.

MINIKUBE_CPUS / MINIKUBE_MEMORY / MINIKUBE_DISK

Cluster sizing (default 24 / 49152 MB / 100g). Lower these if your host is smaller than the recommended spec.

PG_*, KAFKA_*

PostgreSQL and Kafka resources/tuning. Defaults are fine for a PoV.

Stopper #1 — SERVER_HOST is not IP-only

Some older templates wrote the placeholder as REPLACE_WITH_SERVER_IP, which reads as “IP only.” That is misleading. SERVER_HOST happily takes a domain / FQDN too (for example df-005.cloud.example.com). A hostname is in fact required for a warning-free certificate in Profile 3.

Stopper #2 — a wrong IMAGE_* name hangs the deploy

If IMAGE_UI (or any image tag) is mistyped, the deploy does not stop immediately. Kubernetes keeps retrying to pull an image that does not exist, so the script appears to “freeze” — in one client case for ~15 minutes around the image-pull step — before the pod lands in ImagePullBackOff.

Prevention: copy-paste the tags exactly. Detection: if a step seems stuck, open a second terminal and run kubectl get pods -n flink; a ImagePullBackOff or ErrImagePull status points straight at the bad image name. Fix it in .env and re-run.

Secrets and .env

.env is gitignored. Your Docker Hub credentials are never written to .env — you provide them at run time (Step 4). Keep certificate files out of git too (put them under certs/).

Step 2 — Set your TLS profile (optional)

TLS is off by default. If you are doing Profile 1, skip this step entirely — everything will serve plain HTTP. Come back and switch TLS on later if you want; you only edit .env and re-run.

When TLS is enabled, Traefik terminates TLS and the UI, API, Kafka UI and K8s dashboard serve HTTPS on the same port numbers (so http://…:4173 becomes https://…:4173). The Kafka broker and Logstash always stay plaintext.

Profile 2 — self-signed certificate

Set in .env:

TLS_ENABLED=true
TLS_MODE=selfsigned

cert-manager issues a certificate from a self-signed CA, using SERVER_HOST as the certificate name (SAN). Browsers will warn until you import that CA (covered in Step 7). Good for a demo on a LAN by IP.

Profile 3 — your own (wildcard) certificate

Use this when you have a valid certificate and want no browser warnings. Put the PEM files somewhere safe (for example under certs/) and set in .env:

# --- TLS (optional, per-port) ---
TLS_ENABLED="true"
TLS_MODE="provided" # your own cert, not self-signed
TRAEFIK_VERSION="34.1.0" # verify: helm search repo traefik/traefik --versions
TLS_CERT_FILE="/path/to/your/cert" # full chain (leaf + intermediates), PEM
TLS_KEY_FILE="/path/to/your/key" # unencrypted PEM key
  • SERVER_HOST must be a name the certificate covers. For a wildcard such as *.cloud.example.com, use a sub-host like df-005.cloud.example.com — wildcards do not cover the apex cloud.example.com.

  • TLS_CERT_FILE should be the full chain (leaf + intermediates). Check with grep -c 'BEGIN CERTIFICATE' cert.pem (1 = leaf only; append intermediates if clients report an incomplete chain).

  • TLS_KEY_FILE must be an unencrypted PEM key (head -1 key.pem must not say ENCRYPTED; decrypt with openssl rsa -in key.pem -out key.dec.pem).

  • The secret lives in the files, not in .env.

Full TLS reference

Switching modes, trusting the CA, and verifying what is served on the wire are all in Part 5 → HTTPS / TLS.

Step 3 — First run: Docker installs, then the script stops

Make the script executable and run it as your normal user:

chmod +x deploy-detectflow-minikube.sh
./deploy-detectflow-minikube.sh

If Docker was not already installed (or your user was not yet in the docker group), the script installs Docker, adds you to the group, and stops on purpose. You will see something like:

[✓] Docker Engine installed
[i] Adding 'ansuser' to the 'docker' group...
[!] You were added to the 'docker' group. Log out/in (or run 'newgrp docker')
[!] and re-run this script so the group membership takes effect.

Stopper #3 — this is NOT a failure

Linux only applies a new group at the start of a fresh login session, so your current shell still cannot talk to the Docker socket. The script does the safe thing: it stops and asks you to refresh your session. Nothing is broken.

If you ignore this and just re-run, you will get Cannot talk to Docker daemon — same cause, same fix.

Activate the group membership, then run the script a second time:

newgrp docker                      # OR fully log out and back in
./deploy-detectflow-minikube.sh # run again — this time it proceeds

What you should see now

On the second run, the script confirms Docker is reachable and moves on to ask for your Docker Hub credentials (Step 4). You will see lines like [✓] Docker is installed and reachable.

Profile 2 (root) skips all of this

If you run the script as root, there is no docker-group step. But Minikube's docker driver refuses to run as root, so you start it with the force flag instead: MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh.

Step 4 — Provide your Docker Hub credentials

The Prime Detect images are private, so the script needs a Docker Hub login to pull them. The credentials are temporary — they are never written to .env. You have two options.

Option A — export before running (no prompt)

export DOCKERHUB_USER=youruser DOCKERHUB_TOKEN=dckr_pat_xxx
./deploy-detectflow-minikube.sh

Option B — let the script prompt you

Just run the script; it asks interactively (the token is hidden as you type):

Please enter your Docker Hub username: ...
Please enter your Docker Hub access token ...
Login Succeeded
[✓] Docker Hub login successful

What you should see now

After a successful login, the script begins the actual installation. From here on it asks no more questions — you can let it run.

Pulls fail with a 401 / no images?

Your token lacks pull permission for the Prime Detect images, or the credentials were mistyped. See Troubleshooting → Image pull errors.

Step 5 — Watch the installation

The script now deploys the full stack as a numbered sequence (you will see headers like ═══ Step 2/15: Starting Minikube Cluster ═══). It installs any missing tools, starts Minikube, then installs each component in order. All Kubernetes manifests live in manifests/ next to the script; ${VAR} placeholders are filled from your .env at apply time. Nothing is fetched from external git repos, and no images are built locally — they are pulled from Docker Hub.

Roughly what happens, and what it is waiting for:

  • Tools — docker, minikube, kubectl, helm, kcat, envsubst (skipped if already present).

  • Minikube cluster starts with the CPU/RAM/disk from .env (1–3 min).

  • cert-manager comes up (its 3 pods must be Running before the Flink operator installs).

  • Flink Kubernetes Operator, PostgreSQL, Kafka + Kafka UI, Logstash, then the Prime Detect backend and UI.

  • Port-forwards are created last so you can reach everything from your machine.

What “good” looks like while it runs

Pods appear and move through PendingContainerCreatingRunning. A pod sitting in Pending for a minute or two while images download is normal.

Expect the whole second run to take roughly 15–25 minutes on the recommended server — mostly image-download time. Slower hardware or a slow connection takes longer.

If a step seems frozen for many minutes

Open a second terminal and look at the pods: kubectl get pods -n flink. A status of ImagePullBackOff / ErrImagePull means a bad or unreachable image name (re-check your IMAGE_* from Step 1). Pending that never clears usually means the host is out of CPU/RAM/disk (see Troubleshooting).

First run ended after only Docker installed (0/15)?

That is the expected Step 3 behaviour, not a crash. Re-login (newgrp docker) and run the script again — the second run does the real 15-step deploy.

Step 6 — Read the final summary

When the deploy finishes, the script prints a summary block: access URLs (with your SERVER_HOST filled in), the default login, the pod list, and the port-forward PIDs. For Profile 1 it looks like this:

DetectFlow Demo - Local Deployment  
==================================

All components deployed successfully!

Access URLs (SERVER_HOST=10.11.14.59):
UI: http://10.11.14.59:4173
Backend API: http://10.11.14.59:8000
Kafka UI: http://10.11.14.59:8080
Kafka Broker: 10.11.14.59:9094 (external listener)
K8s Dashboard: http://10.11.14.59:8443
Logstash: 10.11.14.59:10501 syslog, 10510 json, 10520 unstructured

Login Credentials:
Email: admin@soc.local
Password: admin

Pod Status:
NAME READY STATUS RESTARTS AGE
admin-panel-be-7d6bb8cd48-t6spc 1/1 Running 0 82s
admin-panel-ui-7c78d784b4-mvq8z 1/1 Running 0 39s
kafka-868f945d5-bgfh4 1/1 Running 0 3m59s
kafka-ui-7966478868-dsnnd 1/1 Running 0 2m42s
logstash-b54784f8d-sjcmj 1/1 Running 0 3m2s
postgres-7f79948ccd-xpqpc 1/1 Running 0 4m35s

The 10.11.14.59 above is only an example — yours shows whatever you set as SERVER_HOST. Every pod should read 1/1 Running.

With TLS on (Profile 2 / 3), the URLs say https

The summary prints https://… on the UI/API/Kafka-UI/Dashboard ports, plus a TLS line. For self-signed, it adds: “Self-signed CA: browsers warn until you import it.” For provided certs it simply says “TLS enabled (provided).” The Kafka broker and Logstash lines stay plaintext in all cases.

The summary also lists port-forward PIDs, for example:

Port-Forward PIDs:
Backend: 16906
UI: 16907
Kafka UI: 16908
Kafka: 16909
Logstash: 16910
Dashboard: 10581
# (TLS on: a single 'Traefik' PID carries UI/API/Kafka-UI/Dashboard instead)

Those forwards are what make the services reachable from your browser. If they die (for example when your SSH session ends), restart them — see Part 5 → Useful commands. To stop them all at once: kill <PID> <PID> ... using the PIDs from your own run.

Step 7 — Open the UI and log in

In a browser, go to the UI URL exactly as printed in your summary, then sign in with the default credentials:

Field

Value

Email

admin@soc.local

Password

admin

Stopper #4 — use the right HTTP vs HTTPS

The single most common login problem is the wrong scheme. If you deployed without TLS (Profile 1), the UI is at http://SERVER_HOST:4173 — typing https:// will fail to connect. If you deployed with TLS (Profile 2/3), it is https://SERVER_HOST:4173 — plain http:// will not work. Always use the URL from your summary (Step 6).

Profile 2: the browser warns about the certificate

With a self-signed certificate, the browser shows a warning until you import the CA. That is expected. To remove the warning, export the CA once and import it into your OS/browser trust store:

Run: kubectl get secret detectflow-ca-keypair -n cert-manager -o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crt, then import detectflow-ca.crt.

What you should see now

The Prime Detect UI loads with a top navigation (Dashboard, Pipelines, Settings) and, under Settings, a left menu (Repositories, Topics, Log Sources, Filters, User Management, Audit Logs, Pipeline Runtime).

Step 8 — Set Pipeline Runtime settings (before your first pipeline)

After the stack is deployed and you log in to the Prime Detect UI, review the Pipeline Runtime settings and tune them to your system resources before you create or run a pipeline.

Settings to review:

  • Parallelism — how many parallel subtasks Flink uses.

  • TaskManager CPU (cores) and TaskManager Memory (MB) — resources per Flink TaskManager.

  • Window Size (sec) — the time window for stateful rule evaluation.

  • Checkpoint Interval (sec) — how often Flink checkpoints its state.

  • Enable Autoscaling — automatically adjust parallelism based on workload.

These values significantly affect performance and resource usage. Adjust them as needed for your environment.

Refer to the screenshot below for an example configuration:

DetectFlow Pipeline Runtime settings screen

Settings → Pipeline Runtime, with example values for a PoV host.

Sensible starting point on the recommended server

Parallelism 2, TaskManager 2 cores / 8192 MB, Window 30 s, Checkpoint 60 s, Autoscaling off. Raise parallelism and memory only if you push high event volumes. Save the settings before moving on.

Step 9 — From deploy to your first detection

Why this section exists

A clean deploy and an open dashboard left earlier readers asking “now what?” This step closes the loop: get one event in, and see one detection out.

9.1 — Send a test event

The fastest way to prove the pipeline end-to-end is to push a sample event into Logstash. The syslog input listens on port 10501 and writes to the Kafka topic logstash-syslog. Replace SERVER_HOST with yours:

# syslog (RFC3164) on 10501
bash -c 'echo "<13>Feb 24 19:02:11 FILESERVER01 sshd[1234]: Accepted password for alice" > /dev/tcp/SERVER_HOST/10501'

Prefer raw Kafka? Produce a structured event straight onto a topic with kcat (see Part 5 for the full example):

echo '{"event_id":4624,"computer":"FILESERVER01","message":"An account was successfully logged on."}' \
| kcat -P -b SERVER_HOST:9094 -t windows-events

9.2 — Confirm it reached Kafka

Read the topic back. If your event prints, ingestion works:

kcat -C -b SERVER_HOST:9094 -t logstash-syslog -o beginning -e

You can also watch this visually in the Kafka UI (SERVER_HOST:8080): open the topic and confirm the message count went up. Logstash auto-creates the logstash-* topics on the first event.

9.3 — Create a pipeline and see the match

In the Prime Detect UI, a pipeline is what applies Sigma rules to a topic and produces detections. At a high level:

  1. Confirm a rule repository is available under Settings → Repositories (the Sigma rules your pipeline will evaluate).

  2. Make sure the topic you sent events to (for example logstash-syslog or windows-events) is visible under Settings → Topics / Log Sources.

  3. Create a Pipeline that reads from that topic and applies the rule set. Use the Pipeline Runtime defaults you set in Step 8 (you can override per pipeline).

  4. Send events that should trigger a rule (re-run the test event, or feed real logs), and watch the pipeline produce detections.

  5. Open the Dashboard / the pipeline's output to view matches as they arrive.

Exact clicks vary by build and loaded content

The available rules, repositories, and log-source connectors depend on what your environment was provisioned with, so the in-product wording may differ slightly from the outline above. The test event → Kafka topic part (9.1–9.2) is the reliable, verifiable signal that the platform is ingesting; from there the UI guides pipeline creation. If a rule does not match, check that the event shape matches the rule's expected fields.

What success looks like

Your test event appears in its Kafka topic, a pipeline is running against that topic, and at least one detection shows up in the UI. That is your first detection — the PoV is live.

From here, see Part 6 — Where to Go Next for tuning, scaling, and day-2 operations.

Part 4 — Troubleshooting

Each entry is written as: what you see, what it means, and how to fix it. Start with the common-mistakes box — it covers most first-deploy surprises.

The five most common first-deploy surprises

“It installed Docker and quit.” Expected. Re-login (newgrp docker) and run again — the second run does the full deploy.

“It froze for ~15 minutes.” Usually a mistyped IMAGE_* in .env (pod stuck in ImagePullBackOff). Fix the tag, re-run.

“Can't reach the UI.” Wrong scheme — use http:// for no-TLS, https:// for TLS. Use the URL from the summary.

“Browser security warning.” Self-signed cert (Profile 2). Import the CA, or proceed past the warning for a demo.

“Everything died after I closed SSH.” The port-forwards ended with the session. Restart them (Part 5 → Useful commands).

Deployment/cluster

First run stops after installing Docker

You see: You were added to the 'docker' group… re-run this script. Meaning: expected behaviour — the new group is not active in the current shell. Fix:

newgrp docker            # or log out / back in
./deploy-detectflow-minikube.sh

Cannot connect to the Docker daemon

You see: Cannot talk to Docker daemon. Is it running? Meaning: you were added to the docker group, but the shell predates it (often after skipping the re-login above). Fix: log out/in or newgrp docker, then re-run. Verify with docker info.

Minikube: “docker driver should not be used with root privileges” (DRV_AS_ROOT)

You see: the deploy refuses to start Minikube as root. Meaning: Minikube's docker driver blocks root. Fix — either run as a normal user in the docker group:

sudo usermod -aG docker $USER   # then log out/in (or: newgrp docker)
./deploy-detectflow-minikube.sh # no sudo

…or force it (root + docker driver):

MINIKUBE_FORCE=true ./deploy-detectflow-minikube.sh

Minikube out of resources

You see: pods stuck in Pending, scheduling/eviction messages, or Minikube failing to start. Meaning: the host does not have enough CPU/RAM/disk for the requested cluster. Fix: lower MINIKUBE_CPUS / MINIKUBE_MEMORY in .env to fit the host, then reset and redeploy:

minikube delete
./deploy-detectflow-minikube.sh

Pods and components

Pods stuck in Pending / CrashLoopBackOff

You see: a pod that never reaches Running. Fix: inspect it — the events and logs almost always say why (no resources, bad image, failed dependency).

kubectl describe pod <pod-name> -n flink
kubectl logs <pod-name> -n flink

Image pull errors (ImagePullBackOff / ErrImagePull)

You see: a pod in ImagePullBackOff, or a long hang during the image-pull step. Meaning: a wrong image name/tag (Stopper #2) or invalid/insufficient Docker Hub credentials. Fix:

  • Re-check the IMAGE_* values in .env against exactly what SOC Prime sent (a typo is the usual cause).

  • Confirm valid Docker Hub credentials were provided at the start of the script.

  • Verify the pull secret exists: kubectl get secret dockerhub-creds -n flink.

  • Confirm your token has pull permission for the Prime Detect images.

To re-create the pull secret, delete it and re-run:

kubectl delete secret dockerhub-creds -n flink
./deploy-detectflow-minikube.sh

Backend can't connect to PostgreSQL

  • Ensure the PostgreSQL pod is running: kubectl get pods -n flink -l app=postgres.

  • Check the DATABASE_URL format: postgresql+asyncpg://detectflow:detectflow@postgres:5432/detectflow.

Backend can't connect to Kafka

  • Ensure the Kafka pod is running: kubectl get pods -n flink -l app=kafka.

  • Give Kafka 30–60 seconds to fully start before the backend connects.

  • The in-cluster bootstrap server is kafka:9092 (not the external:9094).

Flink Operator not starting

kubectl get pods -n flink-operator
kubectl logs -n flink-operator -l app.kubernetes.io/name=flink-kubernetes-operator

cert-manager issues

Wait for all three cert-manager pods to be Running before the Flink operator installs:

kubectl get pods -n cert-manager

HTTPS / TLS (only when TLS_ENABLED=true)

Can't reach the UI at all

Check the scheme first (Stopper #4): http:// for no-TLS, https:// for TLS. Then confirm the port-forward is alive (next item).

Connection refused on the HTTPS port

Meaning: the Traefik port-forward is not running (it can die when the deploy's shell/SSH session ends). Fix:

kubectl port-forward --address 0.0.0.0 -n traefik svc/traefik 4173:4173 8000:8000 8080:8080 8443:8443 &

Browser shows “TRAEFIK DEFAULT CERT” instead of your cert

Meaning: Traefik is serving its built-in default. Fix: confirm the default TLSStore points at df-tls, and the secret holds the right cert, then reload Traefik:

kubectl get tlsstore default -n traefik -o jsonpath='{.spec.defaultCertificate.secretName}'; echo   # df-tls
kubectl get secret df-tls -n traefik -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -issuer -ext subjectAltName
kubectl rollout restart deployment/traefik -n traefik

Provided cert keeps reverting to the self-signed one

Meaning: a leftover cert-manager Certificate is still reconciling the secret. The deploy deletes it automatically; to do it by hand:

kubectl delete certificate detectflow-tls -n traefik --ignore-not-found
kubectl delete certificate detectflow-tls -n flink --ignore-not-found

Certificate name-mismatch warning

Meaning: the certificate's SAN does not cover the name you are using. Fix: confirm SERVER_HOST matches the cert (openssl … -ext subjectAltName); for a wildcard use a sub-host, not the apex.

“TLSStore … spec.defaultCertificate.secretName: Required value” during install

Meaning: TLS_SECRET_NAME was not rendered into the Traefik values. Fix: make sure you are running the current script (it exports the variable for envsubst).

Part 5 — Reference

Detailed material for both tracks: access URLs, the full TLS story, Kafka and Logstash access, and the commands you will reach for after deploy.

Access URLs & credentials

After deployment, the script sets up port-forwards (bound to PF_ADDRESS, default 0.0.0.0) and prints the URLs. Replace SERVER_HOST with the value from your .env (remote clients use the server IP/hostname; locally, localhost works).

Service

URL (default ports)

Notes

Prime Detect UI

http://SERVER_HOST:4173

Main admin panel

Prime Detect Backend API

http://SERVER_HOST:8000

REST API

Kafka UI

http://SERVER_HOST:8080

Kafka monitoring & management

Kafka Broker

SERVER_HOST:9094

External listener for remote access

Logstash Ingest

SERVER_HOST:10501 (syslog), :10510 (json), :10520 (text)

Log ingestion → Kafka

K8s Dashboard

http://SERVER_HOST:8443

Kubernetes cluster monitoring

Ports and scheme

Ports are configurable in .env (UI_PORT, API_PORT, KAFKA_UI_PORT, KAFKA_EXTERNAL_PORT, DASHBOARD_PORT, LOGSTASH_SYSLOG_PORT, LOGSTASH_JSON_PORT, LOGSTASH_TEXT_PORT).

The UI / API / Kafka UI / dashboard use http by default. With TLS enabled, they serve https on the same port numbers — see HTTPS / TLS below.

Default login:

Field

Value

Email

admin@soc.local

Password

admin

HTTPS / TLS (optional, per-port)

TLS is off by default. When enabled, Traefik terminates TLS so the UI, API, Kafka UI and K8s dashboard serve HTTPS on their existing ports (http://…:4173 becomes https://…:4173). The Kafka broker and Logstash stay plaintext. Set in .env:

TLS_ENABLED=true
TLS_MODE=selfsigned # or: provided
# provided mode only:
TLS_CERT_FILE=/path/to/cert.pem
TLS_KEY_FILE=/path/to/key.pem
  • selfsigned — cert-manager issues a cert from a self-signed CA, with SERVER_HOST as the SAN (IP or hostname). Browsers warn until you import the CA. Good for demo / LAN.

  • provided — bring your own cert. Point TLS_CERT_FILE / TLS_KEY_FILE at your PEM files and set SERVER_HOST to a name the cert covers (for a wildcard *.example.com, use a sub-host like df.example.com — wildcards don't cover the apex) for valid, warning-free SSL.

  • TLS_ENABLED=false (default) → all ports stay plain HTTP, exactly as before.

How it works

Traefik is installed via Helm with one TLS entrypoint per service. A single default certificate (df-tls, set as a TLSStore named default in the traefik namespace) is served on every entrypoint — routes match on path only (no Host), so the default cert is what gets served. The deploy port-forwards Traefik on the UI/API/Kafka-UI/dashboard ports.

Provided-cert requirements

  • TLS_CERT_FILE should be the full chain (leaf + intermediates). Check with grep -c 'BEGIN CERTIFICATE' cert.pem (1 = leaf only; append intermediates if clients report an incomplete chain).

  • TLS_KEY_FILE must be an unencrypted PEM key (head -1 key.pem must not say ENCRYPTED; decrypt with openssl rsa -in key.pem -out key.dec.pem).

  • The cert must cover SERVER_HOST. A wildcard *.cloud.socprime.com covers srv-005.cloud.socprime.com (one label) but not the apex cloud.socprime.com.

  • Keep cert/key out of git — put them under certs/ (gitignored).

Trusting the self-signed CA

Export the CA once and import it into your OS/browser trust store to remove the warning:

kubectl get secret detectflow-ca-keypair -n cert-manager -o jsonpath='{.data.tls\.crt}' | base64 -d > detectflow-ca.crt

Verify what's served on the wire

echo | openssl s_client -connect SERVER_HOST:4173 -servername SERVER_HOST 2>/dev/null \
| openssl x509 -noout -issuer -ext subjectAltName

Switching selfsigned ↔ provided

Change TLS_MODE (and the cert paths) in .env and re-run. App pods are not restarted; only the df-tls secret swaps. If the browser still shows the old cert, force a reload (restarts Traefik only, ~5s):

kubectl rollout restart deployment/traefik -n traefik

Kafka broker access

The deploy configures Kafka with two listeners:

Listener

Address

Purpose

PLAINTEXT

kafka:9092

In-cluster clients (Backend, Flink, MatchNode)

EXTERNAL

SERVER_HOST:9094

Remote / host access via kubectl port-forward

The EXTERNAL listener advertises SERVER_HOST:9094 (from .env) and is port-forwarded on 0.0.0.0, so clients on other machines can produce/consume. Example (replace SERVER_HOST with the server IP/hostname):

# Produce events using kcat
cat <<'JSON' | jq -c . | kcat -P -b SERVER_HOST:9094 -t windows-events
{
"event_id": 4624,
"timestamp": "2024-02-24T19:02:11Z",
"channel": "Security",
"computer": "FILESERVER01.domain.local",
"target_user": "DOMAIN\\alice",
"message": "An account was successfully logged on."
}
JSON

# Consume events
kcat -C -b SERVER_HOST:9094 -t windows-events -o beginning

Use 9094, not 9092, from outside the cluster

Port 9092 advertises kafka:9092 in metadata, which is unresolvable outside the cluster. Always connect to SERVER_HOST:9094, and SERVER_HOST must be the server's real IP/hostname (not localhost) for remote clients.

If the port-forward dies, restart it:

kubectl port-forward --address 0.0.0.0 -n flink svc/kafka 9094:9094 & 

Logstash ingest

Logstash runs three input listeners (port-forwarded on PF_ADDRESS, default 0.0.0.0). Each tags events with a type and writes them to a per-type Kafka topic logstash-<type>:

Input

Port (.env)

Protocol/codec

Kafka topic

syslog

10501 (LOGSTASH_SYSLOG_PORT)

TCP/UDP, RFC3164 syslog

logstash-syslog

json

10510 (LOGSTASH_JSON_PORT)

TCP, JSON lines

logstash-json

unstructured

10520 (LOGSTASH_TEXT_PORT)

TCP, raw text

logstash-unstructured

To test each input, send one event to its port (replace SERVER_HOST). Each command opens a TCP connection through bash's built-in /dev/tcp:

# syslog (RFC3164) on 10501
bash -c 'echo "<13>Feb 24 19:02:11 FILESERVER01 sshd[1234]: Accepted password for alice" > /dev/tcp/SERVER_HOST/10501'

# json on 10510
bash -c 'echo "{\"event_id\":4624,\"computer\":\"FILESERVER01\",\"message\":\"logon\"}" > /dev/tcp/SERVER_HOST/10510'

# unstructured text on 10520
bash -c 'echo "raw application log line without structure" > /dev/tcp/SERVER_HOST/10520'

Verify the events landed in Kafka:

kcat -C -b SERVER_HOST:9094 -t logstash-syslog        -o beginning -e
kcat -C -b SERVER_HOST:9094 -t logstash-json -o beginning -e
kcat -C -b SERVER_HOST:9094 -t logstash-unstructured -o beginning -e

Logstash auto-creates the logstash-* topics on the first event (Kafka auto.create.topics.enable is on).

Useful commands after deploy

Inspect

kubectl get pods -n flink                                   # all pods
kubectl get svc -n flink # services
kubectl logs -n flink -l app.kubernetes.io/name=admin-panel-be -f # backend logs
kubectl logs -n flink -l app.kubernetes.io/name=admin-panel-ui -f # UI logs
kubectl logs -n flink -l app=postgres -f # PostgreSQL
kubectl logs -n flink -l app=kafka -f # Kafka
kubectl logs -n flink -l app=kafka-ui -f # Kafka UI
kubectl logs -n flink -l app=logstash -f # Logstash

Restart port-forwards (if they die)

kubectl port-forward --address 0.0.0.0 -n flink svc/kafka 9094:9094 &
kubectl port-forward --address 0.0.0.0 -n flink svc/logstash 10501:10501 10510:10510 10520:10520 &
kubectl port-forward --address 0.0.0.0 -n kubernetes-dashboard svc/kubernetes-dashboard 8443:80 &

# UI / API / Kafka-UI -- TLS OFF: forward each service directly (plaintext)
kubectl port-forward --address 0.0.0.0 -n flink svc/admin-panel-be 8000:8000 &
kubectl port-forward --address 0.0.0.0 -n flink svc/admin-panel-ui 4173:4173 &
kubectl port-forward --address 0.0.0.0 -n flink svc/kafka-ui 8080:8080 &

# UI / API / Kafka-UI -- TLS ON: forward Traefik on the same ports instead (do NOT also run the three above)
kubectl port-forward --address 0.0.0.0 -n traefik svc/traefik 4173:4173 8000:8000 8080:8080 8443:8443 &

Stop/reset

minikube stop                                  # stop the cluster, keep data
minikube delete && ./deploy-detectflow-minikube.sh # full reset and redeploy

Part 6 — Where to Go Next

Once you have a first detection, here is how to get more value out of the PoV. These map to the menus you see in the Prime Detect UI.

  • Connect real log sources. Point your syslog/JSON producers at the Logstash ports, or produce directly to Kafka topics. Manage them under Settings → Log Sources and Topics.

  • Curate rules. Review the Sigma content under Settings → Repositories, and use Filters to scope what each pipeline evaluates.

  • Build more pipelines. Create pipelines per log type or use case under Pipelines, overriding the Pipeline Runtime defaults where a pipeline needs more or less throughput.

  • Tune performance. Revisit Settings → Pipeline Runtime as volume grows — raise parallelism / TaskManager memory, adjust window and checkpoint intervals, or enable autoscaling.

  • Operate the platform. Use User Management and Audit Logs for access and accountability, the Dashboard for live detection visibility, and the Kafka UI / K8s Dashboard for infrastructure health.

  • Secure access. For anything beyond a quick LAN demo, enable TLS (Profile 2 or 3) so the UI/API/Kafka-UI/Dashboard are served over HTTPS.

Day-2 housekeeping

minikube stop preserves your data between sessions; minikube delete wipes everything. Remember that port-forwards end when their shell ends — restart them (Part 5 → Useful commands) or run them under a process manager for a longer-lived demo.

Need more detail or hit something not covered here?

The README included in the package tracks the same material in text form and is updated alongside the product. For PoV-specific questions, reach out to your SOC Prime contact.

Did this answer your question?