Installation

Yagra is two long-running binaries plus a static WebUI: Yagra-core (orchestration, scheduling, the northbound REST API), Yagra-poller (stateless ICMP/SNMP/API workers), and Yagra-web (the WebUI, served by nginx, which reverse-proxies /api to core). This page covers every supported way to install that stack, from a one-command evaluation box to remote-site pollers streaming results home over a TLS bus.

Behind them sit up to six backing services:

Backing service	Role	Needed by
PostgreSQL	Metadata: nodes, configuration, thresholds, users, alert history	core (required)
NATS (JetStream)	The core⇄poller bus: jobs, working sets, results, events	core + poller (required)
VictoriaMetrics	Time-series store — the metrics body	core (required)
Redis	Ephemeral poller liveness/assignment mirror — rebuildable	core (optional)
VictoriaLogs	Passive-event log store — full-text syslog/trap search	core (optional)
ClickHouse	Traffic-flow store — NetFlow/IPFIX/sFlow records	core (optional)

Pollers talk to NATS only — device credentials, job specs, and results all flow over the bus, and a poller never touches the databases. That is what makes pollers stateless, horizontally scalable, and deployable behind a remote site’s NAT.

Choosing a deployment

Two axes: single-node vs. distributed polling, and Docker vs. native processes.

	Docker Compose	Native (no Docker)
Single node	A — build from source · B — pre-built images	C
Distributed pollers	D	E

New to Yagra? Start with A (one command, builds locally) or B (production-shaped, pulls pre-built images). Reach for D or E once you need pollers at remote sites — they connect out to the central bus, so they cross NAT and firewalls without any inbound port at the site.

Scaling out is a config change, not a rewrite: the single-node and distributed deployments run the same images. You add remote pollers, and — because job messages carry device credentials — you turn on NATS TLS + auth before the bus leaves the machine.

High availability is orthogonal to this matrix: any deployment with more than one core against the same stores can enable leader election (YAGRA_ENABLE_HA) for automatic core failover. docker-compose.ha.yml in the repository is a ready-made two-core overlay for trying it:

docker compose -f docker-compose.yml -f docker-compose.ha.yml up --build

Exactly one core (the leader) answers 200 on /readyz; the standby answers 503 until it takes over, within seconds of the leader stopping. Route API and WebUI traffic on /readyz. Details: high availability.

Images & tags

Three images are published to GitHub Container Registry:

ghcr.io/horryworks/yagra-core
ghcr.io/horryworks/yagra-poller
ghcr.io/horryworks/yagra-web

Only releases are published. Development builds never reach the registry, so every tag below is a release you can run.

Tag	Meaning	Use it for
:v<version>	One release (e.g. :v0.1.18)	Production — pin these
:latest	The latest stable release. Pre-releases (-beta, -rc) never move it	Evaluation, staying current
:<git-sha>	Immutable reference to one release	Reproducible deploys, rollback

The Compose files select the tag via YAGRA_IMAGE_TAG (default latest). Upgrading means changing the tag and recreating — see Upgrades & backups. Rollback is the same operation with an older tag.

Note

:latest follows the most recently published stable release, not the highest version number — a hotfix cut after a larger release moves it back. Pin a v<version> tag from the releases page for anything you keep.

A — Single node, Docker (build from source)

The developer / evaluation box. docker-compose.yml builds the images locally and runs the whole stack — core, a poller, the WebUI, and all backing stores — on one host:

git clone https://github.com/horryworks/Yagra.git
cd Yagra
docker compose up --build

Then open the WebUI at https://localhost:8443 and sign in with the one-time admin password printed in the core logs. The certificate is self-signed on a first start, so your browser will warn — see WebUI TLS. The quickstart walks through this path step by step, including first login.

Tip

This composition is fine for evaluation and development. For anything you care about, use B — pinned release images plus a persistent key-encryption key, so stored credentials survive restarts and upgrades.

B — Single node, Docker (pre-built images)

The production-shaped single-node deployment. docker-compose.deploy.yml pulls images from GHCR (no local build), is parameterized via .env, and adds a one-shot kek-init service that writes a persistent key-encryption key so stored monitoring credentials survive redeploys.

1. Clone the repository and create your .env:

   git clone https://github.com/horryworks/Yagra.git
   cd Yagra
   cp .env.example .env

2. Edit .env. The essentials:

   YAGRA_IMAGE_TAG=v0.1.22                # pin a stable release (see the releases page)
   POSTGRES_PASSWORD=change-me            # change for any non-throwaway box
   YAGRA_API_PORT=8080                    # host port for the API (plaintext)
   YAGRA_WEB_PORT=443                     # host port for the WebUI (HTTPS)
   # YAGRA_ADMIN_PASSWORD=choose-a-strong-password   # else a one-time random one is logged
   # YAGRA_PUBLIC_DASHBOARD=false         # true = read-only dashboards without login

   Everything else has a working default. Full variable list: the configuration reference.

3. Pull and start:

   docker compose -f docker-compose.deploy.yml pull
   docker compose -f docker-compose.deploy.yml up -d

4. Retrieve the one-time admin password (unless you set YAGRA_ADMIN_PASSWORD):

   docker compose -f docker-compose.deploy.yml logs core | grep -i password

   Open the WebUI at https://localhost, sign in as admin, and change the password. Your browser will warn about the self-signed certificate — see WebUI TLS for how to replace it.

What’s running. Core, one poller, and the WebUI, plus PostgreSQL, Redis, NATS, VictoriaMetrics, VictoriaLogs (passive-event search), ClickHouse (traffic flows), and an ipasn-updater sidecar that keeps the offline IP→ASN dataset fresh — the only container with internet egress. Migrations run automatically on core startup; named volumes preserve all persistent data across down/up.

Purpose	Host default	Change via
WebUI (HTTPS)	443	YAGRA_WEB_PORT
REST API + /metrics (plaintext)	8080	YAGRA_API_PORT
syslog intake (UDP)	514	YAGRA_SYSLOG_PORT
SNMP trap intake (UDP)	162	YAGRA_TRAP_PORT
NetFlow v5/v9 / IPFIX intake (UDP)	2055	YAGRA_FLOW_PORT
sFlow intake (UDP; listener off until YAGRA_SFLOW_BIND is set)	6343	YAGRA_SFLOW_PORT

The stores stay on the internal Docker network. Optional features switch off by setting their variable empty in .env — e.g. YAGRA_CLICKHOUSE_URL= disables flow monitoring, YAGRA_SYSLOG_BIND= disables syslog intake. Full matrix: ports · configuration.

Point your devices at it. Send syslog to the host’s :514/udp, SNMP traps (v1/v2c, informs included) to :162/udp, and NetFlow/IPFIX export to :2055/udp. Passive events correlate to nodes by the datagram’s source IP — if Docker’s bridge networking rewrites source addresses on your host, switch the poller service to network_mode: host so the real address survives.

Verify. All containers up, and core answering:

docker compose -f docker-compose.deploy.yml ps
curl -fsS http://localhost:8080/healthz

Settings ▸ System Health in the WebUI then shows each backing store’s reachability and the server’s own overall verdict.

Credential persistence — the KEK. Stored monitoring credentials (SNMP communities, SNMPv3 credentials, API tokens) are envelope-encrypted with a master key (KEK). The kek-init service generates a 32-byte KEK into the kekdata volume once and never overwrites it; core mounts it read-only. Without a persistent KEK, core falls back to an ephemeral key regenerated on every restart — and every stored credential becomes undecryptable after a redeploy.

Back up the KEK

Never delete the kekdata volume, and keep a copy of the key off the host:

docker run --rm -v yagra_kekdata:/kek busybox cat /kek/key > kek-backup.key

If the KEK is lost, every stored credential is permanently undecryptable — no backup of the database can bring them back.

WebUI TLS

The WebUI is HTTPS out of the box — there is no plain-HTTP listener to publish by accident. On a first start core generates a self-signed certificate covering loopback and the container’s hostname, so your browser will warn, and will usually object to the name as well: nothing inside the container can know the address you will type.

Fix it from Settings ▸ TLS, which is reachable through the warning. You can either import a PEM certificate chain and private key (pasted or from a file), or regenerate the self-signed one with the hostnames and IP addresses you actually use. An imported certificate is live within seconds with nothing restarted, and Yagra refuses a mismatched pair, an expired certificate, or one with no subject alternative name — saying which, rather than failing at the next handshake.

Set YAGRA_WEB_TLS=off in .env when an external reverse proxy or load balancer already terminates HTTPS in front of the container.

Core's API port is still plaintext

8080 carries the same material as the UI — the login password, bearer tokens, and (pre-encryption) device credentials — and is still published on the LAN. That is deliberate sequencing: move Prometheus scrapes, webhook senders and API scripts to https://<host>/api/v1/… with a certificate they trust first, then set YAGRA_API_BIND=127.0.0.1 to take the port off the network. Doing both at once fails every machine client at the same moment, with two overlapping causes. See security.

C — Single node, native

Running the binaries directly, no Docker. You provision the stores yourself, build the workspace, and run yagra-core + yagra-poller as services (e.g. systemd).

1. Provision the backing stores

Install and start, reachable from the host that will run core:

• PostgreSQL 17 — create a database and role (core runs its migrations itself; it does not create the database):

  CREATE ROLE yagra LOGIN PASSWORD 'yagra';
  CREATE DATABASE yagra OWNER yagra;

• NATS 2.x with JetStream — nats-server -js

• VictoriaMetrics — victoria-metrics-prod --retentionPeriod=12 (12 months of metrics)

• Redis 7 (optional) — enables the poller liveness/assignment mirror; its absence only degrades, never blocks startup

• VictoriaLogs (optional) — full-text passive-event search; without it, events stay entirely in PostgreSQL

• ClickHouse 24.x (optional) — the traffic-flow store; without it, flow monitoring is off and the flow API answers 503

2. Build the workspace

Requires Rust 1.90 and (for the WebUI) Node 22. Build from the repository root so the workspace’s vendored dependency patches apply:

git clone https://github.com/horryworks/Yagra.git
cd Yagra
cargo build --release --workspace           # → target/release/yagra-core, target/release/yagra-poller
cd web && npm ci && npm run build           # → web/dist/  (static SPA bundle)

3. Provision the KEK (before first core start)

The envelope-encryption master key — a persistent 32-byte file. Without it, core boots with an ephemeral dev key and stored credentials won’t survive a restart:

sudo install -d -m 0700 /etc/yagra
head -c 32 /dev/urandom | sudo tee /etc/yagra/kek > /dev/null
sudo chmod 0400 /etc/yagra/kek

Back this file up. Losing it makes every stored credential permanently undecryptable.

4. Run core

export YAGRA_DATABASE_URL="postgres://yagra:yagra@localhost:5432/yagra"
export YAGRA_BUS_URL="nats://localhost:4222"
export YAGRA_TSDB_URL="http://localhost:8428"
export YAGRA_REDIS_URL="redis://localhost:6379"      # optional
export YAGRA_LOGS_URL="http://localhost:9428"        # optional: passive-event log store
export YAGRA_CLICKHOUSE_URL="http://localhost:8123"  # optional: traffic-flow store
export YAGRA_KEK_FILE="/etc/yagra/kek"
export YAGRA_API_ADDR="0.0.0.0:8080"                 # default
# export YAGRA_ADMIN_PASSWORD="choose-a-strong-password"   # else a one-time random one is logged
export RUST_LOG=info

./target/release/yagra-core

On startup core connects to the stores, runs its migrations automatically, seeds the built-in profiles and catalog, and serves /api/v1 + Prometheus /metrics on YAGRA_API_ADDR — check it with curl -fsS http://localhost:8080/healthz. If YAGRA_ADMIN_PASSWORD was unset, grep the logs for the one-time admin password. If any of the three required URLs (YAGRA_DATABASE_URL, YAGRA_BUS_URL, YAGRA_TSDB_URL) is missing, core runs in an in-memory skeleton mode instead of live. For a permanent installation, wrap this in a service unit (systemd or equivalent) with the same environment; core is safe to restart at any time. Full variable list: the configuration reference.

5. Serve the WebUI

web/dist/ is a static bundle; serve it with any web server and reverse-proxy /api to core. Mirror the shipped nginx config (web/nginx.conf) — the essentials are that SSE needs proxy_buffering off and a long proxy_read_timeout, and the SPA needs a try_files fallback:

server {
    listen 80;
    root /var/www/yagra;                    # the contents of web/dist/
    index index.html;

    location /api/ {
        proxy_pass http://localhost:8080;   # → core's YAGRA_API_ADDR
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_http_version 1.1;
        proxy_buffering off;                # stream SSE immediately
        proxy_read_timeout 1h;              # keep SSE connections open
    }

    location / {
        try_files $uri $uri/ /index.html;   # SPA client-side routing fallback
    }
}

Serving the SPA from a different origin

The production pattern is same-origin: the browser calls /api/v1/… on the web server and nginx proxies it to core, so no build-time configuration is needed. If you instead serve the SPA from one origin and the API from another, build the bundle with VITE_API_BASE set to the API origin — e.g. VITE_API_BASE=https://core.example.net npm run build. It is an origin, never a path: do not append /api/v1 (every request path already carries it, and the live-update streams use the same base).

6. Run the poller

The poller needs raw sockets for ICMP. Grant the capability to the binary (so it can run non-root), or run it as root:

sudo setcap cap_net_raw+ep ./target/release/yagra-poller

export YAGRA_BUS_URL="nats://localhost:4222"
export YAGRA_POLLER_ID="poller-1"          # unique per poller; defaults to hostname
export YAGRA_POLLER_POOL="default"
# Optional intake listeners, each off until bound (binding :514/:162 directly
# would additionally need root or CAP_NET_BIND_SERVICE — these high ports don't):
# export YAGRA_SYSLOG_BIND="0.0.0.0:1514"
# export YAGRA_TRAP_BIND="0.0.0.0:1162"
# export YAGRA_FLOW_BIND="0.0.0.0:2055"
export RUST_LOG=info

./target/release/yagra-poller

The poller exposes its own Prometheus /metrics on 0.0.0.0:9100.

Optional: PDF reports

Reports → PDF export shells out to wkhtmltopdf (the patched-Qt build). If it’s not installed, PDF export returns HTTP 503 (pdf_unavailable); HTML and CSV export still work.

D — Distributed pollers, Docker

Run the full stack centrally (as in B) and add pollers at remote sites. Each remote poller polls its site’s devices locally and streams results back over the bus. Nodes carry a pool attribute; core assigns each pool’s nodes across its live pollers by consistent hashing and fails them over automatically. Concepts and behavior in depth: distributed polling.

TLS + auth before the bus leaves the machine

Job messages carry plaintext device credentials. On one host that’s fine — the bus stays on the internal Docker network. The moment it crosses a trust boundary to a remote site, it must be TLS-encrypted and authenticated first. Never publish NATS :4222 plaintext.

Step 1 — Turn on NATS TLS + auth on the central stack

This is the opt-in block already present (commented) in docker-compose.deploy.yml under the nats service. All five steps are required:

1. Generate a server cert into ./certs. The SAN must include the exact host/IP each poller will dial:

   mkdir -p certs
   openssl req -x509 -newkey rsa:2048 -nodes -days 3650 \
     -keyout certs/server-key.pem -out certs/server-cert.pem \
     -subj "/CN=yagra-nats" \
     -addext "subjectAltName=DNS:nats,DNS:core.example.com,IP:203.0.113.10"

   The cert is self-signed, so it is its own CA. You hand out server-cert.pem — the public cert only, never the key — in step 5.

2. Set bus passwords in .env (no defaults, on purpose):

   YAGRA_NATS_CORE_PASSWORD=a-strong-core-bus-password
   YAGRA_NATS_POLLER_PASSWORD=a-strong-poller-bus-password
   YAGRA_NATS_PORT=4222        # host port to publish the bus on
   YAGRA_CERT_DIR=./certs

3. Load the auth/TLS config. In docker-compose.deploy.yml, comment out command: ["-js"] on the nats service and uncomment the block below it — it loads docker/nats/nats-server.conf, injects the two passwords, mounts ./certs, and publishes the bus port.

4. Switch the internal clients to TLS — server-wide TLS leaves no plaintext port, so the co-located core and poller must use tls:// too:

   # on the core service:
   YAGRA_BUS_URL: tls://core:${YAGRA_NATS_CORE_PASSWORD}@nats:4222
   # on the local poller service:
   YAGRA_BUS_URL: tls://poller:${YAGRA_NATS_POLLER_PASSWORD}@nats:4222
   # and on BOTH, the pinned server cert plus its volume mount:
   YAGRA_BUS_CA_FILE: /etc/nats/certs/server-cert.pem

   volumes:
     - ${YAGRA_CERT_DIR:-./certs}:/etc/nats/certs:ro

5. Hand certs/server-cert.pem (public cert only) to each remote poller operator — it becomes their YAGRA_BUS_CA_FILE.

Bring the central stack back up:

docker compose -f docker-compose.deploy.yml up -d

The NATS config gives the core user full access and the poller user least privilege — publish results/events/heartbeats, subscribe only to jobs and working-set assignments. Note that the one poller account is shared: any authenticated poller can read any pool’s assignments, so it is not a tenant boundary. To scope credentials per poller — so a compromised site can only read its own pool’s device credentials — enable the optional Auth Callout step described in the same Compose block. Details: security.

Step 2 — Register the poller in the WebUI

Go to Settings ▸ Pollers ▸ “Register poller”. It generates a ready-to-use .env for the remote host — id, pool, and bus URL. Assign the pool you want this poller to serve.

Step 3 — Run the remote poller

On the remote-site machine, using docker-compose.poller.yml (runs only a poller):

# put the generated .env next to docker-compose.poller.yml,
# and the CA cert from step 1 into ./certs
mkdir -p certs && cp /path/to/server-cert.pem certs/

docker compose -f docker-compose.poller.yml up -d

Three variables are required — Compose errors out if any is unset — and the generated .env supplies them:

YAGRA_BUS_URL=tls://poller:a-strong-poller-bus-password@core.example.com:4222
YAGRA_POLLER_ID=edge-tokyo-1        # stable, unique per poller
YAGRA_POLLER_POOL=tokyo             # the pool this poller serves
YAGRA_BUS_CA_FILE=/etc/yagra/certs/server-cert.pem

Pin the image with YAGRA_IMAGE_TAG here too. Per-scenario extras (intake rate caps, flow collection, the store-and-forward buffer) are all in the configuration reference.

docker-compose.poller.yml uses host networking — passive syslog/trap correlation keys on the datagram source IP (bridge NAT would rewrite it), and raw-socket ICMP wants the host’s interfaces directly — and grants only NET_RAW. The poller appears on Settings ▸ Pollers within a few seconds of starting, and core begins assigning that pool’s nodes to it.

Privileged-port caveat

The remote poller runs non-root (a file capability for raw ICMP only), so it cannot bind :514/:162 (below 1024). Use the default high ports (1514/1162) and redirect on the host firewall — e.g. iptables -t nat -A PREROUTING -p udp --dport 514 -j REDIRECT --to-ports 1514 — or point devices straight at the high ports.

A few properties worth knowing:

• Scaling a pool = run more pollers with the same YAGRA_POLLER_POOL (and distinct YAGRA_POLLER_IDs). Core rebalances the pool across them and fails over on loss.
• A pool with zero live pollers falls back to legacy per-job publish, so no nodes go dark during a rollout.
• WAN outages don’t hole your history. The poller keeps polling locally and buffers results (in memory, spilling to the pollerbuf volume), then replays them on reconnect — metrics are backfilled at their original timestamps. Alerts are deliberately not backfilled; they resume from “now”.
• Flow at the edge: with host networking the poller binds :2055 directly, so point the site’s NetFlow/IPFIX exporters at the poller — flows are aggregated at the edge and streamed to core over the same TLS bus.

Size the site uplink

Pollers carry the original received bytes to core so that forwarding can later relay exactly what a device sent: passive events travel with a copy of the raw datagram, and every flow datagram is relayed verbatim alongside the aggregated stream. For a remote site this is real WAN traffic that scales with the site’s syslog/trap volume and flow rate — on the order of a megabit per second per 1,000 flows/s. Budget the uplink for it, and cap bursts with the per-source and global intake rate limits in the configuration reference.

E — Distributed pollers, native

Same as D, but the remote poller is the native binary instead of a container. The central bus TLS + auth setup (D, step 1) is unchanged.

On the remote host, build (or copy) the yagra-poller binary, drop the CA cert somewhere readable, and run:

sudo setcap cap_net_raw+ep ./yagra-poller

export YAGRA_BUS_URL="tls://poller:a-strong-poller-bus-password@core.example.com:4222"
export YAGRA_POLLER_ID="edge-tokyo-1"       # unique per poller
export YAGRA_POLLER_POOL="tokyo"
export YAGRA_BUS_CA_FILE="/etc/yagra/certs/server-cert.pem"
# Optional intake listeners (see the privileged-port caveat in D):
# export YAGRA_SYSLOG_BIND="0.0.0.0:1514"
# export YAGRA_TRAP_BIND="0.0.0.0:1162"
# export YAGRA_FLOW_BIND="0.0.0.0:2055"
export RUST_LOG=info

./yagra-poller

Run it on the host network (not a private network namespace) so passive-event source-IP correlation and raw ICMP work against the site’s real interfaces. Everything else — pools, registration, failover, buffering — behaves exactly as in D.

Upgrades & backups

Upgrades are designed to be low-effort and to never lose or corrupt data. Skim the changelog for the release you are moving to first — behavior changes an operator or API client could notice are always listed there.

For Docker deployments, change the tag and recreate:

# in .env: YAGRA_IMAGE_TAG=v0.2.0   (or override inline as below)
YAGRA_IMAGE_TAG=v0.2.0 docker compose -f docker-compose.deploy.yml pull
YAGRA_IMAGE_TAG=v0.2.0 docker compose -f docker-compose.deploy.yml up -d

What makes this safe:

• Migrations are expand-contract and run automatically on core startup. N→N+1 is always supported; there is no manual migration CLI.
• The bus is version-tolerant (N/N-1). A new core works with old pollers during a rollout, so upgrade core first and pollers after — including remote sites, one at a time, in any order.
• Pollers are stateless — replace them freely. A pool briefly without pollers falls back to per-job publish, so no node goes dark mid-rollout.
• Persistent data is preserved. The pgdata (PostgreSQL), vmdata (VictoriaMetrics), and kekdata (KEK) volumes — or their native equivalents — survive image upgrades untouched.

Before a major upgrade, take a backup:

# PostgreSQL — nodes, configuration, users, alert history
docker compose -f docker-compose.deploy.yml exec postgres \
  pg_dump -U yagra yagra > yagra-backup.sql

# The KEK — tiny, and irreplaceable
docker run --rm -v yagra_kekdata:/kek busybox cat /kek/key > kek-backup.key

VictoriaMetrics data lives in the vmdata volume — snapshot it with your usual volume tooling (or VictoriaMetrics’ own snapshot API). Redis is rebuildable; losing it is non-fatal and it needs no backup. Rollback = re-run with the previous tag; the immutable :<git-sha> tags exist for exactly this.

Never lose the KEK

Databases can be restored from backup; the KEK cannot be regenerated. If the kekdata volume or KEK file is destroyed, every stored monitoring credential becomes permanently undecryptable. Back it up alongside the database, and store the copy somewhere the host’s failure can’t reach.

Next steps

• Every environment variable, default, and clamp — the configuration reference
• Which port carries what, and what to firewall — the port matrix
• How pools, working sets, and failover actually behave — distributed polling
• Running more than one core — high availability
• TLS everywhere, the KEK, and per-poller bus credentials — security
• Structured logs, Prometheus metrics, and distributed tracing — observability