// blog.guide.kamal

Monitoring Kamal Deployments With LogForge, Without Inventing a Tiny Platform Team.

Keep one LogForge Central on a stable, reachable machine. Then enroll one remote agent on every Kamal-managed Docker host so the hosts phone home over mTLS.

Kamal is nice when the problem is "put this container on that server and keep it there."

Monitoring the servers behind that setup has a different default shape: do not turn every app deploy into an observability deploy. Run one LogForge Central somewhere stable, then put a LogForge remote agent on each Kamal Docker host.

Central can live on an always-on workstation, a home server, a dedicated VM, or another reliable host you control. The important part is not that it is the largest machine. The important part is that it stays online, has persistent storage, is backed up, and is reachable from the Kamal hosts.

The Shape Of The Monitoring Setup

The recommended setup has two parts:

  1. One LogForge Central. This is the web UI, API, storage, auth, alerting, notifications, and agent enrollment service.
  2. One LogForge remote agent per Kamal Docker host. The agent watches Docker, host metrics, logs, and container inventory on that host, then sends telemetry back to Central.

The network direction is the part worth getting right:

admin browser  --->  LogForge Central :8444/unicron
Kamal host     --->  LogForge Central :9443
Kamal host     --->  LogForge Central :9443
Kamal host     --->  LogForge Central :9443

Agents connect outbound to Central on 9443. Your Kamal application hosts do not need inbound LogForge ports. Dashboard access on 8444 can stay private behind a VPN, private network, bastion, or admin-only firewall rule.

That keeps Kamal focused on deploying your applications and keeps LogForge focused on watching the hosts those applications run on.

Step 0: What You Need

On the Central machine:

  • LogForge Central already installed, either with the public installer, Docker Compose, or another stable deployment method.
  • Persistent storage for Central state and backups for that state.
  • A reachable hostname, for example logforge.example.com.
  • Inbound TCP 9443 from every Kamal Docker host that will run an agent.
  • Inbound TCP 8444 only from administrators, a VPN, or a private network if you want to open the dashboard remotely.

On each Kamal-managed host:

  • Linux with Docker installed.
  • SSH access.
  • Outbound access to Central on TCP 9443.
  • Permission to run Docker containers and mount the Docker socket.

In your Kamal repo:

  • A config/deploy.yml that lists the target hosts you want to monitor.

For the examples below, replace this value with your real Central hostname:

logforge.example.com

Step 1: Identify The Kamal Target Hosts

Open the Kamal app's config/deploy.yml and find the servers section. Depending on your Kamal version and app shape, it may look like a simple list:

servers:
  - 203.0.113.21
  - 203.0.113.22

Or it may be grouped by role:

servers:
  web:
    hosts:
      - 203.0.113.21
      - 203.0.113.22
  job:
    hosts:
      - 203.0.113.23

Each Docker host that runs your Kamal-managed containers should get exactly one LogForge agent. Do not install one agent per app, per container, per role, or per deploy. The host is the monitoring boundary.

If your deploy file uses hostnames instead of IP addresses, keep the names. They make better agent names and better dashboards:

app-01.internal.example.com
app-02.internal.example.com
worker-01.internal.example.com

Step 2: Confirm Docker And Central Reachability

SSH into each Kamal host and confirm Docker is running:

ssh deploy@203.0.113.21 'docker ps'

Then test that the host can reach Central's agent mTLS port:

ssh deploy@203.0.113.21 'nc -vz logforge.example.com 9443'

If nc is not installed on the host, use Bash's TCP check:

ssh deploy@203.0.113.21 "timeout 5 bash -c '</dev/tcp/logforge.example.com/9443' && echo 9443 reachable"

Repeat that for every host from config/deploy.yml. If the test fails, fix DNS, cloud firewall rules, host firewall rules, NAT, VPN routing, or tunnel policy before generating enrollment commands.

Step 3: Generate One Enrollment Command Per Host

In LogForge:

  1. Open Settings.
  2. Open Agents.
  3. Click Enroll New Agent.
  4. Use a host-level name, for example app-01, web-prod-02, or worker-01.
  5. Set the Central URL to https://logforge.example.com:8444/unicron.
  6. Generate the enrollment command.

The generated command should include Central values like these:

CENTRAL_URL=https://logforge.example.com:8444/unicron
CENTRAL_MTLS_URL=https://logforge.example.com:9443
CENTRAL_WS_URL=wss://logforge.example.com:9443/unicron/api/agent/ws

Generate a fresh command for each Kamal host. Enrollment tokens are short-lived and each agent gets its own identity. Reusing one command across multiple hosts makes the fleet harder to understand and harder to repair.

Step 4: SSH Into Each Kamal Host And Run The Agent

Paste the generated command on the matching host:

ssh deploy@203.0.113.21
# paste the app-01 enrollment command generated by LogForge

Then check the container:

docker ps --filter "name=unicron-agent"
docker logs -f unicron-agent-app-01

Move to the next host, generate a new command in LogForge, and run that new command on that host. A small fleet usually ends up with a clean one-to-one list:

Kamal host LogForge agent
203.0.113.21 app-01
203.0.113.22 app-02
203.0.113.23 worker-01

Step 5: Verify Agents And Container Inventory

Back in LogForge, open the agents view and confirm each host appears online. Then check the Docker views for the useful signals:

  • The host appears once, under the agent name you chose.
  • Kamal-managed app containers appear in the container inventory.
  • Container logs are visible for the app containers you expect to monitor.
  • Host metrics and Docker events continue updating after a deploy.

If a Kamal deploy replaces an app container, the LogForge agent should stay put. It is monitoring the Docker host, not riding along with the application release.

Firewall Rules That Match The Shape

Put firewall rules on Central deliberately:

Port Allow from Purpose
9443/tcp Every Kamal Docker host that will run a LogForge agent Agent enrollment, mTLS WebSocket, and telemetry ingress
8444/tcp Your admin IPs, VPN, bastion, or private mesh Dashboard and API at https://logforge.example.com:8444/unicron
80/tcp, 443/tcp Only if your normal web proxy needs them Not required for LogForge agents when 9443 is reachable directly

On a simple Ubuntu Central host using UFW, that might be:

sudo ufw allow from 203.0.113.21 to any port 9443 proto tcp
sudo ufw allow from 203.0.113.22 to any port 9443 proto tcp
sudo ufw allow from 203.0.113.23 to any port 9443 proto tcp
sudo ufw allow from 198.51.100.25 to any port 8444 proto tcp
sudo ufw status verbose

Replace those addresses with your real Kamal host egress IPs, VPC CIDRs, Tailscale subnet, WireGuard subnet, or admin ranges.

The Kamal application hosts do not need inbound LogForge ports. They initiate the connection to Central.

SSL And Proxy Notes

LogForge Central exposes an HTTPS dashboard and a separate mTLS endpoint for agents. The mTLS endpoint is not a normal website. Do not put 9443 behind an HTTP reverse proxy that terminates TLS. Agents need to reach Central's mTLS listener.

If you want a cleaner public dashboard URL, keep the separation:

  • Keep the dashboard private on :8444 and access it through Tailscale, WireGuard, or a bastion.
  • Put only the dashboard behind a reverse proxy that forwards WebSocket traffic and talks to the appliance on 8444. Leave 9443 as direct TCP to Central.
  • Use a TCP-capable proxy for the agent port only if it preserves mTLS through to LogForge Central.

For a first install, make 9443 reachable from the Kamal hosts and keep 8444 limited to administrators. Add proxy polish after the agents are online.

Optional: Manage Central With Kamal

The default recommendation is still one stable Central plus agents on the Kamal hosts. If the most reliable place for Central is itself a Kamal-managed Docker server, you can manage Central as a long-lived accessory-style service. Treat that as a Central hosting choice, not as the monitoring pattern.

The LogForge-specific part of the Kamal config can look like this:

accessories:
  logforge:
    service: logforge-central
    image: logforge/unicron:latest
    host: 203.0.113.10
    port: "8444:443"
    env:
      clear:
        TMPDIR: /run/pyinstaller
        UNICRON_APPLIANCE_CONTAINER_NAME: logforge-central
        UNICRON_CENTRAL_FQDN: logforge.example.com
        UNICRON_CENTRAL_PORT: "443"
        UNICRON_PUBLIC_CENTRAL_PORT: "8444"
        UNICRON_CENTRAL_MTLS_PORT: "8443"
        UNICRON_PUBLIC_CENTRAL_MTLS_PORT: "9443"
        UNICRON_DATA_DIR: /var/lib/unicron
        LOCAL_AGENT_CENTRAL_URL: https://unicron.central/unicron
        LOCAL_AGENT_DOCKER_NETWORK: kamal
        CENTRAL_ADMIN_USERNAME: admin
        CENTRAL_ADMIN_RECOVERY_OVERRIDE: "false"
      secret:
        - CENTRAL_ADMIN_PASSWORD
    volumes:
      - unicron-data:/var/lib/unicron
      - /var/run/docker.sock:/var/run/docker.sock
    options:
      publish:
        - "9443:8443"
      read-only: true
      tmpfs:
        - /tmp:rw,nosuid,nodev,mode=1777,size=256m
        - /run:rw,nosuid,nodev,mode=755,size=64m
        - /run/pyinstaller:rw,nosuid,nodev,exec,mode=1777,size=256m
      cap-drop:
        - ALL
      cap-add:
        - CHOWN
        - DAC_OVERRIDE
        - FOWNER
        - KILL
        - SETGID
        - SETUID
        - NET_BIND_SERVICE
      security-opt:
        - no-new-privileges:true
      add-host:
        - unicron-stepca:127.0.0.1
        - unicron-stepca-ra:127.0.0.1
        - unicron.central:127.0.0.1

Store the admin password in Kamal secrets:

mkdir -p .secrets
openssl rand -base64 32 > .secrets/logforge_admin_password
chmod 600 .secrets/logforge_admin_password

Add this to .kamal/secrets or your existing secret provider wiring:

CENTRAL_ADMIN_PASSWORD=$(cat .secrets/logforge_admin_password)

Boot and inspect only the Central service:

kamal accessory boot logforge
kamal accessory details logforge
kamal accessory logs logforge

For upgrades, pin the image tag you want, update config/deploy.yml, then reboot that service intentionally:

kamal accessory reboot logforge
kamal accessory logs logforge

Remember the separation: managing Central with Kamal does not replace installing agents on the other Kamal Docker hosts. The agents are still what monitor the fleet.

Backups

Back up Central, wherever it runs. The state that matters is Central's data directory or Docker volume. If you manage Central with the optional Kamal service above, that volume is:

unicron-data:/var/lib/unicron

For a quiet backup in that optional setup, stop the Central service, archive the volume, then start it again:

kamal accessory stop logforge
ssh root@203.0.113.10 'mkdir -p /root/logforge-backups && docker run --rm -v unicron-data:/data:ro -v /root/logforge-backups:/backup alpine tar czf /backup/unicron-data-$(date +%F).tgz -C /data .'
kamal accessory start logforge

If Central runs on a workstation, home server, or dedicated VM through Docker Compose, back up the Compose volume or bound data directory there instead. The fleet agents can be recreated from LogForge enrollment, but Central state is the thing you do not want to lose.

Troubleshooting

A Kamal host cannot enroll.

Test the agent port from that host:

nc -vz logforge.example.com 9443

If that fails, fix DNS, Central firewall rules, cloud security groups, IPv6 routing, NAT, or VPN policy. The Kamal host needs outbound access to Central on 9443.

The dashboard opens, but agents stay offline.

The dashboard port and the agent port are separate. Confirm the generated command points at the mTLS endpoint:

CENTRAL_MTLS_URL=https://logforge.example.com:9443
CENTRAL_WS_URL=wss://logforge.example.com:9443/unicron/api/agent/ws

Then check the agent logs on the Kamal host:

docker logs -f unicron-agent-app-01

Only one host appears, even though you installed several agents.

Generate a unique enrollment command for each host and give each host a unique agent name. Reusing one command across several hosts can make identities collide or produce misleading inventory.

Kamal deploys work, but LogForge misses containers.

Confirm the agent container has access to the Docker socket and Docker container log directory. The generated command should include mounts for /var/run/docker.sock and /var/lib/docker/containers.

You are not already using Kamal.

Use the normal installer first:

curl -fsSL https://www.logforge.dev/install.sh | sh

Kamal is relevant here because the monitored hosts are Kamal-managed Docker servers. If you only need one Central on one VM, Docker Compose is still a perfectly respectable answer.