// blog.guide.kamal
Monitoring Kamal Deployments With LogForge, Without Inventing a Tiny Platform Team.
Keep one LogForge Central on a stable, reachable machine. Then enroll one remote agent on every Kamal-managed Docker host so the hosts phone home over mTLS.
Kamal is nice when the problem is "put this container on that server and keep it there."
Monitoring the servers behind that setup has a different default shape: do not turn every app deploy into an observability deploy. Run one LogForge Central somewhere stable, then put a LogForge remote agent on each Kamal Docker host.
Central can live on an always-on workstation, a home server, a dedicated VM, or another reliable host you control. The important part is not that it is the largest machine. The important part is that it stays online, has persistent storage, is backed up, and is reachable from the Kamal hosts.
The Shape Of The Monitoring Setup
The recommended setup has two parts:
- One LogForge Central. This is the web UI, API, storage, auth, alerting, notifications, and agent enrollment service.
- One LogForge remote agent per Kamal Docker host. The agent watches Docker, host metrics, logs, and container inventory on that host, then sends telemetry back to Central.
The network direction is the part worth getting right:
admin browser ---> LogForge Central :8444/unicron
Kamal host ---> LogForge Central :9443
Kamal host ---> LogForge Central :9443
Kamal host ---> LogForge Central :9443
Agents connect outbound to Central on 9443. Your Kamal application hosts do not need inbound LogForge ports. Dashboard access on 8444 can stay private behind a VPN, private network, bastion, or admin-only firewall rule.
That keeps Kamal focused on deploying your applications and keeps LogForge focused on watching the hosts those applications run on.
Step 0: What You Need
On the Central machine:
- LogForge Central already installed, either with the public installer, Docker Compose, or another stable deployment method.
- Persistent storage for Central state and backups for that state.
- A reachable hostname, for example
logforge.example.com. - Inbound TCP
9443from every Kamal Docker host that will run an agent. - Inbound TCP
8444only from administrators, a VPN, or a private network if you want to open the dashboard remotely.
On each Kamal-managed host:
- Linux with Docker installed.
- SSH access.
- Outbound access to Central on TCP
9443. - Permission to run Docker containers and mount the Docker socket.
In your Kamal repo:
- A
config/deploy.ymlthat lists the target hosts you want to monitor.
For the examples below, replace this value with your real Central hostname:
logforge.example.com
Step 1: Identify The Kamal Target Hosts
Open the Kamal app's config/deploy.yml and find the servers section. Depending on your Kamal version and app shape, it may look like a simple list:
servers:
- 203.0.113.21
- 203.0.113.22
Or it may be grouped by role:
servers:
web:
hosts:
- 203.0.113.21
- 203.0.113.22
job:
hosts:
- 203.0.113.23
Each Docker host that runs your Kamal-managed containers should get exactly one LogForge agent. Do not install one agent per app, per container, per role, or per deploy. The host is the monitoring boundary.
If your deploy file uses hostnames instead of IP addresses, keep the names. They make better agent names and better dashboards:
app-01.internal.example.com
app-02.internal.example.com
worker-01.internal.example.com
Step 2: Confirm Docker And Central Reachability
SSH into each Kamal host and confirm Docker is running:
ssh deploy@203.0.113.21 'docker ps'
Then test that the host can reach Central's agent mTLS port:
ssh deploy@203.0.113.21 'nc -vz logforge.example.com 9443'
If nc is not installed on the host, use Bash's TCP check:
ssh deploy@203.0.113.21 "timeout 5 bash -c '</dev/tcp/logforge.example.com/9443' && echo 9443 reachable"
Repeat that for every host from config/deploy.yml. If the test fails, fix DNS, cloud firewall rules, host firewall rules, NAT, VPN routing, or tunnel policy before generating enrollment commands.
Step 3: Generate One Enrollment Command Per Host
In LogForge:
- Open Settings.
- Open Agents.
- Click Enroll New Agent.
- Use a host-level name, for example
app-01,web-prod-02, orworker-01. - Set the Central URL to
https://logforge.example.com:8444/unicron. - Generate the enrollment command.
The generated command should include Central values like these:
CENTRAL_URL=https://logforge.example.com:8444/unicron
CENTRAL_MTLS_URL=https://logforge.example.com:9443
CENTRAL_WS_URL=wss://logforge.example.com:9443/unicron/api/agent/ws
Generate a fresh command for each Kamal host. Enrollment tokens are short-lived and each agent gets its own identity. Reusing one command across multiple hosts makes the fleet harder to understand and harder to repair.
Step 4: SSH Into Each Kamal Host And Run The Agent
Paste the generated command on the matching host:
ssh deploy@203.0.113.21
# paste the app-01 enrollment command generated by LogForge
Then check the container:
docker ps --filter "name=unicron-agent"
docker logs -f unicron-agent-app-01
Move to the next host, generate a new command in LogForge, and run that new command on that host. A small fleet usually ends up with a clean one-to-one list:
| Kamal host | LogForge agent |
|---|---|
203.0.113.21 |
app-01 |
203.0.113.22 |
app-02 |
203.0.113.23 |
worker-01 |
Step 5: Verify Agents And Container Inventory
Back in LogForge, open the agents view and confirm each host appears online. Then check the Docker views for the useful signals:
- The host appears once, under the agent name you chose.
- Kamal-managed app containers appear in the container inventory.
- Container logs are visible for the app containers you expect to monitor.
- Host metrics and Docker events continue updating after a deploy.
If a Kamal deploy replaces an app container, the LogForge agent should stay put. It is monitoring the Docker host, not riding along with the application release.
Firewall Rules That Match The Shape
Put firewall rules on Central deliberately:
| Port | Allow from | Purpose |
|---|---|---|
9443/tcp |
Every Kamal Docker host that will run a LogForge agent | Agent enrollment, mTLS WebSocket, and telemetry ingress |
8444/tcp |
Your admin IPs, VPN, bastion, or private mesh | Dashboard and API at https://logforge.example.com:8444/unicron |
80/tcp, 443/tcp |
Only if your normal web proxy needs them | Not required for LogForge agents when 9443 is reachable directly |
On a simple Ubuntu Central host using UFW, that might be:
sudo ufw allow from 203.0.113.21 to any port 9443 proto tcp
sudo ufw allow from 203.0.113.22 to any port 9443 proto tcp
sudo ufw allow from 203.0.113.23 to any port 9443 proto tcp
sudo ufw allow from 198.51.100.25 to any port 8444 proto tcp
sudo ufw status verbose
Replace those addresses with your real Kamal host egress IPs, VPC CIDRs, Tailscale subnet, WireGuard subnet, or admin ranges.
The Kamal application hosts do not need inbound LogForge ports. They initiate the connection to Central.
SSL And Proxy Notes
LogForge Central exposes an HTTPS dashboard and a separate mTLS endpoint for agents. The mTLS endpoint is not a normal website. Do not put 9443 behind an HTTP reverse proxy that terminates TLS. Agents need to reach Central's mTLS listener.
If you want a cleaner public dashboard URL, keep the separation:
- Keep the dashboard private on
:8444and access it through Tailscale, WireGuard, or a bastion. - Put only the dashboard behind a reverse proxy that forwards WebSocket traffic and talks to the appliance on
8444. Leave9443as direct TCP to Central. - Use a TCP-capable proxy for the agent port only if it preserves mTLS through to LogForge Central.
For a first install, make 9443 reachable from the Kamal hosts and keep 8444 limited to administrators. Add proxy polish after the agents are online.
Optional: Manage Central With Kamal
The default recommendation is still one stable Central plus agents on the Kamal hosts. If the most reliable place for Central is itself a Kamal-managed Docker server, you can manage Central as a long-lived accessory-style service. Treat that as a Central hosting choice, not as the monitoring pattern.
The LogForge-specific part of the Kamal config can look like this:
accessories:
logforge:
service: logforge-central
image: logforge/unicron:latest
host: 203.0.113.10
port: "8444:443"
env:
clear:
TMPDIR: /run/pyinstaller
UNICRON_APPLIANCE_CONTAINER_NAME: logforge-central
UNICRON_CENTRAL_FQDN: logforge.example.com
UNICRON_CENTRAL_PORT: "443"
UNICRON_PUBLIC_CENTRAL_PORT: "8444"
UNICRON_CENTRAL_MTLS_PORT: "8443"
UNICRON_PUBLIC_CENTRAL_MTLS_PORT: "9443"
UNICRON_DATA_DIR: /var/lib/unicron
LOCAL_AGENT_CENTRAL_URL: https://unicron.central/unicron
LOCAL_AGENT_DOCKER_NETWORK: kamal
CENTRAL_ADMIN_USERNAME: admin
CENTRAL_ADMIN_RECOVERY_OVERRIDE: "false"
secret:
- CENTRAL_ADMIN_PASSWORD
volumes:
- unicron-data:/var/lib/unicron
- /var/run/docker.sock:/var/run/docker.sock
options:
publish:
- "9443:8443"
read-only: true
tmpfs:
- /tmp:rw,nosuid,nodev,mode=1777,size=256m
- /run:rw,nosuid,nodev,mode=755,size=64m
- /run/pyinstaller:rw,nosuid,nodev,exec,mode=1777,size=256m
cap-drop:
- ALL
cap-add:
- CHOWN
- DAC_OVERRIDE
- FOWNER
- KILL
- SETGID
- SETUID
- NET_BIND_SERVICE
security-opt:
- no-new-privileges:true
add-host:
- unicron-stepca:127.0.0.1
- unicron-stepca-ra:127.0.0.1
- unicron.central:127.0.0.1
Store the admin password in Kamal secrets:
mkdir -p .secrets
openssl rand -base64 32 > .secrets/logforge_admin_password
chmod 600 .secrets/logforge_admin_password
Add this to .kamal/secrets or your existing secret provider wiring:
CENTRAL_ADMIN_PASSWORD=$(cat .secrets/logforge_admin_password)
Boot and inspect only the Central service:
kamal accessory boot logforge
kamal accessory details logforge
kamal accessory logs logforge
For upgrades, pin the image tag you want, update config/deploy.yml, then reboot that service intentionally:
kamal accessory reboot logforge
kamal accessory logs logforge
Remember the separation: managing Central with Kamal does not replace installing agents on the other Kamal Docker hosts. The agents are still what monitor the fleet.
Backups
Back up Central, wherever it runs. The state that matters is Central's data directory or Docker volume. If you manage Central with the optional Kamal service above, that volume is:
unicron-data:/var/lib/unicron
For a quiet backup in that optional setup, stop the Central service, archive the volume, then start it again:
kamal accessory stop logforge
ssh root@203.0.113.10 'mkdir -p /root/logforge-backups && docker run --rm -v unicron-data:/data:ro -v /root/logforge-backups:/backup alpine tar czf /backup/unicron-data-$(date +%F).tgz -C /data .'
kamal accessory start logforge
If Central runs on a workstation, home server, or dedicated VM through Docker Compose, back up the Compose volume or bound data directory there instead. The fleet agents can be recreated from LogForge enrollment, but Central state is the thing you do not want to lose.
Troubleshooting
A Kamal host cannot enroll.
Test the agent port from that host:
nc -vz logforge.example.com 9443
If that fails, fix DNS, Central firewall rules, cloud security groups, IPv6 routing, NAT, or VPN policy. The Kamal host needs outbound access to Central on 9443.
The dashboard opens, but agents stay offline.
The dashboard port and the agent port are separate. Confirm the generated command points at the mTLS endpoint:
CENTRAL_MTLS_URL=https://logforge.example.com:9443
CENTRAL_WS_URL=wss://logforge.example.com:9443/unicron/api/agent/ws
Then check the agent logs on the Kamal host:
docker logs -f unicron-agent-app-01
Only one host appears, even though you installed several agents.
Generate a unique enrollment command for each host and give each host a unique agent name. Reusing one command across several hosts can make identities collide or produce misleading inventory.
Kamal deploys work, but LogForge misses containers.
Confirm the agent container has access to the Docker socket and Docker container log directory. The generated command should include mounts for /var/run/docker.sock and /var/lib/docker/containers.
You are not already using Kamal.
Use the normal installer first:
curl -fsSL https://www.logforge.dev/install.sh | sh
Kamal is relevant here because the monitored hosts are Kamal-managed Docker servers. If you only need one Central on one VM, Docker Compose is still a perfectly respectable answer.