Skip to main content

Heartbeat protocol

A worker holds no open port. Every few seconds it posts the state of each process it owns to a single endpoint, and the Core replies with what it wants those processes to be doing. That is the entire contract — there is no second call, no callback, and no connection from the Core back to the worker.

The loop

POST {coreUrl}/api/v1/workers/heartbeat

The worker sends every process it knows about in one batch. The Core answers with a desired state for each.

The starter runs this loop for you. You configure the interval and never call the endpoint yourself:

application.yaml
synapsys:
core-url: http://localhost:8080
worker-name: billing-jobs
heartbeat-interval: 5s

Commands are matched by token, not by state

The obvious design — compare desiredState against currentState and act on the difference — replays the command on every beat until the state catches up. A process that takes thirty seconds to stop would receive six stop commands.

So the Core does not ask "are these states different?". It increments a desiredToken every time a user issues a command, and the worker acts only when that token is newer than the one it last acknowledged.

if (desiredToken > ackToken) {
apply(desiredState);
ackToken = desiredToken;
}

The worker reports its ackToken on the next heartbeat. Core clears an acknowledged start immediately. It clears an acknowledged stop after the worker also reports a terminal state (idle or failed). In both cases it resets desiredState to idle.

caution

A desiredState of idle means "no command", never "stop". Stop is stopping. A client that treats idle as an instruction will shut down every process it owns on the first heartbeat after a command completes.

What each side is responsible for

The workerThe Core
Opens the connectionalwaysnever
Reports currentStateyes
Reports ackTokenyes
Issues desiredStateyes
Increments desiredTokenon each user command
Clears a completed startonce ackToken >= desiredToken
Clears a completed stoponce acknowledged and terminal

One bad process does not fail the batch

If a worker sends ten processes and one of them is unreadable — an unknown state, a missing name — the Core keeps the nine it understands and drops the one it does not. The heartbeat is rejected only when nothing in it is readable.

This is what keeps an older Core forward-compatible with a newer client: a field it does not recognise costs one process, not the whole worker.

Next