Services and health checks

A health check is a command that the SSH gateway runs on a schedule to tell whether a service is up: exit code 0 means healthy. A service is a named entry on a connection that says what runs on the host, such as nginx, with an optional description and at most one health check. The SSH gateway is the part of Athenode through which your AI agent runs commands and transfers files on your SSH hosts, each action approved by you or by a rule you set. A connection is one SSH host in the SSH gateway, with its login, its host key and its rules.

Your AI agent reads the services of a connection with get_connection and list_connections, so the services and their descriptions also tell the agent what runs where. When a service goes down or recovers, Athenode notifies your browser, and on Athenode Pro Team and up also a webhook, if you set one.

Field of a health check	Label in the web UI	Default	Range
command	Check command	None	Required, up to 10,000 characters
intervalSec	Interval (seconds)	300 (5 minutes)	30 to 86,400 (24 hours)
timeoutSec	Timeout (seconds)	30	1 to 600 (10 minutes), and never above the interval
failureThreshold	Failures before down	2	1 to 10
enabled	Enabled	On	On or off
sudo	Run with sudo	Off	On or off

The command runs on the host in a POSIX shell, as an exec command does. With Run with sudo it runs as root with the connection's stored sudo password, and stdout and stderr are merged. The option can be turned on only when the connection has a sudo password stored.

Services, health checks and browser notifications are in every edition: the athenode daemon, the ssh-gateway package and every Pro tier. Service alert webhooks need Athenode Pro Team or Enterprise. On Athenode Pro, what you may do with a connection's services, checks and webhook depends on your project role, as Project roles lists; a member who may view the connection sees the services and checks read-only.

Add a service to a connection

Add a service for each thing on the host that you or your agent need to know about. You need a connection, as the SSH gateway describes.

1. Open Connections in the project's sidebar and select the connection.
2. On the Overview tab, find the Services card.
3. Enter the service's name, such as nginx, and optionally what it does.
4. Select Add service.

The service appears in the list. A name has up to 100 characters and is unique on the connection, whatever the letter case. A description has up to 1,000 characters and supports Markdown. A service without a check shows "No health check: nothing watches whether this service is up." in its sheet.

Add a health check to a service

1. In the Services card, select Edit next to the service.
2. In the Health check section of the sheet, select Add a health check.
3. Enter the Check command, such as systemctl is-active nginx.
4. Optional: change Interval (seconds), Timeout (seconds) and Failures before down.
5. Optional: turn on Run with sudo.
6. Select Save.

A check you save in the web UI is in effect at once, with no approval. The service shows Not run yet until the first run, which starts within 5 seconds.

Change or remove a health check

To change a check, select Edit next to the service, change the fields in the Health check section and select Save. A changed check shows Not run yet until its next run, which starts within 5 seconds, and a run that started before the change does not count for the changed check. To pause a check without removing it, turn off Enabled and save.

To remove a check, select Remove check in the sheet and confirm.

Run a check now

Run a check outside its schedule to see whether a fix worked. Select Run now in the service's sheet, or in the run history of the service. The run's result appears next to the button, such as "Run: Healthy (exit 0, 42 ms)", and the run is added to the history. A run started with Run now needs no approval, because the check was approved when it was saved.

Read the health of a service

A service shows one of these states, in the Services card, in the connections table and in the Galaxy view, where a service without a check stays grey:

State	Meaning
No check	The service has no health check.
Not run yet	The check has not run since it was created or changed.
Healthy	The last run exited with code 0.
Unhealthy	The command exited with another code, as many times in a row as Failures before down.
Timed out	The command ran longer than its timeout, as many times in a row as Failures before down, and was stopped.
Unreachable	The last run could not connect to the host.
Error	The last run could not start the check, for example because the connection has no sudo password stored, sudo authentication failed or the connection dropped before the command ended.
Paused	The check does not run, for the reasons in when health checks run.

An unhealthy run and a timeout count as failures. Below the threshold, the service keeps its state and shows the number of failures. The first healthy run sets it to Healthy again. Unreachable and Error show at once and do not count towards the threshold, and the connection's own status already tells you when a host cannot be reached.

The sidebar and the connections table sum up each connection, such as "3 healthy · 1 failing".

The run history of a service

Select a service's name to open its details. Under the service, a strip shows the outcome of its last 30 runs. The run history card below the list shows a chart of the durations of the last 50 runs, coloured by outcome, with the share of healthy runs, and the Check history list. Each run in the list has its start, duration, outcome, exit code or signal, and the first 4 KB of its output, stdout then stderr. Binary output is shown in base64.

The history keeps runs for 7 days and at most 1,000 runs per service, whichever limit is reached first. Health check history and Health check runs kept per service on the Settings page change both limits, as History and gateway settings describes. Scheduled runs and Run now are kept in this history only, not in the History of agent actions.

When health checks run

Health checks run while the daemon or the Athenode Pro server runs. After the server starts, the first runs of the checks are spread over their intervals, and each next run starts one interval after the previous run ended. Runs of one check never overlap. At most one check runs at a time on a connection and four across the server, and the others wait their turn. A run that is in progress when the server stops is dropped.

A host that cannot be reached is tried again after 15 seconds, then after twice as long each time up to 5 minutes, and never more often than the check's interval.

A check is Paused while:

• Enabled is off;
• the connection's status poll is suspended after an authentication failure or a host key failure, until you fix the login or trust the key;
• polling is off for the connection, which its page shows as Polling off.

The checks start again when the cause is gone. Setting disableHealthChecks or ATHENODE_DISABLE_HEALTH_CHECKS=1 stops every scheduled check of the daemon, as Configuration describes. On Athenode Pro, checks always log in with the connection's stored credentials, as the gateway on Athenode Pro describes.

Get browser notifications for services

Athenode notifies you when a service goes down, that is turns Unhealthy or Timed out after the threshold, and when it recovers, that is turns Healthy after being down. Unreachable, Error, Paused and Not run yet send no notification.

1. Open the server's Settings page.
2. In the Notifications section, select Allow and allow notifications for the site when your browser asks.

The section then says "Notifications are enabled." A notification reads, for example, "nginx is down on web-1 (exit 3)" or "nginx recovered on web-1". The setting applies to this browser only, and notifications appear while a tab of the web UI is open. The same setting turns on the notifications for actions that wait for approval. If the browser blocks notifications for the site, allow them in the browser's settings.

Send service alerts to a webhook

A webhook receives a JSON POST for each down and recovered event, so a chat channel, an incident tool or a script of yours can act on it. The server has one gateway webhook, and each connection can have its own, which replaces the gateway webhook for that connection's services.

The athenode daemon and the ssh-gateway package deleted the webhooks stored on them when they were upgraded to the version that moved webhooks to Athenode Pro. On Pro Solo, the webhooks stay stored, nothing is sent, and they apply again after an upgrade, as Pro tiers describes.

The webhooks are set in the web UI or in Pro's config.json and never through the Athenode MCP server, so an AI agent cannot read or redirect alerts that carry command output.

Set the gateway webhook

1. Open the server's Settings page.
2. In the Service alerts section, enter the Webhook URL.
3. Optional: enter a Signing secret.
4. Select Save webhook.

The section shows "Webhook saved", and later the time of the last delivery and the last error, if any. A stored secret is never shown again: the field reads "A secret is stored (type to replace it)", and Remove the stored secret removes it. To remove the webhook, empty the URL and save. Only administrators see and change the gateway webhook.

serviceCheckWebhookUrl and serviceCheckWebhookSecret in Pro's config.json set the gateway webhook at start while none is stored, as Configure Athenode Pro describes. After that, the value saved in the web UI applies. The daemon and the ssh-gateway package ignore both keys with a warning. A secret in config.json stays readable in that file, so keep the file private or set the secret in the web UI.

Set a webhook for one connection

You need the owner role in the connection's project.

1. Open the connection's page and select the Notifications tab.
2. In the Service alerts section, enter the Webhook URL and, optionally, a Signing secret.
3. Select Save webhook.

A connection without a webhook of its own uses the gateway webhook. To go back to it, empty the URL and save.

What the webhook receives

{
  "event": "down",
  "connection": { "id": "<connection-id>", "name": "web-1" },
  "service": { "id": "<service-id>", "name": "nginx" },
  "status": "unhealthy",
  "exitCode": 3,
  "output": "inactive\n",
  "outputEncoding": "utf8",
  "time": "2026-10-01T10:00:00.000Z",
  "url": "http://127.0.0.1:3717/connections/<connection-id>"
}

Field	Meaning
event	down or recovered
connection, service	The id and name of each
status	The state of the service: unhealthy, timeout or healthy
exitCode	The exit code of the run, or null
output	The first 1,024 characters of the run's output
outputEncoding	utf8, or base64 for binary output
time	When the run started
url	The connection's page in the web UI

Each attempt waits up to 10 seconds for an answer. A network error, a timeout or an answer outside the 2xx range, redirects included, is retried 3 times, after 1, 4 and 16 seconds. The last error is shown under the webhook.

Verify the signature of a webhook

With a signing secret, each request carries the header X-SSH-Gateway-Signature with the value sha256= followed by the HMAC-SHA256 of the exact body, in hexadecimal, with the secret as the key. To check that a request came from your server:

1. Read the body as raw bytes, before any JSON parsing.
2. Compute the HMAC-SHA256 of those bytes with your secret and write it in hexadecimal after sha256=.
3. Compare the result with the header, with a constant-time comparison.
4. Reject the request when they differ.

Manage services and health checks from your AI tool

From your AI tool, your agent manages services with the tools of the Athenode MCP server:

Tool	What it does
add_service	Adds a service, with an optional check
update_service	Renames a service, changes its description, or changes its check; check: null removes the check
remove_service	Removes a service with its check and history, without a confirmation step
service_health	Returns the check, the state and the run history, newest first: 20 runs by default, at most 200
run_service_check	Runs the saved check now and returns the run; refused when the service has no check or the check is paused

An agent that creates a check, changes its command, sudo, interval, timeout or threshold, or turns it on, waits for approval as an exec of the same command would: the same allow rules, denylist and confirmation mode apply, as Approvals and rules describes. Always allow on such a request saves an allow rule for that command on the connection. A rejection or a timeout saves nothing. Renaming a service, describing it, turning its check off, removing the check and removing the service need no approval. run_service_check and service_health never wait for approval.