Fix: a registered storage server shows as unreachable

What you're seeing

In the ImpulseObjects admin tab, the server row's status badge is amber or red. The admin-proxy health check has stopped ticking green. Any provisioning, quota, or replication task that targets this server stalls in the cron runner or fails outright. The audit log under mod_impulsecore_audit_log filtered to module=impulseobjects shows connection-refused, TLS, or HTTP timeout exceptions.

Why it happens

ImpulseObjects talks to each storage host over two channels:

• The S3 endpoint on TCP 443 (HTTPS) for tenant operations.
• The admin proxy on TCP 9099 (default) for create/delete bucket, user, quota.

If either channel is broken, the server is reported as unreachable. The usual culprits are firewall rules (especially missing IPv6), a TLS certificate that didn't renew, or a per-region admin-proxy bearer token that has drifted from what's stored in WHMCS.

Fix

1. Get a clean baseline. From the WHMCS box, confirm the host is up at the IP level:

   Copy code

   ping -c 3 us-central-dallas.example.comping6 -c 3 us-central-dallas.example.com

2. Test the S3 endpoint without TLS validation. This isolates a cert problem from a connectivity problem. Hit the RustFS health endpoint, /health:

   Copy code

   curl -kI https://us-central-dallas.example.com/health

   200 OK means the storage engine is up and your only issue is TLS. Connection refused or hang means a firewall or down service.

3. Test the admin proxy. Substitute the bearer token from the server row's Edit form:

   Copy code

   curl -H "Authorization: Bearer <token>" \  https://us-central-dallas.example.com:9099/admin/v1/health

4. Confirm UFW rules cover both IPv4 and IPv6. On the storage host:

   Copy code

   ufw status numbered

   Every WHMCS IP needs a rule for ports 443 and 9099 in both tcp and tcp6. A v4-only rule looks correct in casual inspection but blocks any WHMCS box that resolves to IPv6 first.

5. If only the admin proxy is failing, re-check the bearer token. Open the server row in the addon, click Edit, and confirm the admin-proxy port and Bearer token match what's installed on the host. Per-region tokens are not interchangeable.

6. If TLS specifically is failing, SSH into the storage host and check certbot:

   Copy code

   sudo certbot certificatessudo systemctl status nginx

   A stale cert usually means certbot lost DNS API permission. Re-issue the DNS provider token with Zone:Read and re-run certbot renew.

How to confirm it worked

The server row's status badge flips back to green online within one cron tick (typically under five minutes). The admin-proxy last-check timestamp on the row updates. The audit log stops appending new exceptions for this server. Any provisioning task that was stuck retries on its own.

Related

• Fix: a customer's bucket is stuck provisioning
• Fix: ImpulseObjects license validation fails