Fix: a registered MinIO server shows as unreachable

A storage server registered with ImpulseMinio is showing as unreachable on the admin tab, and any provisioning task that touches it stalls or fails.

Last updated 20 days ago

Fix: a registered MinIO server shows as unreachable

What you're seeing

In the ImpulseMinio admin tab, the server row's status badge is amber or red. The admin-proxy health check has stopped ticking green. Any provisioning, quota, or replication task that targets this server stalls in the cron runner or fails outright. The audit log under mod_impulsecore_audit_log filtered to module=impulseminio shows connection-refused, TLS, or HTTP timeout exceptions.

Why it happens

ImpulseMinio talks to each MinIO host over two channels:

  • The MinIO S3 endpoint on TCP 443 (HTTPS) for tenant operations.
  • The admin proxy on TCP 9099 (default) for create/delete bucket, user, quota.

If either channel is broken, the server is reported as unreachable. The usual culprits are firewall rules (especially missing IPv6), a TLS certificate that didn't renew, or a per-region admin-proxy bearer token that has drifted from what's stored in WHMCS.

Fix

  1. Get a clean baseline. From the WHMCS box, confirm the host is up at the IP level: 
    ping -c 3 us-central-dallas.example.com ping6 -c 3 us-central-dallas.example.com
  2. Test the MinIO endpoint without TLS validation. This isolates a cert problem from a connectivity problem: 
    curl -kI https://us-central-dallas.example.com/minio/health/live

    200 OK means MinIO is up and your only issue is TLS. Connection refused or hang means a firewall or down service.

  3. Test the admin proxy. Substitute the bearer token from the server row's Edit form: 
    curl -H "Authorization: Bearer <token>" \ https://us-central-dallas.example.com:9099/admin/v1/health
  4. Confirm UFW rules cover both IPv4 and IPv6. On the MinIO host: 
    ufw status numbered

    Every WHMCS IP needs a rule for ports 443 and 9099 in both tcp and tcp6. A v4-only rule looks correct in casual inspection but blocks any WHMCS box that resolves to IPv6 first.

  5. If only the admin proxy is failing, re-check the bearer token. Open the server row in the addon, click Edit, and confirm the admin-proxy port and Bearer token match what's installed on the host. Per-region tokens are not interchangeable.
  6. If TLS specifically is failing, SSH into the MinIO host and check certbot: 
    sudo certbot certificates sudo systemctl status nginx

    A stale cert usually means certbot lost DNS API permission. Re-issue the DNS provider token with Zone:Read and re-run certbot renew.

How to confirm it worked

The server row's status badge flips back to green online within one cron tick (typically under five minutes). The admin-proxy last-check timestamp on the row updates. The audit log stops appending new exceptions for this server. Any provisioning task that was stuck retries on its own.

Related