portal-relay-diagnose
Find which layer of a self-hosted Portal relay is failing - container and process, open ports, embedded DNS and NS/glue delegation, DNSSEC and the parent DS, TLS and SNI routing, the QUIC UDP backhaul
SKILL.mdplugins/portal-relay/skills/portal-relay-diagnose/SKILL.md
GitHubDiagnose a Portal relay
Work from the bottom layer up and stop at the first layer that fails: a DNS check means nothing while the container is down. Every check here is read-only and bounded (curl --max-time 10, dig +time=3 +tries=1, --tail 200 on logs). Do not restart, recreate, or reconfigure anything during diagnosis; propose the fix and let the operator approve it.
Run checks from two places when possible, because a check on the relay host passes through no cloud firewall:
- Host: a shell on the relay machine.
- Outside: any other network, for example the agent's own environment.
Say which place each result came from.
1. Process and container (host)
docker compose ps portalis running and not restarting.docker compose logs portal --tail 200: look for bind errors (address already in useon443or53),permission deniedonIDENTITY_PATH(the image runs as uid/gid65532), embedded DNS start failures, and theds_recordline.sudo ss -tulpn | grep -E ':(443|53) ': the relay owns443/tcp,53/tcp,53/udp, and443/udpwhen UDP leases are on. Another process on53is oftensystemd-resolved.
2. Ports (outside)
curl -fsS --max-time 10 https://<relay>/api/healthz. A timeout with the container healthy points to a cloud firewall or host firewall, not to Portal.dig +time=3 +tries=1 @<relay IP> <relay domain> SOAand the same with+tcp: one passing and one failing means only one of53/udpand53/tcpis open.- Compare with the provider's firewall rules. OCI images also reject inbound in host iptables; Lightsail and Hetzner use their own firewalls in front of the host.
3. DNS delegation (outside)
dig @<relay IP> test.<relay domain> Aanswers: the embedded DNS works.dig +short NS <relay domain> @1.1.1.1returnsns.<relay domain>, anddig +short A ns.<relay domain> @1.1.1.1(andAAAA) returns the relay address: the parent delegation and glue are right.- If 1 passes and 2 fails, the problem is at the parent zone: a missing NS, a wrong glue address (common after a public IP changed), a proxied glue record, or propagation still in progress. Read the parent's records with
dig @<parent NS> <relay domain> NS +norec. - An empty AAAA on an IPv4-only relay is expected.
4. DNSSEC (outside)
dig DS <relay domain> @1.1.1.1compared with theds_recordlog line: key tag, algorithm13, digest type2, and digest must match exactly.dig +dnssec <relay domain> SOA @1.1.1.1:SERVFAILfrom a validating resolver with a DS present, whiledig +cdsucceeds, means a DS mismatch or a replaceddnssec-csk.json. The fix is to publish the DS from the current key or to restore the old key file, never to delete the key.- No DS at the parent means DNSSEC is simply not chained yet; it is not a failure.
5. TLS and SNI (outside)
openssl s_client -connect <relay IP>:443 -servername <relay domain> </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -datesfor the relay's own name.- A tunnel hostname that fails the TLS handshake while the relay name works means no routable lease for that name (offline, pending approval, denied), not a certificate problem. Check it in layer 7.
- With
ACME_DNS_PROVIDERset to an external provider, look for provider authentication errors in the logs.
6. Transport (outside, only when enabled)
- UDP leases need
443/udppublished and open, because the QUIC backhaul uses the publicPORTAL_URLport. - TCP/UDP lease ports: the Docker published range, the host firewall, and the cloud firewall must all cover the same
MIN_PORT-MAX_PORTrange. Probe one allocated port only, for examplenc -vz -w 5 <relay IP> <port>. Never scan the range. - Error codes from a publisher:
udp_disabled,tcp_port_disabled(lease type off),*_capacity_exceeded(max_leasesreached), and*_port_exhausted(range too small).
7. Policy (needs portal-relay-admin)
lease_rejected: the lease waits for approval inmanualmode.lease_denied: the identity is denied or banned.source_not_allowed: the publisher's address is outsideTUNNEL_SOURCE_CIDRS. Behind a proxy that also needsTRUST_PROXY_HEADERSandTRUSTED_PROXY_CIDRS.- Read the lease state through
portal-relay-admin; do not change policy as part of diagnosis.
Report
Give the first failing layer, the evidence (the command, where it ran, and its output), the most likely cause, and the fix as a proposal. List the layers that passed and the checks that could not run (for example no shell on the host). Do not print ADMIN_TOKEN, provider tokens, or the contents of IDENTITY_PATH.