Cloudflare Tunnel connected but not working: what the status hides

Almost every tool that sits between you and a service will tell you it is connected. Very few of them are telling you that the thing you want to do will succeed. They are answering a narrower question - usually “can I reach my own control plane” - and you are reading it as “the path works end to end”.

Two failures from the same month made this concrete for me. In both, the status output was green and correct, and the service was unreachable.

Case 1: the tunnel was connected, the hostname did not exist

I was exposing a hypervisor web interface through a tunnel so that its management port stayed closed to the internet. The connector was installed as a systemd service, the logs showed a registered connection, and the tunnel appeared in the tunnel list. Every signal said it was up.

The browser disagreed, and so did DNS:

;; status: NXDOMAIN

;; QUESTION SECTION:
pve3.example.com.        IN  A

;; AUTHORITY SECTION:
example.com.  IN  SOA  ns1.provider.net. dns.provider.net.

NXDOMAIN means the name does not exist. Not “the origin refused”, not “the tunnel dropped it”. Nothing had yet tried to reach my network at all.

A tunnel is three independent systems that happen to share one status line:

  1. connector -> provider    "am I registered?"    <- the status line
  2. hostname  -> tunnel      "is there a route?"   <- broken here
  3. tunnel    -> origin      "can I reach it?"     <- untested

Layer 1 was healthy, which is what the connector reports on and all it can report on. Layer 2, the DNS route binding that hostname to that tunnel, was missing or attached to a different tunnel. Layer 3 had not been exercised because no request ever got that far.

The thing that kept me looking in the wrong place was the origin’s self-signed certificate. I knew it was there, I knew it would need handling, and it sat in my head as the likely culprit. It was a real problem, on layer 3, and it could not possibly produce NXDOMAIN. The suspicious detail you already know about is not automatically the one biting you.

Config for layer 3 handling, once the route exists:

tunnel: <TUNNEL-UUID>
credentials-file: /root/.cloudflared/<TUNNEL-UUID>.json

ingress:
  - hostname: pve3.example.com
    service: https://10.0.0.61:8006
    originRequest:
      noTLSVerify: true   # trusted private origin only

  - service: http_status:404

And the route itself, the piece that was missing:

cloudflared tunnel route dns <TUNNEL-NAME> pve3.example.com

Worth being clear about noTLSVerify, because it looks alarming out of context: it governs the hop from the connector to a private origin on my own network. Browser-to-edge traffic stays fully encrypted. It is not “turn off HTTPS”.

I want to flag the honest limit here. I have the diagnosis and the corrected configuration written down; I did not record a successful browser load afterwards, so I am calling this diagnosed rather than confirmed fixed.

Case 2: the CI runner could reach the server and was refused anyway

Different stack, same shape. This is the first row of the failure table in the deploy pipeline writeup, and it deserves the longer version. A CI job needed to deploy to a private server over a mesh VPN, so nothing had to listen on a public port. The runner joined the network, appeared in the peer list, and could see the target. Then:

tailnet policy does not permit you to SSH to this node
Connection closed by 100.x.x.x port 22

I had permitted TCP 22 in the network ACL. My assumption was that permitting port 22 permits SSH. On this stack it does not, because there are two separate authorization layers and the ACL is only the first one:

The packets arrived. The session was refused by policy, which is exactly what the error says once you stop reading it as a connectivity message.

The fix is a rule in the second layer that names source, destination, and the permitted Unix users:

{
  "tagOwners": {
    "tag:ci":     ["autogroup:admin"],
    "tag:server": ["autogroup:admin"]
  },
  "acls": [
    { "action": "accept", "src": ["tag:ci"], "dst": ["tag:server:22"] }
  ],
  "ssh": [
    { "action": "accept", "src": ["tag:ci"], "dst": ["tag:server"],
      "users": ["deploy"] }
  ]
}

Check the policy syntax against your provider’s current version before pasting this; these schemas move.

There is a second route that I would take if you are on a deadline: turn the VPN’s own SSH feature off and use ordinary OpenSSH across the private network.

sudo tailscale set --ssh=false

You keep port 22 closed to the internet, you keep the ACL restricting which peers may reach it, and you go back to a key-based SSH setup that behaves the way twenty years of documentation says it behaves. One less authorization layer to hold in your head at 2am is a legitimate design choice, not a defeat.

The pattern

Both cases are the same mistake. I read a status that was true and generalised it into a claim it was not making.

The habit that fixes it: before trusting a green light, ask what specific question that component is in a position to answer. A connector knows whether it registered with its control plane. It does not know whether your DNS name resolves. A VPN client knows whether it reached the mesh. It does not know whether policy will grant you a session. A container knows whether it started. It does not know whether it can write to its own cache directory.

Then test the layer you care about directly, with the operation you actually want to perform:

# not "is the tunnel up" but "does the name resolve"
dig myservice.example.com

# not "is the peer online" but "does this identity get a session"
ssh -i ~/.ssh/deploy_key deploy@<host>

# not "is the origin healthy" but "can the connector reach it"
curl -kI https://10.0.0.61:8006

Each of those runs from the position of the layer being questioned, and each fails for exactly one reason. Together they cost about thirty seconds and they would have saved me two evenings.

If the status is green and the thing is broken, the status is not lying. You are asking it a question it was never designed to answer.