Ansible play fails with UNREACHABLE after changing sshd port mid-run
Ansible play fails with UNREACHABLE after changing sshd port mid-run
A play rewrites /etc/ssh/sshd_config to Port 2222, reloads sshd, and the very next task dies:
UNREACHABLE! => {"msg": "Failed to connect to the host via ssh: ssh: connect to host 10.0.0.5 port 22: Connection refused"}
Sometimes it hangs instead, burning through ANSIBLE_SSH_RETRIES before giving up. The host itself is healthy — sshd is running, the new port is open — but the play can't touch it anymore, and now you're debugging a connection failure you caused.
Why the ansible ssh port change unreachable error appears
Ansible's SSH connection plugin opens a fresh session for every task. The port it dials comes from the connection variable ansible_port (default 22), read from inventory, group_vars, or play vars [link needed]. Editing sshd_config changes where sshd *listens*; it does nothing to where Ansible *connects*. The task after the restart dials 22, nothing answers, and because no TCP session is ever established Ansible marks the host UNREACHABLE rather than FAILED.
A second trap: wait_for or a retry loop run from the control node with the default connection still tests port 22 unless you change the port first, so the retries succeed against the old socket and prove nothing.
Fix the ansible ssh port change unreachable loop with a safe task order
1. Open a second session to the host from a terminal and leave it logged in until the play completes. It's your way back in if the new port doesn't bind.
2. Allow the port everywhere sshd can be blocked: firewall rule (firewall-cmd --add-port=2222/tcp --permanent or ufw allow 2222/tcp), and on enforcing SELinux hosts semanage port -a -t ssh_port_t -p tcp 2222 [link needed].
3. Validate config before it's live: write to a temp file and run sshd -t -f /etc/ssh/sshd_config.new.
4. Move the file into place, then reload the unit. Reload re-execs sshd and re-reads config without tearing down established sessions on common distro units [link needed].
5. Prove the new port answers before anything else connects to it, from the control node:
```yaml
- name: Wait for the new port to accept connections
ansible.builtin.wait_for:
host: "{{ inventory_hostname }}"
port: 2222
delay: 2
timeout: 20
delegate_to: localhost
```
6. Only then repoint the connection:
```yaml
- ansible.builtin.set_fact:
ansible_port: 2222
```
set_fact applies for the rest of the play. If the port change is the play's last act, skip this and put ansible_port=2222 in inventory for the next run instead.
7. Persist it in host_vars/group_vars so the next run and any ad-hoc commands work, and keep serial: 1 on the first rollout.
Verify while the old port is still open
Check sshd -T | grep -i '^port' for effective config and ss -ltnp | grep sshd for the listening socket. If sshd -T still shows 22, your file edit didn't land or a drop-in under /etc/ssh/sshd_config.d/ overrides it — the first matching Port wins, so an early drop-in silently defeats the later edit [link needed].
When NOT to do this
- sshd under socket activation (
ssh.socketon newer systemd distros):Portinsshd_configmay be ignored entirely; you needListenStream=in the socket unit [link needed]. - Hosts behind a security group or NAT you don't control: open the port there in a prior change window; the play can't do it for you.
- A single host with no console or IPMI: run a throwaway second
sshd -p 2222instance and cut over only after confirming it authenticates. - Large fleets in one shot: port 22 failures across 50 hosts at once are far harder to triage than with
serial.
Checklist
- [ ] New port open in firewall and SELinux, confirmed before reload
- [ ]
sshd -tclean against the exact file being installed - [ ] Drop-in overrides checked
- [ ] Second session held open as a lifeline
- [ ]
wait_foragainst the new port, delegated to localhost - [ ]
ansible_portwritten to inventory, not justset_fact - [ ]
serial: 1for the first rollout