Non-Interactive SSH Session Does Not Terminate – SSHD Process Waits Forever

linuxssh

We run a simple deployment script remotely using a command like ssh deployer@10.170.4.11 sudo /root/run-chef-client.sh. It started to hang today because sshd waited forever on the 10.170.4.11 even after sudo had finished already. We started sshd in debug mode and got two different kind of logs. The following is a normal log when the session does not hang:

debug1: Received SIGCHLD.
debug1: session_by_pid: pid 23187
debug1: session_exit_message: session 0 channel 0 pid 23187
debug1: session_exit_message: release channel 0
Received disconnect from 10.170.4.6: 11: disconnected by user

And when it hangs we get the following:

debug1: Received SIGCHLD.
debug1: session_by_pid: pid 24209
debug1: session_exit_message: session 0 channel 0 pid 24209
debug1: session_exit_message: release channel 0

Our understanding is that the server process waits for some communication from a client side and never gets it. It's hard to tell if it is a client side or a server side problem.
We tried to run sshd under strace but did not succeed because a SUID bit on sudo was ignored it this case. So, what else should we try to debug/prevent this situations?

Best Answer

Using ssh -t (forced PTY allocation) on a client side solved the problem:

debug1: Received SIGCHLD.
debug1: session_by_pid: pid 31701
debug1: session_exit_message: session 0 channel 0 pid 31701
debug1: session_exit_message: release channel 0
debug1: session_pty_cleanup: session 0 release /dev/pts/1
Received disconnect from 127.0.0.1: 11: disconnected by user
debug1: do_cleanup
debug1: PAM: cleanup
debug1: PAM: closing session
debug1: PAM: deleting credentials

sshd is controlled by a pseudo TTY not by a client anymore.

Related Solutions

Ubuntu SSH Login – Permission Denied, Please Try Again

Are you certain that the user account you're attempting to access is correctly configured? If you log in as root on the system, can you su to the user account?

# su - username

What do you see in your logs after a failed connection attempt? On many systems, sshd will log to something /var/log/secure or /var/log/auth.log. Also, I note that you have PasswordAuthentication enabled but ChallengeResponseAuthentication disabled. Do you see the same behavior if you enable ChallengeResponseAuthentication?

Here are some general diagnostic steps to use when you have ssh problems:

Enable verbose diagnostics in ssh:
```
ssh -v host.example.com
```
This will cause the client to output a variety of diagnostic messages as it negotiates the connection. This will often provide a clue to the problem.
Run the server in debug mode.

On your server, stop sshd, then run it from the command line like this:
```
/usr/sbin/sshd -d
```
This will produce verbose debug logging on stderr that will very often contain useful information.

If neither of these helps you figure out what's going on, would you add the output to your question?

SSH-forwarded X11 display from Linux to Mac lost after some time

The ssh sessions started after I changed the Mac client's /etc/ssh_config to include the line:

ForwardX11Timeout 596h

are all working fine and have been all day. By now they all would have been refusing to start new xterms. So I believe this is the answer, and luckily a simple solution, but the timeout will still happen 3-1/2 weeks from now.

Best Answer

Related Solutions

Ubuntu SSH Login – Permission Denied, Please Try Again

SSH-forwarded X11 display from Linux to Mac lost after some time

Related Topic