8 ms·
How to kill an unresponsive SSH session
- hahainternet 13y agoIf you happen to be a few sessions deep, ~~ will send ~ to the next session along. A casual ~~~~~~~~~~~. or so later and everything is wonderful again!
- oarmstrong 13y agoThanks for the tip! I don't tend to nest sessions more than two deep but this could be useful for those who have deep nestings.
- makomk 13y agoThough you have to be nested really deeply to do ~~~~~~~~~~~. - unlike some other escaping schemes, it only requires one extra tilde for every layer of nesting rather than doubling each time.
- hahainternet 13y agoThe worst problem is trying to figure out just how many layers deep you are. I was being a little ridiculous though.
- kps 13y agoThe Telnet Song from CACM, April 1984: http://delivery.acm.org/10.1145/1040000/1035691/p347-steele.pdf http://delivery.acm.org/10.1145/1040000/1035691/p347-steele.... Plain text at http://alcor.concordia.ca/~smw/home/telnet_song.html http://alcor.concordia.ca/~smw/home/telnet_song.html and no doubt elsewhere. (This does not refer to the Unix version of telnet, which defaults to ^] as the default escape character.)
- oarmstrong 13y agoIn case anyone is having difficulty with the font used on the page, the escape sequence is: newline followed by tilde (~) and then period (.).
- Sharlin 13y agoThe default escape character ~ does not work if the tilde key in your keyboard layout is a dead key [1], like it is in many European layouts. It can be changed via the EscapeChar config option or the -e command line parameter. It seems, though, that not just any old character is accepted - I tried to use §, which, in the Finnish layout, is in the same physical position as ~ is in the US version, but ssh complains about "bad escape character". EDIT - I suppose it must be an ASCII character, which is not an entirely unreasonable requirement. [1] http://en.wikipedia.org/wiki/Dead_key http://en.wikipedia.org/wiki/Dead_key
- lzm 13y agoThis must be why mosh's escape sequence never worked for me (ctrl-^ or ctrl-^.)
- hansjorg 13y agoSeems to be fixed (to use the same as SSH): https://github.com/keithw/mosh/issues/215 https://github.com/keithw/mosh/issues/215
- cpa 13y agoI use a french mac layout and it works well. But you have to type ~ then press space, then press shift+; (because shift+; is .)
- cnvogel 13y agoIt works, you'll just have to type <tilde> <space> .
- Sharlin 13y agoWeird - I did try that before writing the comment and it didn't work. May depend on the terminal or something.
- rlvesco7 13y agoYou can change what the default escape character is in the ssh config file.
- adaml_623 13y agoAnother useful trick to remember if you're using Putty and you ever accidentally hit Ctrl-S and find that you've frozen the terminal. Just type Ctrl-Q and you will unfreeze the connection. Credit due to: http://raamdev.com/2007/recovering-from-ctrls-in-putty/ http://raamdev.com/2007/recovering-from-ctrls-in-putty/
- ojbyrne 13y agoThis is not news to anyone (in tech) over 40. https://en.wikipedia.org/wiki/Software_flow_control https://en.wikipedia.org/wiki/Software_flow_control
- delinka 13y agoBut the world is full of neophytes, re-inventors, etc. It's good to remind people periodically and educate the next generation.
- ojbyrne 13y agoOf course. I was actually just expressing a feeling of nostalgia.
- nkurz 13y agoAnd besides, the older I get (41) the more I can use frequent reminders of the things I used to know. It's disturbing the number of times I've searched the web for answers, and discovered an answer I wrote myself. And I hadn't known this could be easily turned off with 'stty' as suggested in another answer. Or maybe I did once know that?
- delinka 13y agoThe following quite becomes more relevant every year: "I've forgotten more than you'll ever know!" --Insulting Old Fogey
- emmelaich 13y agoThat wiki page doesn't mention tilde does it? I know tilde escapes from cu(1) (part of uucp) http://www.delorie.com/gnu/docs/uucp/cu.1.html http://www.delorie.com/gnu/docs/uucp/cu.1.html But it may well have originated before then.
- spindritf 13y agoOr use mosh[1] on top of SSH and stop worrying about that stuff. It works much better over high-latency links (mobile). It is not bothered by saturated links, tolerates IP changes and losing the underlying connection like when you suspend your laptop and take it elsewhere. I now have mosh connect to several servers in tabs when I run gnome-terminal the first time, and only disconnect on reboot. I also run a mosh-capable Irssi Connectbot fork on the phone[2]. It's a massive improvement, fixing many of the little annoyances of ssh. [1] http://mosh.mit.edu/ http://mosh.mit.edu/ [2] http://dan.drown.org/android/mosh/ http://dan.drown.org/android/mosh/
- oijaf888 13y agoHas anyone audited mosh's security model/encryption? Just curious since ssh is pretty tried and tested.
- qznc 13y agoAuthentication and initial key exchange is via ssh, so nothing to audit here. Afterwards data is sent AES-encrypted, which relatively simple.
- darkarmani 13y ago> Afterwards data is sent AES-encrypted, which relatively simple. It's simple to encrypt using AES, but that doesn't mean it is simple to encrypt in a secure way (i'm looking at you ECB mode). There are too many ways to accidentally mess up.
- ReidZB 13y agoIt uses ssh to set up the connection, so that setup is fine. The actual traffic is encrypted using AES-128-OCB, which is a great mode of operation. I haven't digged too deeply into their security, but from what the main page says, it seems pretty trustworthy. Hopefully they're using the appropriate authenticated data, with packet numbers and so forth, to prevent silly things like replay and reordering attacks.
- gbog 13y agoDoes it work too with ctrl-\ ? This has been my process killer recently and it's powerful (and the only way I know to get out of xtail.)
- Nursie 13y agoenter-tilde-dot It is useful, yes. Here's another thing I picked up last week - how do you reboot a remote linux box that's somehow lost its root drive but you still have a shell open (because you left ssh running on another machine)? echo 1 > /proc/sys/kernel/sysrq echo b > /proc/sysrq-trigger
- deleted 13y ago[deleted]
- Nursie 13y agoThe machine in question has a zfs pool that it would be nice to sync, and a MySQL database running off one of the filesystems on that pool. So yes, all of this would be lovely. But will it work? You're sitting on an ssh session that's just about hanging on. There's no other way into the system (it's remote). As soon as you issue e (or definitely i) the shell dies, the ssh session dies and your reisub job gets killed, no? I tried this: echo e >/proc/sysrq-trigger; echo b >/proc/sysrq-trigger; but what I described above happened, and the box did not reboot. (--edit-- deleted comment above had a perfectly reasonable explanation of a safe shutdown using the "Magic SysRq" stuff, but it applied to a machine you're at locally rather than the b0rked remote box I described)
- Hello71 13y ago$ killall mysqld $ while pgrep mysqld; do sleep 5; done # waiting for mysql to shut down $ echo s >/proc/sysrq-trigger $ watch dmesg # wait for "sync complete" msg $ echo b >/proc/sysrq-trigger # or if you can force-reboot remotely $ echo o >/proc/sysrq-trigger # /proc will no longer be mounted after this
- Nursie 13y agoThis would reboot it but might not achieve much- $ killall mysqld /usr/bin/killall - file not found $ while pgrep mysqld; do sleep 5; done /usr/bin/pgrep - file not found $ echo s >/proc/sysrq-trigger $ watch dmesg /usr/bin/watch - file not found $ echo b >/proc/sysrq-trigger Unfortunately in this situation only the bash builtins seem to work. I even had an old system drive mounted under /altroot, but it wouldn't execute any of the commands in /altroot/bin or /altroot/usr/bin either (can't find something it needs to execute stuff. ld? libc? unknown). So really only the 'b' seems to help. (edit: 's' is probably useful, even if you can't kill everything) Of course the long term solution is to stop using a USB stick on an unreliable/badly supported USB3 add-on card for the root drive :)
- kdazzle 13y agoAnother great solution is to just use the ServerAliveInterval option.
- XorNot 13y agoI can never decide how I want to set this. Most sites recommend 180 seconds or so, but with the default ServerAliveCountMax set to 3, this is 9 minutes before a dead terminal is actually disconnected. I've started to set it really really low personally - like, 5 seconds, so the connection drops after 15 seconds. I'm tempted to go down to 1, but I have a lot of long running sessions and I start to worry about the traffic counts. But, after all this there's still a problem: none of it seems to work with any of the connection mux'ing options - once the background session dies, I still have to manually kill it to get anything working again.
- deleted 13y ago[deleted]
- epo 13y agoPlease don't let HN become a substitute for RTFM. This should be known by all SSH users who have skimmed the man page. Fair enough as a blog post but for this trivia to get 46 points so far is deeply depressing. Maybe I should write a blog post about the use of CTRL-Z in the shell and post that here, should get me Kilo-karma points if this is anything to go by.
- peterwwillis 13y agoIt should also be known by anyone who has used Telnet, since that's where it originated. (For more shortcuts, use '~?' from the ssh session)
- __david__ 13y agoNot any telnet I've ever used. Telnet is ^] to get to the command prompt and then "quit". It even says so right when you connect: $ telnet google.com 80 Trying 74.125.239.103... Connected to google.com. Escape character is '^]'.
- gpvos 13y agoApparently it's rsh, not telnet. The telnet manual does mention it though, since you can put telnet in a rsh-like mode where ~ does work. (May depend on telnet version.)
- peterwwillis 13y agoAh, my bad! Guess i'm older than I thought...
- hhw 13y agoman 1 telnet on FreeBSD or OpenBSD contains: "When in rlogin mode, a line of the form ~. disconnects from the remote host; ~ is the telnet escape character. Similarly, the line ~^Z suspends the telnet session. The line ~^] escapes to the normal telnet escape prompt." although on the Debian man page, it appears slightly differently as: "-r Emulate rlogin(1). In this mode, the default escape character is a tilde. Also, the interpretation of the escape character is changed: an escape character followed by a dot causes telnet to disconnect from the remote host. A ^Z instead of a dot suspends telnet, and a ^] (the default telnet escape character) generates a normal telnet prompt. These codes are accepted only at the beginning of a line. " telnet is not installed by default on CentOS, but has the same man page as on Debian.
- jerf 13y agoRead the SSH man pages every so often, even if you think you know how to use it. There's a lot of features in there. Don't miss the "AUTHORIZED_KEYS FILE FORMAT" in sshd's man page for the uber-cool "command='command'" options for authorized keys (restricts a given key to just be able to run a certain command, very useful). See also SSH's port forwarding, -D, learn how to use ssh-agent, and "man ssh_config".
- voltagex_ 13y agothe command= syntax is (was?) how gitolite works - I'm not sure what the difference between that and a jail is, though.
- stormbrew 13y agoThey're not really related concepts. A jail is allows you to run arbitrary things in a restricted environment while ssh's command feature just forces a particular command to run no matter what's passed. If your authorized_keys line for a key has command="blah" on it and you go: ssh server ls the server will, rather than running ls as it normally would, run blah and pass the command you specified in as an environment variable. Then blah can parse it and do whatever it wants with it. You could use command to establish a jail or virtual environment of some sort in which to run the program specified if you wanted to.
- MrDOS 13y agoA true BSD chroot jail restricts the user's mobility around the file system, effectively limiting them to running only what binaries have been explicitly placed within that jail. It means, among other things, that the fork hacks that command= is vulnerable to (e.g., vim running other programs) don't work because non-permitted programs basically don't exist as far as the user can see. The downside to jails is that, without automation, they're tedious to maintain, and disk overhead is not insignificant as you have to have a duplicate set of system binaries for each user. In practice, chroot jails end up being very similar to OpenVZ “virtualization” on Linux.
- 13y ago
- kbenson 13y agoWhile I've frequently used this to kill connections, my favorite thing I've done with it is to list existing and dynamically add new forwarding ports through SSH.
- tankenmate 13y agoThe thing that amazes me about this is that people don't realise that this comes from BSD 4.2 rsh released in 1983.
- revscat 13y agoWhy does that amaze you? That is a pretty trivial piece of knowledge. Interesting, though.
- verbatim 13y ago<enter>~Ctrl-Z will suspend the ssh session, too. I've also found it useful to do <enter>~C - then you can configure port forwarding without having to open a new ssh session. (~C opens a command line, enter "help" for available commands.)
- microcolonel 13y agoI'm surprised/apalled at how many people upvote this, considering how this place is supposed to be "hacker news"...
- lobo_tuerto 13y agoSometimes I upvote articles where the comments are good, present new information or just have better information than the article itself.
- badmadrad 13y agoyou really don't HAVE to know this to be a good "hacker" That being said, some people are new to the game. No reason to be elitist.
- spudlyo 13y agoHitting '.' at a prompt used to be a common idiom for exiting a program. I first saw it when I was a kid working on an HP-3000 system where the system programming language was BASIC and all programs followed this convention. Don't know where it came from originally, but you can still see it in places like rsh/SSH etc.
- smutticus 13y agoAnd SMTP where a single "." followed by CR ends the session.
- jlkinsel 13y agoI'm a little surprised this is on HN? To me this is the equivalent of a blog post about using %d with printf. Not complaining, just a little surprised something so novice would get attention...
- recursive 13y agoYou should post it if you think it will gain traction. Not everyone is experienced with C.
- bostonvaulter2 13y agoThis usually doesn't work for me. Perhaps it's because my sessions are usually multiplexed via "ControlMaster auto"?
- Nick_C 13y agoOn a side note, does anyone know what ~B actually does? Does it send a SIGINT to the remote terminal? What does ssh mean by the phrase "send a BREAK to a remote system"? I've tried to use it without success to kill a runaway listing of megabytes of scrolling text, but frantically hitting ctrl-C seems to work much better.
- thristian 13y agoIn the beginning, there was RS232, where each character was a fixed number of bits, optionally with a stop bit and a parity bit. If the sender transmitted too many 0 bits to represent a valid character, that protocol failure was called a 'break', and some receiving equipment would detect such a failure and do something about it, like reset itself to a known-good state. Thus, a 'break' was occasionally a useful thing to send, so sending equipment would often have a special keystroke to cause a 'break' condition. The Unix 'tty' subsystem was basically designed to support simple serial terminals, and so it had a bunch of behaviour designed to interoperate with the pre-existing 'break' conventions. If a Unix system's serial port received a break, Unix would (optionally) send any processes running via that connection a SIGINT, to represent the 'reset to known-good state' behaviour (this is controlled by the stty command's 'brkint' flag). Also, if the user's terminal didn't provide a specific 'send a break signal' command, Unix could be configured to send a break signal when it received some particular character (^C by default; this is controlled by stty's 'intr' setting). Of course, nobody uses physical RS232 terminals anymore, but for compatibility reasons the Unix tty API lives on, and the "psuedotty" implementation used for things like terminal emulators maintains compatibility. ssh is basically a tool for exporting the tty API over the network, and so for compatibility it too must have a way to transmit the information "pretend a break condition has occurred on the RS232 connection we're pretending to use." To summarise: yes, unless you've messed with the stty command, ~B will probably result in a SIGINT. Ctrl-C is probably more reliable since you can hit it much faster than you can type <Enter>~B, but ~B is still useful if your terminal is in 'raw' mode, where ^C is not converted to SIGINT (for example, if you're running an app that wants to bind ^C to some other function). See also: https://en.wikipedia.org/wiki/Universal_asynchronous_receiver/transmitter#Break_condition stty(1) tcsendbreak(3)
- anuraj 13y ago[Enter] Shift+~ .