Problem ssh:ing into a Linux machine

Go To Last Post
14 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hello,

 

I have a device running a Linux system that I am having issues SSHing into. It keeps refusing all of my attempts. I can connect to the device through a serial cable and do some debugging.

 

On the client side I did a tcpdump and got the following when trying to SSH into the device:

11:33:35.205600 IP [my computer] > [the_device]: Flags [S], seq 2666683521, win 64240, options [mss 1460,sackOK,TS val 2735211423 ecr 0,nop,wscale 7], length 0
11:33:39.237315 IP [my computer] > [the_device]: Flags [S], seq 2666683521, win 64240, options [mss 1460,sackOK,TS val 2735215455 ecr 0,nop,wscale 7], length 0
11:33:47.430088 IP [my computer] > [the_device]: Flags [S], seq 2666683521, win 64240, options [mss 1460,sackOK,TS val 2735223648 ecr 0,nop,wscale 7], length 0
11:34:01.322396 IP [the_device] > [my_computer]: Flags [R.], seq 816065780, ack 747600049, win 8192, length 0
11:34:03.557950 IP [my computer] > [the_device]: Flags [S], seq 2666683521, win 64240, options [mss 1460,sackOK,TS val 2735239776 ecr 0,nop,wscale 7], length 0
11:34:32.235436 IP [the_device] > [my computer]: Flags [R.], seq 719236264, ack 2666683522, win 8192, length 0

So it keeps sending the same package with seemingly no response and then the device seems to reset the connection...!? The on-board Linux does not have tcpdump installed so I cannot get the same log from there. However, I can see that the SSH process is running and with netstat I can see that it is listening on port 22:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN  

When trying the command "ssh testuser@[ip-address]

It simply responds with

connect to host [IP-address] port 22: Connection refused

On the device if I try "telnet localhost 22" I get

Connection closed by foreign host 

and "lsof -i" 

COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd      299 root    3u  IPv4   1279      0t0  TCP *:ssh (LISTEN) 

 

I'm utterly confused at this behavior. Clearly the ssh daemon is running - why is it not responding to anything?! 

1010001010111101110111

Last Edited: Tue. Nov 23, 2021 - 11:01 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is your host SSH configured such that the client must supply a previously exchanged key ?

 

Usually a new key gets "doled out" to all newcomers but this isn't exactly secure. Perhaps the default behaviour has changed.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Have a look at /var/log/auth.log on the server that is running sshd. I see stuff like this (after I just SSHd into it to look at the log!):

 

Nov 23 11:15:01 lxd0308u CRON[13704]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 23 11:15:01 lxd0308u CRON[13704]: pam_unix(cron:session): session closed for user root
Nov 23 11:17:01 lxd0308u CRON[13708]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 23 11:17:01 lxd0308u CRON[13708]: pam_unix(cron:session): session closed for user root
Nov 23 11:18:38 lxd0308u sshd[13711]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=172.19.249.58  user=itsme
Nov 23 11:18:38 lxd0308u sshd[13711]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=172.19.249.58 user=itsme
Nov 23 11:18:38 lxd0308u sudo:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/opt/continental/bin/lib.sh pam_script
Nov 23 11:18:38 lxd0308u sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
Nov 23 11:18:38 lxd0308u sudo: pam_unix(sudo:session): session closed for user root
Nov 23 11:18:38 lxd0308u sshd[13711]: pam_listfile(sshd:account): Couldn't open /etc/ssh.server.user.allowed
Nov 23 11:18:38 lxd0308u sshd[13711]: pam_listfile(sshd:account): Refused user itsme for service sshd
Nov 23 11:18:38 lxd0308u sshd[13711]: message repeated 2 times: [ pam_listfile(sshd:account): Refused user itsme for service sshd]
Nov 23 11:18:38 lxd0308u sshd[13711]: Accepted password for itsme from 172.19.249.58 port 55331 ssh2
Nov 23 11:18:38 lxd0308u sshd[13711]: pam_unix(sshd:session): session opened for user itsme by (uid=0)
Nov 23 11:18:38 lxd0308u systemd-logind[1967]: New session c11 of user itsme.
Nov 23 11:19:07 lxd0308u sudo: itsme : TTY=pts/0 ; PWD=/home/itsme ; USER=root ; COMMAND=/bin/cat /var/log/auth.log
Nov 23 11:19:07 lxd0308u sudo: pam_unix(sudo:session): session opened for user root by itsme(uid=0)

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Mithrandir_ wrote:
It simply responds with

connect to host [IP-address] port 22: Connection refused

Is the  Linux system set up to block access on port 22 ?

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


OK so I wondered if you could telnet to the SSH port (22) to actually interact with the SSH server. So at first I tried:

 

 

Telneting in to 22 prompted it to respond "SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3". At this stage I did not know how to respond so I typed "just testing" which it did not expect so it closed the connection.

 

So then I read:

 

https://stackoverflow.com/questi...

 

which took me to the RFC for SSH:


http://www.networksorcery.com/en...

 

From which I learned that a possible response was "SSH-2.0-billsSSH_3.6.3q3<CR><LF>". So I tried that:

 

and it looks like I have started to have a conversation with the sshd. 

 

If you were patient you could read the RFC and learn how an entire SSH dialog might proceed and try each step to see if you can see at what point the SSH is rejecting your approach.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thank you all for your input very much. I will go through your suggestions one by one and report any progress - however I am going home for today so please do keep an eye out tomorrow morning/afternoon. Thanks again.

1010001010111101110111

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I just re-read this thread. First time through I missed the fact you had said:

Mithrandir_ wrote:

On the device if I try "telnet localhost 22" I get

Connection closed by foreign host 

in which case it does look like a port 22 access problem and is probably caused by a firewall. Maybe try:

sudo ufw status verbose

If it is active and denying port accesses try:

sudo ufw allow ssh

and see if that helps.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


It's inactive on my vanilla Mint installation; I've never had cause to use it.

 

 

Neil

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Mithrandir_ wrote:

When trying the command "ssh testuser@[ip-address]

It simply responds with

connect to host [IP-address] port 22: Connection refused

On the device if I try "telnet localhost 22" I get

Connection closed by foreign host 

 

I didn't used Linux in decade.But seems the port 22 didn't open.

 

Nevertheless We already entered the Moria long time ago.laugh

www.tokopedia.com/madagang .Buy and Donated cheap electronics and manuscripts.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So I'm returning to this today. I had some other fires to put out last week so please excuse the delay. I have made some progress. I am able to ssh into the machine using Bitvise now. It turns out that another engineer had connected to my device and messed around with the autorized_keys file. However, I am still unable to ssh from my Linux workstation. I need to be able to use the terminal to ssh into the device because later on we will be using shell scripts to update and do various other things to these devices. It will not be possible to manually do it through bitvise for every device.

 

N.Winterbottom wrote:

Is your host SSH configured such that the client must supply a previously exchanged key ?

 

Usually a new key gets "doled out" to all newcomers but this isn't exactly secure. Perhaps the default behaviour has changed.

 

 

Yes - it is. I have copied the private key over to my workstation and edited the /.ssh/config file appropriately. I have also edited the key file to "chmod 600". Still nothing.

 

Host instrument
    HostName 192.168.137.2
    User testuser
    IdentityFile ~/.ssh/ssh_instrument_0001.priv
    Port 22

alas, the response is the same:

ssh -v instrument

OpenSSH_7.6p1 Ubuntu-4ubuntu0.5, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /home/westman/.ssh/config
debug1: /home/westman/.ssh/config line 1: Applying options for instrument
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.137.2 [192.168.137.2] port 22.
debug1: connect to address 192.168.137.2 port 22: Connection refused
ssh: connect to host 192.168.137.2 port 22: Connection refused

clawson wrote:

Have a look at /var/log/auth.log on the server that is running sshd. I see stuff like this (after I just SSHd into it to look at the log!):

 

The system is built with buildroot so the structure of the files is a bit different than a normal linux system and I do not have this log file.

 

I think we can narrow this down to some problem with my Linux workstation as I am able to ssh into the device using Bitvise. Regarding your followup post using telnet - my issue is that I get stuck at "Trying <ip address>". I never reach the point where I can start talking to the server. I simply get the response:

Trying 198.168.137.2...
telnet: Unable to connect to remote host: Connection refused

 

The firewall is set up to allow ssh connections - as confirmed by the fact that I can ssh into the device using bitvise. ufw is inactive on my workstation.

 

So clearly the issue is on my client side - not the host side. Does anyone have any more clever ideas?

1010001010111101110111

Last Edited: Mon. Nov 29, 2021 - 04:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well your own keys will be in ~/.ssh so just make sure your public key is registered with sshd on the server.

 

Explore ssh-copy-id

 

For example see: https://www.ssh.com/academy/ssh/copy-id 

Last Edited: Mon. Nov 29, 2021 - 04:15 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:

Well your own keys will be in ~/.ssh so just make sure your public key is registered with sshd on the server.

 

Explore ssh-copy-id

 

For example see: https://www.ssh.com/academy/ssh/...

 

It doesn't work - I have tried it previously. It would also not work company wide. The host has to accept connections from anyone with a specific private key.

1010001010111101110111

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0


What do you see if you run:

sudo service ssh status

on the host - does it report "active (running)" ?

 

Also what do you see for:

sudo netstat -ltnp | grep sshd

On a machine where things are working I see:

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Maybe check the config files in, e.g., /etc/ssh, exp sshd_config.   On my embedded device, which I ssh to using keys, I had to make a change from the default:

device:/etc/ssh# diff sshd_config*
33d32
< AuthorizedKeysFile     %h/.ssh/authorized_keys
40c39
< HostbasedAuthentication yes
---
> HostbasedAuthentication no
device:/etc/ssh#