Howtoo SSH over TLS

[Author's Note: I don't actually know anything about fingerprinting technology so if I give you the impression that I am an expert or something, I am not! (Advisory for North Korean wanna-be defects {I don't want them to get deleted thinking they're safe - I don't actually know what fingerprinting technology will say about TLS connections besides "they're TLS but ... X" and I do worry what 'X' could be. But for most people having TLS is completely fine.})]

Many moons ago I when I was away from my desktop I had my laptop and my phone. Perhaps I realized that I wanted to edit something on my computer, so I thought a thought that most people would think: "I'll just use my phone's hotspot and SSH into my computer!"

Unfortunately, this did not work. I remembered to allow SSH on 443 and setup port forwarding since I was behind a NAT for the next time I was out and about.

Bafflingly, this does not work either. It was on port 443! ... I realized that the answer must lie not only with ports but also what the connection looks like because for some reason my phone's ISP is fingerprinting*.

A word about 'modern' TLS/SSL that old guides don't mention because it wasn't a requirement at the time. A cert and a key file is required, if it is not present this will not work unless some software (probably OpenSSL (my current version is 1.1.1l)) is outdated. Here is what I am using, those marked with an '*' are entirely optional and you can use what you want.

I'll assume you are smart enough to figure out if you have these or some reasonable equivalent installed or not. My guide will be based these specifically but I will include 'honorable' mentions. I will also assume you have basic understanding of how *nix operating systems works, how to configure software, and you will at least look at the man pages for each software.

Please note that is does require setting it up before hand; if you are on vacation and are in a similar position as me, you're SOL. Make sure to bookmark this page and revisit it when you get back home, eh? (Also there are A LOT of tcp-over-tls services and I would not be surprised if they were all ran by one person.)

Eas{y,ier} way

You should make a directory where your certs will be. Ideally, only root will be able to read and write in it.

The very first thing to do is to generate the cert or nothing will work and you will get very angry. So, run the following: '# openssl req -x509 -nodes -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 1000'. When it asks for information, in this case, it doesn't matter. You might as well just press enter on all of them. In this instance, we will not verify that the cert is 'valid.' Of course, if you have a real cert along with a real domain name that is issued by a real CA then use that.

Nginx 1.20.1

http {
      ...
}

stream {
       upstream ssh {
               server 127.0.0.1:22;
       }
       server {
               # IP address under the NAT
               listen              192.168.0.68:443 ssl;
               #listen              [::]:433 ssl;
               ssl_certificate     /var/cert/cert.pem;
               ssl_certificate_key /var/cert/key.pem;
               ssl_protocols       TLSv1.2 TLSv1.3;
               ssl_ciphers         HIGH:!aNULL:!MD5;
               proxy_pass          ssh;
       }
}

The above block cannot be in the http block. So, if you have a `sites-enabled/` directory and in nginx.conf `include sites-enabled/*.nginx;` IN the http block, the block above will not work. Why? Because SSH protocol is not HTTP. They are different.

The simplest 'fire-and-forget' way to connect to it (that I've found) would be with socat.

$ ssh -p 443 -o ProxyCommand='socat STDIO OPENSSL-CONNECT:%h:%p,cipher=HIGH,verify=0' 192.168.0.68

At this point your SSH connection should be concealed by TLS on port 443. Fingerprinting will think it is just normal TLS traffic, i.e. HTTPS traffic. With that the guide is over... But typing ssh -o ProxyComannd='...' every time is tiresome! Edit your ~/.ssh/config:

host ssh_tls
        Hostname <IP address>
        Port 443
        ProxyCommand socat STDIO OPENSSL-CONNECT:%h:%p,cipher=HIGH,verify=0
        User <username>

With above you can very quickly ssh like this: '$ ssh ssh_tls'. A lot quicker, eh? (What is this? Hint: '$ man 5 ssh_config')

Honorable Mentions

Socat server: # socat OPENSSL-LISTEN:443,fork,cipher=HIGH,verify=0,certificate=/var/cert/cert.pem,key=/var/cert/key.pem TCP-CONNECT:localhost:22

Socat client: $ ssh -p 443 -o ProxyCommand='socat STDIO OPENSSL-CONNECT:%h:%p,cipher=HIGH,verify=0' 192.168.0.68

Socat (1.7.4.1) is an honorable mention because I do some web-development and turning off Nginx/Apache was not really an option. The verify option can be removed if you have a real cert that isn't self-signed. Socat is an interesting piece of software and can do many other things, not just this.

Stunnel (5.60)

pid=/tmp/stunnel-ssh.pid
[ssh]
client = yes
accept = 9048
connect = 192.168.0.68:443

Stunnel is a little confusing to understand at first but after you understand it, it becomes 'ezpz.' Basically, you set up stunnel first, which binds to a port and stunnel forwards everything that goes into that port to go to a specific location with TLS. So in this case everything on *:9048 should be encrypted with TLS and go out to 192.168.0.68 at 443. Don't forget to turn off/kill stunnel after you finish using it. (Check the reference for stunnel to get a 'fire-and-forget' script.)

Apache (2.4.51) and proxytunnel (1.10.20210128)

# Change IP to yours under the NAT
<VirtualHost 192.168.0.68:443>
       SSLEngine on
       SSLProtocol -all +TLSv1.2 +TLSv1.3
       SSLCertificateFile    /var/cert/cert.pem
       SSLCertificateKeyFile /var/cert/key.pem

       # Normally you want this turned OFF
       # otherwise you are an open relay
       ProxyRequests on
       # Only allow port 22
       AllowConnect 22
       <Proxy *>
               # Disallows proxying for everything...
               Require all denied
       </Proxy>
       <Proxy 127.0.0.1>
               # But allows proxying to 127.0.0.1
               Require all granted
       </Proxy>
</VirtualHost>

Make sure you have enabled: ssl_module, proxy_module, proxy_connect_module. Additionally, make sure you actually have a `Listen 443` somewhere. Turn Apache on or restart it.

Try this: '$ ssh -p 443 -o ProxyCommand='proxytunnel -z -E -p %h:%p -d 127.0.0.1:22' 192.168.0.68'. If it works, it works and you should test if Apache will allow proxying to other locations (it should not and if it does you did something very wrong). If it results in an error, check if Apache is turned on, then check Apache's error logs.

The difference between connecting to Nginx or Apache is how the proxy pass is done. Nginx proxy passes to 127.0.0.1 automatically (reverse proxy) and with Apache you tell the server that you want to proxy to 127.0.0.1 on port 22 (forward proxy [admittedly, only to itself]). I'd like to reverse proxy with Apache but Apache is best suited as a web server rather than a proxy server. Don't use a spoon as a fork - kind of thing.

Now, if I had many services that I wanted to offer but only had port 443 I'd use forward proxying which Apache and/or Nginx (with some difficulty I must mention) can do.

OpenSSL (1.1.1l).

$ ssh -p 443 -o ProxyCommand="openssl s_client %h:%p" 192.168.0.68

Similar to socat but less typing I guess. It is an honorable mention because if a packet is bad then it gives up entirely and I haven't figured out how to tell it to ignore them. If you look at the man page '$ man s_client' you might notice a `-reconnect` but that does not do what you think it does. It is also somewhat noisy.

Closing thoughts

Brief thoughts on SSH over TLS: double encryption with TLS and default SSH to me seems like wasted CPU cycles. If possible it maybe better to use something like OpenVPN or WireGuard but not every phone will have that and it is a question if a device connected to hotspot will even be able to connect to other devices with OpenVPN or WireGuard because connections are being fingerprinted. With this, it is a guarantee that you will be able to connect to your computer, at least until fingerprinting software figure out how to do more privacy invading things. Also use ssh-keys with SSH and disallow passwords.

I suppose one could use rlogin or telnet with TLS (trivial with with stunnel) to reduce the CPU cycle waste but I won't write the guide on that - I don't want to be responsible for anything bad happening (since rlogin and telnet send things plainly) and it doesn't seem like that big of an overhead anyway. (What? Your computer can't handle TLS and SSH at the same time or something? lol! Stop using a 3rd world computer already. ... Although if it is that old then it probably isn't back doored... Hmm...)

If you have a spare computer (under the same NAT) do the same commands and if it works then you're golden. If it does not work then obviously it won't work when you use hotspot. I'd avoid connecting with the hotspot if you are unsure if you set it up correctly - when they disconnected me I couldn't connect back to my home IP address. I waited for about a day for the block to expire (not that I was constantly testing).

Jake, you are a liar and a techlet! This will not work!

Compare these files then, please. Wireshark is pretty good at it.

If you see TLS inside the NAT you will see TLS outside the NAT as well, meaning hotspot will work. If you are on the technical side you will be able to see how you can configure a lot of this in somewhat different ways. Now, if you are exposing your stuff to the outside world it goes without saying: make sure your stuff is protected/hardened/hard-to-break-into.

Other software can do proxying as well, like Squid, but I chose to focus on Nginx and Apache since I actually use them on a daily basis.

I compiled this from various sources so future me will not have to spend hours tweaking search phrases.

This post would not have been possible without the following resources:

Disclaimer: * = When I was able to connect to my computer for like 5 seconds then suddenly it would no longer work my first thought was "THEY'RE FUGGING ME!" and caused me to write this blog post in mainly because I wanted to fugg them back (my reasoning was that they could block SSH but they won't block TLS). I don't know what happened but I can SSH into my computer with hotspot. I am not sure what to make of this.