Hacker News

points by rietta 2 days ago | 0 comments

I am fairly glad I have stuck with stock OpenSSH that ships with Linux. This incident proves I am glad with that choice.

Over the decades, I have maintained remote access through port forwarding on my router - at one point in 2008 I put SSH on port 80 so I could access it from the restricted wifi network at the hospital my grandmother was at. I have also used openvpn. I have also used cloudflare tunnels. But in the end, I always connect to hardened openssh with key only authentication.

[–] selfmodruntime 1 day ago

Your use of OpenSSH in a single private network does not equate in any way or form using and distributing a non-ACL managed SSH client.

[–] rietta 1 day ago

Clearly, but of all the ways to manage authorized keys on a fleet of endpoints was this particular methodology the only or best? Were there other ways to activate and deactivate keys without introducing the same complexity and brittleness. Clearly the ACL methodology provided a false sense of security in this instance in the presence of this bug.

[–] InsidiousWombat 1 day ago

sounds like just adding complexity over traditional method of OpenSSH + key distribution (either via configuration management or ldap)

and now, bugs

[–] selfmodruntime 1 day ago

No, it doesn't sound like it. Tailscale reduces complexity. It literally is key distribution with an ACL.

I work at a finance org with hundreds of engineers and thousands of technical users. You cannot be compliant with finance compliance frameworks using OpenSSH without audits and traceability and some source of identity management. That is exactly what Tailscale (or an alternative product) provides.

> and now, bugs

If you do it yourself, you now own that software entirely. Your comment lacks understanding of the complexity of real world scenarios.

[–] ferngodfather 1 day ago

You just need to implement an SSH CA. You can roll your own. Vault has an enterprise one. StepCA is suitable for smaller orgs.

All the benefit of the core SSH protocol as it's just key signing. It's honestly underrated.

[–] selfmodruntime 23 hours ago

How does having an SSH CA help you roll out detailed access to hosts/ports/services with fine grained permissions via access control lists derived from your identity provider? How do you make sure you're giving out least privileges with a compliant onboarding/offboarding and audit process? That is where the real meat is.

If you've solved all these problems yourself at your org: congrats, you re-implemented Tailscale with a bespoke solution that takes orders of magnitudes more $ to maintain and audit regularly than just buying an industry trusted solution.

Tailscale is, all things concerned, really cheap. It costs less at my large org than a single engineer. Doing it ourselves would take a team. Maybe two. Once again, HM primarily comments from a naive engineer's perspective, not from a leadership view.

[–] nisegami 1 day ago

Adding complexity would imply you're doing both. In this case it would be more correct to say you're trading the complexity of key distribution and access control via keys for the complexity introduced by tailscale ssh.

[–] waynesonfire 1 day ago

yep, this was obvious from day one.

[–] speedstyle 2 days ago

I gather the bug allows people you've explicitly authorized to ssh into some account on your machine, to access more than that account can. So if you only use ssh yourself, or only give ssh keys for users with sudo, or only to people you trust not to exploit this, or if other vulnerabilities allow local shell users to escalate, or if you only use tailscale for port forwarding, then you're unaffected. Arguably even less exposed than a public openssh instance, though I can't say I have any issue doing that

[–] LtWorf 2 days ago

Privilege escalation is now nothing to worry about?

[–] sroussey 2 days ago

Privilege escalation is nothing to worry about on root accounts

[–] eternauta3k 2 days ago

I guess if you have an industrial arm connected then they could escalate to physical access

[–] LtWorf 1 day ago

The "root account" is your assumption now no?

[–] speedstyle 1 day ago

Of course it is, if you rely on that boundary. The 'consumer' use of tailscale is just as a reverse proxy, and that's how I figured GGP sees it. I don't have anyone on my tailnet, and when I did they didn't have ssh keys, and if they did I imagine there'd be many paths to privilege escalation, certainly it's a bigger surface than sshd. So I was clarifying it seems like an escalation from authorized execution to privileged execution, and not e.g. from unauthorized users to RCE

[–] minimaltom 2 days ago

I'm with you- the attack surface of openssh (and my trust in damien) is a better match imo

[–] LeBit 2 days ago

Doing ssh through port 80 or 53 or 443 is immediately flagged.

I understand port 443 is always opened but nowadays you can’t just run ssh traffic on 443 without it being detected as not being https

[–] octoberfranklin 2 days ago

Flagged by who? The Internet Police? This comment is absurd.

[–] novafunc 2 days ago

By restricted networks, like hospitals, workplaces, schools, etc.

If their firewalls can detect it not being typical http/s traffic, they can block it.

[–] QuantumNomad_ 2 days ago

I run WireGuard VPN on port 443 UDP. Many times I can connect to that from various public WiFi networks like coffee shops and public libraries. And then I ssh into my machines at home over my self-hosted VPN.

I’ve been hoping that HTTP3 adoption would make 443 UDP accessible from even more places. But maybe they already do the kind of thing you said for that too, looking at the traffic flow to determine that it is not HTTPS.

Weirdly sometimes there are WiFi networks where I am not able to connect to my WireGuard VPN on 443 UDP, but I can connect to servers over ssh on standard port 22 TCP. Perhaps they sometimes block all UDP traffic because they don’t want people to use BitTorrent etc, but one would think that with all the online meeting platforms that do video streaming there would be a lot of problems for people if all UDP traffic was blanket blocked.

I might eventually try out AmneziaWG instead of vanilla WireGuard, but at the moment I am fine not being able to Connect to my VPN sometimes. I only use it to connect to my other machines anyway, not as a VPN tunnel for my web browsing etc.

[–] teravor 2 days ago

it's possible commercial routers such as the ones utilized by restaurants allow wireguard through these days? wireguard makes no effort to be stealthy when emitting.

some VPN apps bypass wireguard blocks automatically by using https tunnels and whatnot.

[–] Imustaskforhelp 2 days ago

Does mosh help with it? Seems to be based on udp, and it looks good. I am not entirely sure if it helps the issue that you are talking about so hoping for someone more experienced to chime in

https://mosh.org/

[–] rietta 2 days ago

It worked in 2008 - 18 years ago. I have other options now like said cloudflare tunnel.

[–] lmm 1 day ago

A lot of networks still just do dumb port blocking. Yes, any network that is making a serious effort to prevent ssh will succeed. But a lot aren't.

[–] megous 2 days ago

Naive. There's not much that prevents you from running ssh over https. Not even traffic patterns will give you up in the age of ubiquitous websockets and/or http2 streams.

[–] toast0 1 day ago

> Not even traffic patterns will give you up in the age of ubiquitous websockets and/or http2 streams.

SSH is dead simple to detect, because the server sends an outbound greeting in ASCII text; and the client doesn't send anything until it gets the greeting. The binary protocol does not look like a TLS stream at all either.

[–] InsidiousWombat 1 day ago

It will absolutely and trivially be detected by any decent IDS/IPS

SSH does nothing to hide it is SSH (tho "I'm just https" mode would certainly be welcome addition)

[–] sandos 1 day ago

Even with corporate MITM setups? I somehow doubt it..

[–] megous 1 day ago

MITM only works if you allow trust in corporate CA on the outbound HTTPS connections. And why would you?

[–] jon-wood 1 day ago

Its not always the case but many of MITM-ing corporate firewalls are going to immediately drop any TLS connection they can't decrypt on the basis they're coming from an unauthorised device.

[–] LeBit 1 day ago

No need for MITM.

DPI immediately can tell it is ssh.

[–] megous 1 day ago

Yeah, sure. By magic I guess. :)

[–] p_l 1 day ago

No, by DPI. SSH has recognizable signature as does web sockets etc.

Takes flipping one switch for 443 to be made HTTPS + HTTP2 + websocket only by egress firewall if you paid for one with DPI

[–] LeBit 2 days ago

"Naive”. How ironic.

[–] jen20 1 day ago

I think you're likely attributing too much competence to the kind of place that restricts outbound traffic to 80 and 443 in the first place.