Hacker News

points by trjordan 2 days ago | 0 comments

I like the analogy. It feels right. But I don't think it actually is, because it leans hard on the _feeling_ of failure, vs. the actual failure.

The failure that actually matters here is product failure. Angry users. Security incidents. Downtime. Broken workflows.

Stressed and uncomfortable developers have, historically, been a leading indicator of product failure. Software is complex and when devs can't handle the breadth of ownership + the depth of expertise at the same time, it means bad work that leads to product failure. It's not clear that agents fail in the same way, or that our collective intuition about how and when to trust agents is sufficiently developed to predict product failure in the same way.

As anecdata: I know a bunch of growth-SaaS/last-batch-of-IPO companies where the devs are all stressed, the feature ship rate is up, and incidents are down. All at the same time.

So yes, quick learners and just-in-time depth is important, but so is intuition about where discomfort is going to produce true failure. The discomfort isn't a feeling to throw away, simply trusting that you'll learn everything in time. Ownership requires holding a model of the system in your head and proactively steering where you dive.

[–] augment_me 2 days ago

I think that this was true pre-attention economy. If you look at security incidents and other prod issues, they rarely move the needle today. I work at a hyperscalar and we have been shipping 40% more bugs to prod since 2023 year-to-year, yet user growth and revenue is up about 25% similarly due to more features and more ads.

I think that the whole premise of that failure affects things is based on a older model. You kind of assume that:

1) users are able to easily gain information about alternatives. We have seen that since LLMs replaced search engines, we have much less churn.

2) users are not behaviorally addicted/pushed into patterns. We have patterns that utilize users to keep using the app by exploiting their cognitive weaknesses

3) users have the mental fortitude and stamina to learn new tools instead of accepting flaws in the one they are used to

Some failures and bugs in prod don't matters as much

[–] taurath 2 days ago

Almost the entire structure of software oriented companies, and why devs are expensive, are to prevent these sorts of errors. That job has gotten harder, and in some places impossible as management forces more trust on LLMs and reduces the capacity of devs to look. This is less of a problem in existing products in production and existing cultures - revenue is protected. Eventually those structures break down, as does the user perception of quality. Once the brands have taken enough knocks, it accelerates.

[–] matkoniecz 2 days ago

>We have patterns that utilize users to keep using the app by exploiting their cognitive weaknesses

I hope you are aware that you are outright evil. I also hope that you will manage to stop justifying it as acceptable.

[–] pona-a 2 days ago

HN is one of the few sites where someone can launch into an actual villain monologue, the kind you'd read in leaked corporate emails, and receive approximately no negative reaction.

If this is the face of our industry, that might explain a lot of things...

[–] augment_me 2 days ago

You can take the right path and in the end no-one wants to use your product because it's not as engaging.

Instagram tried to take the "good" path in 2014 and lost 90% market share to TikTok who attentionmaxxed things. Then they had to take the exploitative path to claw back users.

As long as it's not regulated for everyone at the same time, you need to do this for survival of your product.

[–] unsui 2 days ago

maybe that product doesn't deserve to survive.

"Hmmm, but if I cap the Torment Nexus, the competing Torment Core will win out, we can't have that!"

[–] augment_me 2 days ago

It's more like it doesn't matter if you sell the most organic, free range, ethically sourced, nutritious, tasty ice cream if your competition is allowed to put cocaine in theirs and get all your customers.

TikTok would survive and do just fine even if Instagram decided to cede the market to them

I am afraid there is a demand for the torment nexus, and until govt steps in, you're out of luck and behavioural surplus is there for the taking by any means.

[–] geye1234 2 days ago

"It's ok that I devote my career to preying on people's mental weaknesses and getting them addicted, because the government isn't regulating it."

I think I'd need to take a zofran if I met you IRL.

[–] matkoniecz 1 day ago

I am well aware that being more evil can be profitable.

Being evil for no gain whatsoever is quite extreme absurdity, though it also happens to some.

[–] unsui 2 days ago

Agreed.

The real alignment issue isn't in the challenges getting AI to align to human values.

It's in humans understanding what is worth aligning to.

"He chose... poorly."

[–] geye1234 2 days ago

Many bad things that people do stem from weakness or inner pain. So it's helpful to be reminded from time to time that some people deliberately, thoughtfully, calmly and systematically devote their lives to harming other human beings -- that is, to evil. We should be grateful to augment_me for this reminder.

[–] trjordan 2 days ago

I agree that not all failures matter (most don't, in fact). On the other end of the spectrum, I know of a bunch of seed/series A startups that basically operate under a model where they ship everything and fix the stuff people complain about. As long as you do that fast enough, the median user experience is pretty good, because only the first n% actually see the problem before a fix is live.

I'll go out on a limb and guess that at a hyperscaler, shipping a bug to prod means hitting canary or 1% of traffic. Seems fine!

I think my point still stands, even as the shape of failures shift. There's certain ones that still hurt (data corruption, exploited security holes, etc.) because they're hard to notice. It might even be that the primary goal of ownership is to develop intuition about where those failures are likely to happen.

[–] ahartmetz 2 days ago

Wait, how do working at a hyperscaler (that's a datacenter provider) and using an addictive app for uhm retention go together? I imagine a gamified AWS provisioning and billing app, but surely that's not it?

[–] ChrisLTD 2 days ago

There are prod issues, then there are issues that cost me money as a customer.

[–] Arkada 2 days ago

This very thing. "Shipment of 40% more bugs and fixing only in response to user complaints" could be done in social streams or advertising tech, but not in finance and trading where margin for errors is close to zero.

A small bug implemented by an agent in the way R-multiples, stop loss distance, or P&L heatmaps are computed will never be complained about by the user—either he will trade incorrectly and lose his money, or see that single off-by-one difference compared to exchange ledger and churn forever.

[–] adrq 2 days ago

> Ownership requires holding a model of the system in your head and proactively steering where you dive.

Absolutely, and the feeling of discomfort is a signal that you need to dive, a signal that your mental models might need updating. I think that builds up over time through many dives