Hacker News

points by Tiberium 1 day ago | 0 comments

"No LLMs for finding bugs." I really don't get rules such as these, plus they're not really enforceable. There's a difference between using the LLM to write the bug report and using the LLM to find the bug (if you suspect there is one) to then report after manual checking, in human words.

[–] nine_k 1 day ago

Go ahead, use all the LLMs in the world for finding bugs! It's not different than finding bugs by fuzzing, sitting with gdb, or receiving a revelation in your dream.

But then use your own brain to understand the bug, its implications, maybe ways to fix it. Again, use whatever tools, as long as your mind forms a clear understanding, which you can operate on independent on any LLMs or whatever.

The problem with "LLM bug reports" is that the submitters run a machine tool to find something that looks like a bug, but have a poor / no idea themselves what the issue is, and whether it's really an issue. It's easy to flood maintainers with such reports. The "no AI" policy pushes the responsibility of doing proper assessment onto you, the submitter.

If you can use LLMs to produce a bug report with the quality of assessment equal or above that of a diligent qualified human bug reporter, godspeed.

[–] sigmar 1 day ago

>Again, use whatever tools, as long as your mind forms a clear understanding, which you can operate on independent on any LLMs or whatever.

that isn't what the policy says. It isn't just about writing the report.

> If you use a chatbot to give you advice on a comment on the issue tracker, that comment is unwelcome.

[–] nine_k 1 day ago

I assume the policy was written by rational, reasonable people, with a practical goal in mind: to prevent "AI slop", that is, dubious-quality, high-volume contributions by people who don't exactly understand how their PR works, and how the project works, but "computer says yes" because tests (which they also do not understand well) seem to pass.

If a submission (PR, bug report) is high-quality, and the person behind it is responsive and clueful, so that the useful parts of the submission can be taken, and problematic parts, fixed, then it does not matter what process has produced the code. It could as well be a Boltzmann brain [1].

Given that the author of the contribution actually got a grip on the problem, and on the solution, the limitation of "no AI use" is no longer load-bearing. Not only might it have been counterproductive — it's also unverifiable in practice. It becomes absurd, akin to a demand that the text must have been edited by Vim and not VSCode. It must be noted that a misguided policy may indeed force the contributor to delve in the depths of the process of PR creation, and showcase every manually undertaken step in an attempt to prove that no AI was involved. Ironically, it is no secret that many less scrupulous participants might leverage AI to imitate just that. It must be noted though that the decision ultimately belongs to the project maintainers.

(The above paragraph was hand-crafted for your and my entertainment.)

In short: by default I respect the authors of the policy and hence assume that they enact the policy for reasons that are rational, not religious.

[1]: https://en.wikipedia.org/wiki/Boltzmann_brain

[–] bilinguliar 1 day ago

Without a policy, I refuse to have a clear understanding!

[–] GuB-42 1 day ago

Then it should be clarified.

To me "no LLMs for finding bugs" means what it means, which implicitly means "we prefer to have bugs than for you to use a LLM". It is weird, may result in worse code, but it can be justified for ideological reasons, and I mean, why not, it is your project, your rules.

In the unlikely even I contribute one day, reading this rule, I will turn off my LLMs completely and my submissions will probably have bugs that could have been avoided using a LLM based debugging pass, but that's the rule, and I won't discuss it because it is also the rule.

If the intent is to avoid flooding maintainers with bug reports, say "no LLMs for bug reporting".

[–] BoorishBears 1 day ago

That's not my read, at least not if you give them enough credit to use their software.

My read is any LLM use should be opaque to them. You should be able to explain your bug find in your own words, have formatted as the project expects, not have any LLM generated comments (which trend towards being useless anyways)

The wording of the post is very clever, if you're green you read it and you say "I'm banned because I use Claude". If you're experienced enough you can read the message between the lines: "if you use LLMs, it should still have gone through your brain before it reaches mine, so I'm not turned into a meat puppet by proxy"

[–] GuB-42 1 day ago

The fact we have different interpretations is the reason why it should be clarified.

Some people reject AI on a matter of principle. They may have a problem with the energy cost, the idea that AI companies are stealing the world knowledge for their own profit, that any use of LLMs will make you stupid, or that it does more harm than good overall. The code of conduct links to a short story by Isaac Asimov, you typically don't do that when your motives are purely technical.

[–] BoorishBears 1 day ago

They can't clarify it.

If you say the quiet part out loud, the lemmings will take that as approval.

"Only thoughtful application of AI allowed" -> "Of course my usage of AI was thoughtful! My custom pi harness with 1M context project brain is very thoughtful!"

Not sure why you think this wouldn't be something they did on principle either though. Most open source projects are built on someone's personal principles: that's what motivates them to keep working.

(hence the recommendation someone's free to fork it and do the work themselves)

[–] Gigachad 1 day ago

I've been battling this at work where management dumps a wall of claudeslop text in to a JIRA ticket highlighting some theoretical issue and it's incredibly difficult to understand what it's actually on about. And of course you can't ask the submitter because they don't know either.

[–] rootusrootus 17 hours ago

This is when being very senior can help, as long as nobody above you has been consumed by the psychosis.

> And of course you can't ask the submitter because they don't know either.

At that moment I abandon the ticket with a comment that they can reopen once they can explain what they found, why it is an issue and should be fixed, etc.

[–] Rapzid 1 day ago

> No LLMs for finding bugs.

That's what it says, literally. I'm sorry but for a programming language project.. That's the most human slop of a policy I have seen in a long time.

Low effort, human slop.

[–] iperov 1 day ago

If an LLM finds a bug, is that almost the same as an LLM writing code for you? You can write lazy code as a sketch, for example without thinking about nested loops in your head, and then just use an LLM that will fix this loop in the bug report? This can be viewed as a level of AI usage: 10% (for example), where vibe coding is 100%. I think any percentage of usage affects your personal mental sharpness. We know that if something gained somewhere, something lost somewhere. The only question is where the "loss" is, and how it will affect everything and everyone.

[–] handoflixue 1 day ago

> We know that if something gained somewhere, something lost somewhere.

Yeah, but that doesn't imply equivalent exchange.

If you care about being the best sprinter in the world, yeah, put in the time and effort! But if you just want to get to work, there's no way you can beat the speed of a car.

If you're one of the better bug-hunters in the world, you probably lose something by using an LLM. For the rest of us, it's increasingly like trying to race a car.

We definitely need both types - it's important having the humans who do have that top 1% understanding. But increasingly, for everyone else, LLMs outperform doing it by hand.

[–] zamadatix 1 day ago

I suppose that's the split between "writing code" the action and "writing code" the overall development process. I do agree there is some nuance in thoughts here often lost by the term getting overloaded.

> We know that if something gained somewhere, something lost somewhere.

I don't know that.

[–] Tiberium 1 day ago

Plus some of the issues are clearly written with LLMs, and apparently there's an active contributor who has been using LLMs to translate comments and help with code - https://github.com/ZigEmbeddedGroup/microzig/pull/1087:

> Yeah we know you're working with an LLM. That is fine (imo), as long as you're not doing drive-by slop PRs. These have been rather good quality, and you've been testing on hardware (which is important)

So it feels odd for them to suddenly change from this (clearly working) setup.

[–] paaloeye 1 day ago

Keep in mind the project HAS just adopted the policy

[–] Tiberium 1 day ago

My point is that it seems extremely strange to do that when they have developers who use LLMs already contribute to the project. I feel like they'll either alienate people who are interested, or just not care to enforce their no-LLM policy.

[–] paaloeye 1 day ago

Said developers will decide: either fork or retire their LLMs. As simple as that. Nothing strange at all.

[–] vkaku 1 day ago

The productive ones will just fork and accelerate faster. I understand the need for the mental muscle, this framing just sounds terrible at this point. Something something human scrutiny is not even an option the people want to consider.

[–] latexr 1 day ago

> The productive ones will just fork and accelerate faster.

Zig has had a similar policy for a while. Where are these myriad forks which accelerate faster, and the communities behind them?

[–] paaloeye 1 day ago

Yeah, where are they?

[–] paaloeye 1 day ago

Let's see when they deliver real features alone which are widely adopted and battle-tested in production.

[–] handoflixue 1 day ago

https://www.ign.com/articles/we-vibe-coded-a-new-epic-games-...

It's not hypothetical to say that an LLM can write widely adopted, battle-tested, production code.

Claude Code would be another example.

Linux is still mostly human, but even Linus Torvalds has warmed up to the idea that LLMs are tools a skilled user can use to contribute there.

There's a lot of quieter, private examples where some company rewrote their entire back-end or internal tooling with an LLM. You just tend not to hear about it because "we used AI" is currently negative marketing/PR, so why say it out loud?