points by monegator 3 days ago | 0 comments
one of the points is that LLMs are NOT deterministic so you can't run them yourself a thousand times and expect the same output.
[–] bayindirh 3 days ago
[–] khaelenmore 2 days ago
And LLM "summarizes" the tool output as a completely unrelated and at times self-contradicting text.
If there are deterministic tools involved - just return their output as is. It's usually quite readable if needs a bit of adjusting
[–] pantalaimon 2 days ago
When was the last time you used a LLM?
This is not an issue with current models.
Debian has an extensive toolchain to check things. Having a set of deterministic tools which feed LLM with data can be beneficial, though.
Again, I'm not defending the LLM here, just want to show how hard the problem is to begin with.