[–] sweetdingoes 3 days ago
[–] bmitc 3 days ago
I use these latest available public models daily in max effort modes, and they barely seem to work and need extreme guidance and as much context and constraints as possible.
[–] robswc 3 days ago
I find that unless the outcome I'm looking for from the model is almost binary and easily verifiable, it is basically faster to do the work myself.
Assumptions made by the models multiply and by the non-deterministic nature (dbf, similar to work by humans) I'll have something end up light years away from where it should be. During a normal workflow, I have to contend with every line I write forcing a very tight feedback loop... I suppose I could also sit there and spend 9 hours writing out a "perfect spec" but we've already tried that with waterfall development!
So, not sure if its the models, rather the class of problems I'm applying them to... but it is a bit ironic they are allegedly able to solve these conjectures but not stop to ask about assumptions.
[–] bmitc 2 days ago
And a perfect (or near-perfect) spec is actual source code. And that's where I see myself with LLMs. They are great about generating source code. They are terrible at writing well-specified software, and so they need a lot of context: docs, plans, existing source code, existing templates or examples. And even then, they still need a lot of guidance.
[–] nonethewiser 3 days ago
2. We dont have the actual frontier models
[–] SpaceNoodled 3 days ago