points by arjie 14 hours ago | 0 comments
Gemini team has a long history of releasing great benchmark charts and disappearing into the ether. There has never been a moment where I have preferred them to Claude/Codex. When I get access I eval them on my home assistant set, a small coder benchmark, etc. and they are too expensive, and too low quality.
The benefit for Google is their chat interface has privileged access to their data (maps, etc.) but their model itself is low quality and, if I'm being honest, has the promotion-oriented measurable-output bias that FAANGs tend to have. The later Llamas had the same problem.
[–] SadWebDeveloper 14 hours ago
[–] conception 8 hours ago
I love this never-ending people having vastly different experiences situation with LLMs. In my experience, Gemini flash 3.8 is an exceptional model and is the bulk of my agentic use as someone who bounces around models as soon as they come out. I would say it’s the best flash model until 1017 came out. It might still be better than 1017.
It is unfortunate that having such an amazing engineers that are capable to solve any leetcode exercise in 5 minutes on the spot with a white board, they aren't able to even deliver something useful on the AI space.