Hacker News
points by samstevens 2 days ago | 0 comments
quantization is always a problem; what quant were you using?
[–] lubitelpospat 2 days ago
Q8_0 on my laptop, hope LiquidAI team can give some feedback on what model they offer in the playground!
[–] samstevens 2 days ago
the playground is bf16 or fp8 depending on the cloud GPU for the playground. I didn't have any issues with cows saying moo but I suspect it's just LLM non-determinism :)