Hacker News
points by prophesi 11 hours ago | 0 comments
They were given day zero access to release their quantizations[0]. The lowest quant will still require 256GB VRAM to run inferences, but as always I'm amazed by the Unsloth team by how much they can reduce the requirements for these powerful LLM's, and the easy-to-follow guides to get started with using them.

[0] https://unsloth.ai/docs/models/mistral-large-4

[–] jacquesm 11 hours ago
Ugh, ok that's out of my league :) Almost though, another 32GB and it would work.

Thanks for the link, appreciated!

Hacker News