points by prophesi 11 hours ago | 0 comments
They were given day zero access to release their quantizations[0]. The lowest quant will still require 256GB VRAM to run inferences, but as always I'm amazed by the Unsloth team by how much they can reduce the requirements for these powerful LLM's, and the easy-to-follow guides to get started with using them.
Thanks for the link, appreciated!