Hacker News

points by saaaaaam 2 days ago | 0 comments

> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB

So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.

Honestly, just buy some books.

[–] dvt 2 days ago

Or just.. download the books, and slap a search engine on top of them. It'll be like 10x smaller than the model. "Survival LLM" is absolutely a non-relevant use case.

[–] saaaaaam 2 days ago

When you’re living in an igloo trying to stay warm in the hollowed out belly of a cow having a few books around the place is probably more useful than some files on a laptop that ran out of battery just before it got morbidly damp. Plus you can burn them if you really need to.

[–] drusepth 2 days ago

On the other hand, the laptop struggling to power a good-enough LLM will probably be a great heat source for you in that igloo... for about 45 minutes.

[–] saaaaaam 2 days ago

As I said elsewhere, you can always burn the books! And eating a laptop is considerably crunchier and more fraught than chewing through some good old fashioned leather facings, horse glue and nice heavy 120gsm wove.

[–] recursive 2 days ago

On the other hand, if you know enough to build an igloo, you're probably going to be ok regardless.

[–] octoberfranklin 2 days ago

"cow in an igloo" sounds like something an LLM would hallucinate.

[–] saaaaaam 2 days ago

Yes I slightly mangled that one. But I’m sure there is a survival technique that involves crawling into the body cavity of a hollowed out cow. You probably wouldn’t want to do that in an igloo though. I’m not even sure you could get a cow into an igloo. If standard issue humans are this flawed it’s hardly surprising that top flight LLMs get things so wrong sometimes.

[–] natebc 1 day ago

I'd have gone with Bantha in an ice cave myself.

[–] taneq 2 days ago

And/or install Kiwix and grab an offline copy of Wikipedia.

[–] lossyalgo 1 day ago

Wikipedia rarely (never?) gives enough information to actually build or accomplish anything - it just has a nice clean summary of lots of things.

[–] scotty79 2 days ago

If you have AI model there it can make any search engine for you as per your needs.

[–] anthk 1 day ago

Recoll it's far smaller and it runs on a potato. It's a desktop search engine which can parse tons of files. Under GNU/Linux just install all the recommended dependencies for APT/RPM. Or just use Flatpak and to hell with transitive dependencies.

[–] arjie 2 days ago

The advantage of a turn-by-turn flow (honestly the chat interface was genius) is that with a little human intervention you can get much better results out of even silly models. The model of choice is strange, though. Not a particularly good one.

[–] taneq 2 days ago

Either it depends a lot on the model, or I’m bad at chat, or both, because in my experience once a chat starts going off the rails it’s pretty hard to correct the initial misconception. I find it works better to backtrack to just before the misconception and edit my previous response to head it off at the pass.

[–] arjie 2 days ago

Definitely agreed that it's better to not have some text in the context than to have some that needs to be opposed. I suppose I meant steering prompts are a non-obvious innovation to LLM interaction.

[–] Alpha3031 2 days ago

People apparently like Granite for RAG specifically. I'm not aware of any evaluations with numbers though, so I'm not sure if it's a real difference or just vibes.

[–] bityard 2 days ago

What is your favorite book on emergency appendectomies?

[–] saaaaaam 2 days ago

We had a set called something like “How to do anything in an emergency” which had instructions for this and more. Alongside lots of creative uses of potassium permanganate.

I’m pretty sure it also told you how to butcher a sheep.

I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.

[–] anigbrowl 2 days ago

I have a bunch of military field manuals and survival guides, but they all implicitly presume the existence and contactability of civilization sooner or later. Realistically, some pocket computer(s) and solar charger(s) will store far more information, be far more portable, and give you more carrying capacity for things like medical supplies, rope, tools, and other hard-to-substitute items.

[–] taneq 2 days ago

Have you re-read it recently? Some of those books were fun to read but not really something to use as a field manual. Then again some of them were great! I had a book that started with explaining how a battery and a light globe worked, went through how to make a DC motor out of copper wire and a cork, and ended with a crystal radio and I think a radio homing beacon (the latter might have been in another book.)

[–] saaaaaam 2 days ago

I have not, sadly. But I’m pretty sure I also had the book you mention. My brother and I made our own motor and many other things that would now be considered entirely unsuitable for 10 year olds to be doing. Copper sulfate crystals seem like a great idea until you have to explain why the playroom carpet has melted.

[–] moezd 16 hours ago

Exactly. Instead of getting a confident answer that I can store insulin at 95 F, I would like to know that it goes bad at 85. That 10 degree difference determines whether you store your medicine properly or not.

[–] dofm 2 days ago

Granite is not great at any size. Wonder why they picked that.

[–] saaaaaam 2 days ago

Given the whole thing reeks of vibe I don’t think anyone picked anything.