Hacker News

points by rnd0 2 days ago | 0 comments

Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM?

This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.

[–] nico 2 days ago

Not sure about how the code in this project works. But the way I see it, an LLM+chat is a great interface for information. If you then connect it to a db with the knowledge/articles you need, then you get a great way to navigate it, without needing the model to memorize all the info

[–] mapontosevenths 2 days ago

I think that if I were to build this I might skip the LLM and just setup a nice interface to the source files using qmd style search. [0]

That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.

I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.

[0] https://github.com/tobi/qmd

[–] nico 2 days ago

Awesome, I didn't know qmd style search, thank you!

I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)

0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...

[–] rafterydj 2 days ago

I agree with the OP; this technology does not seem an appropriate fit for survival equipment. By definition survival scenarios can be life and death. Even one "You're absolutely right!" level mistake from an LLM could give a user bad information, leading to John Doe eating a mushroom he really should not have eaten.

[–] tourmalinetaco 2 days ago

I agree with your point, but if John Doe is foraging mushrooms without experience of the local flora (which I doubt this archive contains), then John Doe is kinda asking for it.

The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.

It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.

[–] lukan 2 days ago

Slight OT Survival tip, if you are starving don't go for mushrooms in the first place (unless you really, really know) - even if you get the right one, they don't contain much energy for you to burn anyway. The larves you often found in them, probably contain more. So mushrooms are a nice addition (and in rare cases good medicine), but for survival you should focus on something else.

[–] ASalazarMX 2 days ago

Why not both? After the collapse of civilization, as long as you can maintain the computer running, you could educate the next generation as soon as they could write. Maybe they will bypass the local agent as they learn their way into the library, to save energy.

Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.

[–] Sky_Knight 2 days ago

Survival implies that energy would definitely be a very limited resource. Using it to replace knowledge is just... I can't describe it even. Why not just learn it on our own and keep a small guidebook in your backpack. Our overrelience on computers and now LLMs, even for basic life-saving knowledge is simply our way of going the way of the dodo... I use frontier models at work, I definitely get more done. However I will never use it as a replacement of my own knowledge.

[–] drcode 1 day ago

lots of us have solar cells these days, so energy isn't going to be such a limiting factor

[–] munk-a 2 days ago

Nvidia GPUs in the post-apocalyptic radioactive barrens are going to be stupid pricey.

[–] AdieuToLogic 2 days ago

> After the collapse of civilization, as long as you can maintain the computer running ...

One of these situations prohibits the other.

[–] asdff 1 day ago

A book is a lot more of a sure bet than you maintaining a computer. We have ancient books today that have survived several civilization collapses.

[–] datsci_est_2015 2 days ago

Both. Both is good.

A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.

But “how do I effectively forage for edible mushrooms” is a great stepping stone.

[–] rebolek 2 days ago

I think that for situations like this we should invent a device that can hold information for an arbitrary amount of time even without electricity. Maybe we could call it "BOOK" or something like that.

[–] anigbrowl 2 days ago

This isn't clever or insightful. You can of course condense down critical information into a survival handbook, but in existential crises you don't necessarily know in advance what you'll need to know. Wikipedia (as a benchmark repository of knowledge) is ~5 million pages and would require a decent-size room to store if printed out. By contrast, a laptop + solar panel can fit in a backpack. SD cards/USB sticks with the capacity to store all of Wikipedia are a few $ each.

If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?

[–] dns_snek 1 day ago

I don't think Wikipedia is a good benchmark here. Try 30 random articles and see how much of that information you would want to have with you in an emergency survival scenario.

Either way the comparisons here are unfair. An LLM that runs off a backpack-sized solar panel and a laptop might be a substitute for a broad, low quality, surface-level encyclopedia but it's many orders of magnitude worse than a library of books.

I would argue that in a survival situation the accuracy of information becomes even more important. If an LLM to misidentifies a plant (or a medical condition, its treatment, etc.) it could put you in an ER, but in a survival situation that could kill you.

[–] anigbrowl 11 hours ago

It's not important that it's Wikipedia specifically; I just picked that as a benchmark because it's large and comprehensive. The point is that you can't carry a library of books around easily.

You're arguing from two opposite positions. On the one hand you want information that's relevant to an emergency survival situation, presumably specific instructions on on emergency medical procedures or protection against hazards.

On the other hand you are also arguing for a physical library of books, which requires a building to house safely (and a structurally sound building, not on where the roof can fall in and the books are quickly lost to damp/fire/wildlife) and a full size cargo container if you want to move it around.

If you can maintain a physical library safely enough to hedge against the risks of it being your only library for the foreseeable future, you can probably also maintain a hospital and essential infrastructure of civilization. You're operating something equivalent to a small university campus with a high degree of physical security even if you've had to fall back on 19th century technology because power and water infrastructure have collapsed. Perhaps people are investing in that sort of thing as a hedge against a Carrington event or other abrupt technological collapse, but I'm not aware of any organized effort to build a network of knowledge-preserving monasteries at fairly significant expense for each one.

A digital encyclopaedia/library isn't ideal by any means, but it's portable and maintainable at low cost, for individuals or groups. In the space/weight that a single large book takes up, you can fit abundant storage, several e-ink readers (which will probably last very well given their low power consumption), a battery, a large solar collection sheet, and and a faraday bag to store it all in, for under $1000.

[–] _superposition_ 2 days ago

The pedanticty (or is it pedanticness???) on hn is astonishing sometimes.

If there's any place to get an answer it's here.

Bravo my friend. And I mean that in a good way. Love me some diversity.

[–] dorianmariecom 2 days ago

and then we have a lot of “books” so we collect them in “shelves” then in “libraries” then we want to search them etc etc

[–] mtw 1 day ago

For one-off questions, a local archive makes sense.

This is more for elaborate projects where you don't know precisely what you are looking for. Questions I can think of in a survival scenario:

- my solar panel connector is broken. We only have 8 hr battery left. Can you tell me how to DIY repair it with the following parts

- we want to receive emergency FM broadcasts from the state. Give me steps on how to receive it on Flipper Zero with this breakout board I found in my spare parts

- with the dewalt drill and these parts, can you tell me how to build an outboard motor for the canoe. We need to get outta here before the flood goes up further

[–] IshKebab 2 days ago

> the tendency even now of LLMs to hallucinate

That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.

Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.

In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.

Obviously this local LLM is going to hallucinate all over you though.

[–] x3haloed 2 days ago

Decent LLMs can run on anything now: https://github.com/he-be/turbo-fieldfare

Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.

Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.

[–] iAMkenough 2 days ago

I like this "both / and" approach. Restructure this project to have the LLM source from local Kiwix data you've selected to make available offline, rather than a source of data you're unfamiliar with. References point directly to your Kiwix library.

[–] xamuel 2 days ago

>can run on anything now: (link to turbo fieldfare)

Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"

[–] lrvick 2 days ago

> and the hardware requirements for any kind of LLM

You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.

Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.

I work on compilers, kernels, and operating systems using exclusively consumer GPUs.

[–] tourmalinetaco 2 days ago

It already is a local archive that can be searched independently, but the LLM does RAG and is, according to them, 95% accurate. It can be useful if you don’t remember where to find a source/what words to search for, but even a durable device with Kiwix would be useless in a a lot of survival situations people imagine.

Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.

[–] bravoetch 1 day ago

I might not know the trouble-shooting steps for a problem I'm having. LLMs may give poor answers, but they can help me find documents with good answers. That would be enough for most people.

[–] h05sz487b 1 day ago

It does, you can ripgrep all through that archive.

[–] classified 1 day ago

But this one generates more clicks because "AI" is hot right now and has religious following.