Survival LLM: AI assistant and emergency library that runs on your own computer

zabriel_goss

Nice idea! I wonder how the installed IBM Granite 4.2 8B LLM does on machines with "modest" RAM

rnd0

Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM? This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.

nico

Love this. And as a trend, I'd love to see more local/small task-specific models/agents. Experts at helping you with that one thing, leaving the everything-and-the-kitchen-sink to the frontier models/agents

majorchord

I like to use llamafile for cases like this: https://github.com/mozilla-ai/llamafile

anonbeat

it would be useful if it would shrink a little bit

good1andrew

pretty cool I like it. could you support more (small) models in the future, e.g. from the qwen or gemma family?

TheRealPomax

With all due respect: why does this need an LLM, and this much data? Surely a "classic" knowledge base with a more traditional search bar, running on top of the same data this LLM was trained on would be way smaller, and actually as trustworthy as the source material it surfaces for you?

johnnudd

On the setup process: "You only click for about 5 minutes; the rest is downloading." On the energy consumption: "A small generator or power station runs it easily" Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.

armcat

Back in 2018 I was working on a project for educating schools in certain parts of Africa and Middle East, on AI. The issue was it had to be air-gapped (no internet guarantees) and small/portable. I ended up downloading Wikipedia (not all of it), and then created a lightweight inverted index (for search) and TextRank (for summaries). The whole thing was on a Raspberry Pi, and installed in a small plastic enclosure. We had many of these and could easily ship them. There was an "AI Learning Module" as well we built, that started with Scratch, and then moved into Python. But if I was to do it all over today, I would probably add a tiny LLM and maybe a small ASR and a TTS model too, and you basically have a completely self-contained AI knowledge assistant in a cigarette box. The point being, you basically also have a "survival AI" you can take with you. Of course you still need electricity...

walrus01

If you're actually offline and need information in a 'survival' situation, I'm not so sure the hallucinatory nature of an LLM is the based way to distribute a repository of information. Additionally, there's no need to fit an LLM entirely in memory. In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.

saaaaaam

> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”. Honestly, just buy some books.

yoble

The required specs seemed a bit high to me, and I was just looking for alternatives. I found Project Nomad ( https://projectnomad.us ) which looks more polished. Any other suggestions?

TGower

It's disappointing that Gemma E4B seems to perform poorly on this (42% accuracy according to the GitHub). It would be nice if a $25 phone could be loaded up with this and stuffed in a EMP proof metal box with a small solar charger. Maybe a finetune or Gemma 5 will get us there.

x3haloed

I made it a while ago now, but similar: https://github.com/x3haloed/eowkit Helps you download a local LLM and offline Wikipedia. Probably easier to just ask your favorite agent to glue something together yourself these days.

abujazar

No thanks, I'll use a generic model along with trusted sources. Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.

doublerabbit

How on earth are you going to operate this if you have no electricity?

mentalgear

This could be made a lot more practical (and safer) if made searchable via a simple vector search / embedding that can run on any ordinary computer. The source (data collection) seems the most important feature here worth reusing.

x3haloed

This topic is bringing out the hallucination-panic crowd. It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice. Use the LLM as search technology: string match -> semantic search -> search intelligence. This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation. Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first. Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.

tobadzistsini

No GNU/Linux? Meh.

jonathaneunice

Alternate idea: Download Unsloth Studio https://unsloth.ai/docs/new/studio#quickstart From the command line: unsloth studio Browse to the URL it gives you (eg http://127.0.0.1:8888/ ) Download a good fast local model e.g. Gemma-4-12B (small machine) or Qwen3.6-35B-A3B (larger memory). Or a Nemotron model, if you prefer that. Then you'll have both your backup/emergency assistant that runs on your own computer and be gaining skills in a common, frequently-updated, well-supported program.