Hacker News

points by schaefer 4 days ago | 0 comments

Pi.dev at home.

And I’m slowly writing my own. But it’s garbage at the moment.

I wrote one and kept iterating by using it to fix bugs in itself

I still also use other harnesses, but mine is already at the point where I prefer it more often than not

You should totally keep working on yours. It’s a cool learning experience. And you might stumble on some good ideas that nobody else has had

[–] ylrd91 4 days ago

Don’t write your own. Fork maki.sh and customize it like I did. It’s built in Rust and is absolutely amazing.

[–] schaefer 4 days ago

Thanks for the suggestion, but that’s not a fit for me.

I’m writing my own harness to learn. And I’m writing it in zig.

But, I’ll absolutely add maki.sh to my reading list. So, that’s a win.

[–] crossroadsguy 4 days ago

You should do that. I am also summoning the motivation to start on my own. I guess the worse will be that I fail. Might as well fork sth more basic, or build on top of sth SDK/library like.

[–] schaefer 3 days ago

I don't know if you'll be interested, but there is an excellent book on the topic: Build an AI Agent (From Scratch), by Jungjun Hur and Younghee Song (Manning Publication).

By chapter four you have a minimal working codebase in python. Later chapters add out features like tool calling, planning, and reflection.

[–] crossroadsguy 4 days ago

I am thinking about it but I think forking will miss the point. I'd rather do in a way when there is a prevalence of various types of agentic/harness SDKs for people to make their own apps (aka harnesses) from.

[–] ylrd91 3 days ago

That’s what pi.dev exist for but I’m not touching it because like all of the other harnesses it’s built in JS and they’re all terrible when it comes to cpu and memory use. Maki is the only one I found the be the most efficient harness. It uses 1% cpu idle vs 20-30%.

[–] leereeves 4 days ago

What kind of hardware do you need to run a useful LLM at home? To train one?

[–] schaefer 4 days ago

> "What kind of hardware... "

At home, I mostly use openrouter.

A few bucks will realistically last you forever. I also bought a DGX Spark at launch. I've learned a ton. but it's not usually my daily driver. I do run overnight tasks on it, like "read this book and write up notes on each chapter..."

In retrospect, given the dirt cheap prices of apis and or renting hardware, buying hardware for inference was likely a mistake (but not enough of a mistake to sell it now).

[–] margalabargala 4 days ago

There are a lot of tasks I have, e.g. integration with homeassistant, that I absolutely will not trust a cloud provider for. On prem or nothing.

I'm not willing to spend thousands for those features, so my house will remain dumber than it could be until prices come down in a year or two.

[–] CamperBob2 4 days ago

Take a look at the docs for https://github.com/niko1221/strata for some ideas, particularly https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#... .

[–] raxxorraxor 3 days ago

I think building a OpenAI compatible RAG-pipeline that also perhaps includes the anthropic sdk let's you have the best of both worlds. You can build it in a way that a local agent can escalate to a frontier model when needed and also substitute prompts data that you want to hide from being "public". For example. I think that is meant by harness here.

And with some smart feedback loops, you will see that for home assistant type tasks local AI can pretty quickly fulfil all requirements. For quick responses or voice analysis you might want to start with an external provider.

Otherwise, I think for complex home automation around 24-32gb vram are a realistic house number, but more experienced users will probably need less here.

People say that tools are called by the model. That isn't entirely correct. The caller calls the tools, the model just formulates the syntax for it. So the model part can be externalised without you needing to compromise you system fully (just to the degree the tools would allow anyway)

[–] mynegation 4 days ago

Schaefer says they write their own harness, not necessarily training or running a local model.

[–] schaefer 3 days ago

Correct. I am only working on building my own harnesses.