Hacker News
points by aleph_minus_one 4 days ago | 0 comments
> It's going to boil down to cost of the hardware vs. pay per use cloud models. I'm on the 100$/month chatgpt plan (~88euro). With all the resets, credits, and all the rest I typically don't run out of tokens. This is fine for me. I might go to the 200$/month plan at some point.

It's also about "insurance protection" against the AI company raising the prices or changing their terms (e.g. becoming more frugal with respect to token drops).

[–] kevmo314 4 days ago
If they raise their prices you can switch pretty quickly, insurance isn’t worth much.
[–] jillesvangurp 3 days ago
The speed at which you can switch is probably proportional to your dependence. But the thing with insuring against risk is that you need to cover all of them and not just cherry pick them.

If you are running your own models on your own hardware you take a bunch of risk as well. Is your hardware capable enough to run your models? Do you have enough of it and access to more when you need to? And is the stuff that you picked still going to be good enough in six months? And can you trust those model providers that you picked to keep up with the rest of the industry or are they going to fall behind and leave you stuck with obsolete but still very expensive technology.

Of course, this is an article about IBM who are in the business of selling consulting and services at a premium. They would probably love companies to self host their models on infrastructure that IBM supplies and consults on (for obvious reasons). The question is, should you buy what they are selling or can you do better? My simple observation here is that if companies like IBM are getting excited it's probably because they are looking to separate a lot of companies from some pretty significant cash. So, I'd be wary to buy what they are selling. But that's just me.

In a rapidly evolving market with extreme levels of uncertainty around tech and what will happen next, being reluctant to get pinned down on the wrong solutions is probably a good instinct. The biggest risk here is making the wrong bets. And you need to look at what it's going to cost you if you get it wrong and what it's going to cost you to fix it.

[–] aleph_minus_one 4 days ago
This depends a lot on how deeply integrated you and your workflows are into the idiosyncrasies of some specific LLM or class of LLMs.