Darkbloom – Private inference on idle Macs

DeathArrow

Why only Macs? If we think of all PCs and mobile phones running idle, the potential is much larger.

rvz

Should have called it “Inferanet” with this idea. Away this looks like a great idea and might have a chance at solving the economic issue with running nodes for cheap inference and getting paid for it.

nl

They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE) The key question here is how they avoid the outside computer being able to view the memory of the internal process: > An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communication channels that could be observed, with optional hypervisor mem- ory isolation that extends protection from software- enforced to hardware-enforced via ARM Stage 2 page tables at zero performance cost.[1] I was under the impression this wasn't possible if you are using the GPU. I could be misled on this though. [1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...

kennywinker

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

chaoz_

That solution actually makes great sense. So Apple won in some strange way again? Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. I think batch-evals for non-sensitive data has great PMF here.

bentt

I thought this was Apple’s plan all along. How is this not already their thing?

TuringNYC

I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute. Any suggestions from anyone? We currently run ollama but manually manage the pools

pants2

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around $700/yr.

BingBingBap

Generate images requested by randoms on the internet on your hardware. What could possibly go wrong?

pants2

You might not even know it as a user but the payment/distribution here is all built on crypto+stablecoins. This is a great use case for it.

ramoz

Unfortunately, verifiable privacy is not physically possible on MacBooks of today. Don't let a nice presentation fool you. Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution. It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.

dr_kiszonka

"These are estimates only. We do not guarantee any specific utilization or earnings. Actual earnings depend on network demand, model popularity, your provider reputation score, and how many other providers are serving the same model. When your Mac is idle (no inference requests), it consumes minimal power — you don't lose significant money waiting for requests. The electricity costs shown only apply during active inference. Text models typically see the highest and most consistent demand. Image generation and transcription requests are bursty — high volume during peaks, quiet otherwise."

dcreater

I cant buy credits - says page could not load

stuxnet79

So basically ... Pied Piper.

tgma

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.