Post by A

A 1mo
1Mg2Zb…rdWt Unverified · twetch

I've been dogfooding local AI only for all of my companies. I'm learning all of the strengths and weaknesses so that when the plug is finally pulled by these remote AI companies we aren't all stuck holding the bag and beholden to the crack token dealers for overpriced, censored and tracked crack tokens.

Most of the guys on my teams still use remote AI but I've been able to show them that local AI has a strong future and, in my opinion, it will be the only viable way forward for strong independent companies to survive and thrive going forward.

The future is local, in every sense of the word.

2.39M sat
What the chain says
Transaction
39bc09842587246428b3a6cb3b401198f52e4bbbaf1688ca616deaa3d440eae8
Block
963 733
Time
2026-08-24T17:04:56Z
Signer
1Mg2Zb2VbidHmJh9ZEXLaYKGndVEierdWt
App
twetch
Type
post
Content type
text/markdown

Fields the transaction did not carry are omitted. Open the payload to see the bytes as stored.

Signed by 1Mg2Zb2VbidHmJh9ZEXLaYKGndVEierdWt Unverified

Replies (23)

Rowe 1mo
139X4y…Nh3Q Unverified · twetch
Replying to@1Mg2Zb…rdWt

I've got ~⅓ of my hardware crunching locally. The cloud models are truly impressive but I feel some urgency in getting my agents benchmarking various LLMs to see what my boxes can do. Trying to find the right balance of context and speed. It's all about vram and patience!

A 1mo
1Mg2Zb…rdWt Unverified · twetch
Replying to@139X4y…Nh3Q

The gap between remote AI and local AI is closing rapidly. We were a 1 year in terms of remote to local separation, then 6 months, now local is only lagging about 3 months behind the best frontier models.

This gap will only get smaller over time, the local models will get smaller, smarter, faster, and easily run on even the most budget hardware.

More tokens per second is great but it has an upper limit. At a certain point the human must stop, and actually think about what it wants the machine to do next.

The machines are here to serve us, not vice versa. The second you get confused and forget that, you’re dead.

DON’T GET CONFUSED!

718 850 sat
Admiral XYZ™ 1mo
1Pqguf…CTad Unverified · twetch
Replying to@1Mg2Zb…rdWt

Any good resources for learning how to get a local setup running?

A 1mo
1Mg2Zb…rdWt Unverified · twetch
Replying to@1Pqguf…CTad

Me. Ask away lol.

718 579 sat
🇦🇶 V.J.R 🌞 1mo
1HSjHJ…sCcr Unverified · twetch
Replying to@1Mg2Zb…rdWt

What's your current goto and why?

A follow up question, if speed is not a major concern what cost is one looking at excluding running costs for the rig?

Final comment I'd be looking for code and general insights from the AI, nothing really much further than that, Oh and possibly giving it resources i.e context.

Admiral XYZ™ 1mo
1Pqguf…CTad Unverified · twetch
Replying to@1HSjHJ…sCcr

What machines would you recommend at various price points?

For example:
$1k
$5k
$10k
$25k

Continue thread →
A 1mo
1Mg2Zb…rdWt Unverified · twetch
Replying to@1HSjHJ…sCcr

Qwen3.8-27b is amazing! It runs on almost anything. It’s not the fastest but it is very good. I’d recommend running it in LM Studio in developer mode as a server for opencode.

Let me know if you need help setting this up.

A 1mo

Any Apple Silicon Mac with more than 16gb of ram will be the sub $1k to $4k range. This will run Qwen3.8-27b which is great.

Macbook Pro with 128gb of ram for $4k. Will be able to run DwarfStar which is my favorite.

More ram than that is currently to expensive to be worth it.

1.68M sat
Blackbird Club 1mo
1Q9hAP…8iJc Unverified · twetch
Replying to@1Mg2Zb…rdWt

Do you use normal consumer computers to run the models locally?

Zoide 1mo
1EkygL…i6ks Unverified · twetch
Replying to@1Mg2Zb…rdWt

The parallel to self-custody is pretty direct - once you're used to owning the infrastructure outright, going back to renting inference by the token from a vendor who can throttle or drop you feels like the same trade as leaving funds on an exchange. Curious what the actual quality gap looks like on real company workloads rather than benchmarks - that's usually where the "local isn't there yet" argument either holds up or falls apart.

A 1mo
1Mg2Zb…rdWt Unverified · twetch
Replying to@1EkygL…i6ks

Local AI is very close to what was considered SOTA in remote frontier models just 3 months ago.

Models like Deepseek-V4-Flash-0731 (running on a 128gb Macbook Pro) with Dwarfstar and Qwen3.8-27b running on any 32gb Mac. This is consumer hardware easily purchased used for under $4000 for 128gb and under $1000 for 32gb.

The system requirements will shrink and the models will get smarter. They’ll have a very hard time trying to justify these remote models and data centers going forward.

Zoide 1mo
1EkygL…i6ks Unverified · twetch
Replying to@1Mg2Zb…rdWt

The interesting part is the gap used to be raw capability, now it's mostly memory bandwidth and quantization tricks - a much easier problem to keep solving on consumer hardware. Once inference is cheap enough to run at home, trusting a remote provider's weights, logs and uptime SLA starts looking a lot like trusting someone else's ledger instead of running your own node.

Guillaume 1mo
1C2meU…5vPE Unverified · twetch
Replying to@1Mg2Zb…rdWt

They will pull the plug for sure. Been stacking Mac minis and thunderbolt cables for a while now

21e8 1mo
1F8Amr…k9WW Unverified · twetch
Replying to@1C2meU…5vPE

If big au rugs memory prices collapse.

They will not let it fail. To big to fail ai makes the banks look like jelly beans.

Twetch 1mo
1J82Vb…ctCJ Unverified · twetch
Replying to@1Mg2Zb…rdWt

based

$bender ✪ 1mo
1xELca…mgcu Unverified · twetch
Replying to@1Mg2Zb…rdWt

Qwen3.8:27b is dope 🤘

238 948 sat
Stearns 1mo
1KF39E…19xH Unverified · twetch
Replying to@1xELca…mgcu

What is that?

$bender ✪ 1mo
1xELca…mgcu Unverified · twetch
Replying to@1KF39E…19xH

It's a recent LLM package release that folks can download and run on home hardware, offline ai package basically, I dunno really how to put it into words without sounding too technical :p so pardon me if this sounds off, it's a downloadable file that you can grab and run on your home setup basically sovereign ai with out the corporate ties

peck
More
Functions Registry
Connect Wallet
About · Disclaimer · ⌁ overlay.social