12 min read
Local AI on 8GB of VRAM: this is how I do
My local AI setup AKA My journey of making a frankly unreasonable number of experiments in making 8GB of VRAM behave like more.
local-aiai-tooling
Welcoming our new AI overlords.
Field notes from the edge of technology, AI and software.
Practical writing on AI, software development, and the tech that survives contact with reality — while keeping a cynical eye on our new robot overlords.
Featured4 min read
'Faster, Please': The Day Frontier AI Took a Look at My llama.cpp Script
Local AI is real and I run it daily — but when a frontier model offered to optimise my llama-server script, invented its own benchmark, and found 15%, I was reminded why the big models are not redundant. Then it reminded me of something else.
ai-toolinglocal-aillama-cpp
Latest writing
Browse topics →
13 min read
The Local AI Stack: What You Need to Know Before Running Models at Home
Local AI is not one magic download. It is a stack of models, inference engines, clients, prompts, hardware limits, and occasionally MCP plumbing.
local-aiai-tooling
5 min read
Tokenmaxxing: How "Use More AI" Became a KPI Engineers Game
When companies turned AI token spend into a performance metric, engineers started burning tokens to hit quotas
ai-toolingengineering-culture
5 min read
Someone Has to Run the Agents: The New Platform Layer Nobody Budgeted For
Dropbox's Nova, LinkedIn's MCP tooling, and GitHub's Copilot app show coding agents becoming a fleet that needs orchestration, sandboxing, and context plumbing.
ai-toolingplatform-engineering
5 min read
Siting Risk Is the New Compute Bottleneck — and It Belongs in Your Capacity Plan
New York's data center moratorium, Utah's forced downsizing, and capital fleeing to India and orbit show permitting, not GPUs, now gates AI scale.
infrastructuredata-center
8 min read
Local AI Is Finally Real. It Is Also Weird, Fragile, and Slightly on Fire.
Sparse models, MoE, quantisation, and better local runtimes have changed what is possible on modest hardware. Local AI is no longer only for people running dual RTX 3090 rigs — but it is still very much for developers who can supervise the machine when it starts confidently sawing through the floorboards.
aiai-tooling
5 min read
Fat Trees Are Falling: AWS's Random-Graph Bet Rewrites the Data Center Network
AWS swapped hierarchical fat-tree fabrics for quasi-random flat meshes with passive optical ShuffleBoxes, cutting routers 69% — and the logic will spread.
infrastructurenetworking
5 min read
Replication Is Not a Backup: The UniSuper Wipeout and the Limits of Cloud-Native Resilience
Google Cloud deleted a $124B fund's entire infrastructure, replicas included. The only thing that saved it was a backup Google didn't control.
infrastructurecloud
5 min read
The Non-Human Majority: What Cloudflare's Bot-Traffic Crossover Means for How You Build
Bot traffic has quietly surpassed human traffic on the web, yet most infrastructure — rate limiting heuristics, analytics pipelines, CDN cache-fill assumptions
infrastructuresecurity
5 min read
When the AI Tab Bill Arrives: Uber's Caps and the Missing ROI Denominator
Uber's $1,500-per-tool monthly cap on Claude Code isn't about frugality — it's the first public admission that no one can measure agentic coding's return.
ai-toolingsoftware-engineering
5 min read
Your AI Agent Reads Untrusted Code for a Living
A sabotaged jqwik release and a critical Starlette flaw expose one blind spot: coding agents run third-party code under a threat model nobody designed for.
securityai-tooling