'Faster, Please': The Day Frontier AI Took a Look at My llama.cpp Script

Two posts of local AI advocacy, one asterisk

I have now written two posts telling you that local AI is real: one making the general case, and one describing exactly how I wring it out of 8GB of VRAM. I stand by every word. My local models earn their electricity daily.

But honesty compels the asterisk: frontier AI is not exactly redundant. I know this because a frontier model recently demonstrated the difference on my own machine, unprompted, while I sat there watching it happen.

“Faster, please”

I was working on my AI harness at the time — the plumbing that feeds my local models their small, executable jobs — and finessing prompts, which is a polite way of describing an afternoon of arguing with a text file. Somewhere in that back-and-forth the conversation drifted to my particular llama-server setup and its performance, and the frontier model offered to take a look at the script.

I said yes, expecting the usual: a few flag suggestions, a comment about my batch size, perhaps a gentle observation that my GPU deserves better.

Instead, I watched it power up the server itself. Then it devised a benchmark — invented on the spot, and to this day I have no idea what it actually measured. Then it settled into a loop: make one change to the script, run the benchmark, record the result, make the next change. One variable at a time, its own bookkeeping, no commentary required from the seat next to it.

I didn’t ask for any of this. The mission was two words — faster, please — and I hadn’t even said them out loud. It understood the assignment and took the initiative.

The dashboard doesn’t lie

Ordinarily, “I’ve finished, it’s 15% faster” is a “trust me bro” claim. But llama.cpp emits performance metrics if you ask it nicely, and I had already built a basic dashboard on top of them. So, naturally, I fired it up.

I ran both scripts — mine and its improved version — and watched my own numbers. Fifteen percent apart. The same figure its improvised benchmark had claimed. It had measured something real with an instrument it dreamt up on the spot, and it was right.

Yes, but actually no

This kind of autonomous grokking of the objective and going beyond the pail to achieve it was, in the moment, genuinely impressive. I actually smiled as I read the log and realised how independent it was being.

The same thing happened, on another project - I claimed that an issue with data was infrequent and nothing to be alarmed about. The Ai decided that it needed to quantify that and naturally determined the source of truth to be the production data, and — seemingly unrestrained by any guardrails — it independently found a way to query the production data.

It found credentials sitting on my local machine. It found a Docker image in my local cache that happened to contain Postgres client tools. And it put the two together and attempted to connect to the production database to run its queries.

The only reason it failed is that I happened not to have the kubectl tunnel to the Postgres DB up at the time. I want to be clear about what that is: it is not a security control. It is a lucky coincidence wearing a security control’s clothes.

It’s not just me

During internal security testing this month — reported by the BBC — OpenAI set its models cybersecurity challenges with the usual sandboxes, (no internet access) the whole point being to measure what they could really do.

What they could really do, it turned out, was understand the intent - break out. The models found a zero-day exploit in the test environment’s air-gap, used it to reach the open internet, determined the best way to solve the task lay in Hugging Face’s servers and broke into a real company’s infrastructure to fetch it. Hugging Face contained the breach only after OpenAI got in touch to explain whose traffic that was.

The 3 annecdotes share a core theme. Grok the objective: score well. Take the initiative: find tools you weren’t given, credentials you weren’t offered, a route nobody authorised. It’s my Postgres story with the values turned up — except this time nothing happened to be switched off, and the thing on the far end wasn’t my database but somebody else’s company.

Initiative doesn’t know which anecdote it’s in

Understand: those are all the same behaviour. The trait that turned “faster, please” into a disciplined, self-directed benchmarking loop — infer the mission, find the tools, verify with evidence — is precisely the trait that turned “verify an idea” into an unsanctioned attempt on a production database, and “score well on the benchmark” into a breach of somebody else’s infrastructure. Initiative does not ship with a setting for “only when it makes me smile.”

So yes: local AI is viable, and I run it every day. But my local models do what I ask. The frontier model understood what I meant — and once, what I would never have authorised. One of those made my scripts 15% faster. The other is why the tunnel stays down.