ScrappyLabs · The Log · Field Notes

Two years to the taste of the stew.

A build-in-the-open account of how one workshop got from fighting its own operating system to running a fleet of local models that finally feel real — with a memory layer a frontier model just graded A−, and an opinion about the buggy-whip crowd.

Nobody hands you the shape of a thing. You have to build the wrong version a few times until your hands know it. Two years ago I was gluing AI into my work by copy-pasting out of a browser tab. Today a rack of GPUs in a spare room answers a business phone line, edits video, and rewrites its own memory from academic papers the week they drop. Here's the honest path between those two points — the friction, the turns, and the moment the whole game changed.

Era One
The Windows tax

I started where most people start: on Windows, trying to force a developer's workflow onto an operating system that fights you the whole way. WSL helped — until it didn't. WSL isn't Linux. It's Linux wearing a visitor's badge, and every real project eventually hit the seam: a filesystem boundary, a networking quirk, a permissions wall, a "works in the terminal, breaks in the tool" afternoon you never get back.

For a long time I treated that friction as the cost of doing business. It wasn't. It was a decision I hadn't made yet.

Era Two
Coming home to Linux

So we made it. Screw the compromise — we re-embraced the thing I'd loved in the first place and went Linux, all the way down. The workflow problem didn't get better; it disappeared. Overnight the machine stopped being an obstacle and started being a platform.

From there it compounded. Docker, then Kubernetes, then K3s for the lean edges — the boring, load-bearing plumbing that turns a pile of computers into one system you can actually deploy onto. That's the spine the rest of this is built on. Unglamorous, and worth every hour.

The lesson wasn't "Linux good." It was: stop paying a tax you can walk away from.

Era Three
Copy-paste, then the MCP moment

Early AI work was a clipboard. Ask the chatbot, copy the answer, paste it where it needed to go, do the work yourself. Useful, but it was a smart person trapped behind glass — all advice, no hands.

Then MCP showed up, and the temperature of the whole room changed. Suddenly it was all about tool use. The model could read the file, run the command, hit the API, check the result, and try again — not describe the work, do the work. That's the line between a chatbot and a coworker, and once you've crossed it you can't go back to the clipboard.

Era Four
The memory-layer race — and running our own

Once the model had hands, the obvious next wall was memory. An agent with no memory is a brilliant amnesiac: it solves the same problem beautifully, every single day, having learned nothing. So everyone started building memory layers, us included — Obsidian, markdown files, a place for the system to write down what it learned and read it back tomorrow.

Most people stopped there. We didn't, for one reason: if you read the papers as they drop, you can rebuild your own memory layer to match — or beat — what the labs just published. When AutoMem and AgenticSTS hit arXiv, we had their ideas running on our fleet inside a day: memory management treated as a measurable, improvable skill, with per-layer ablation and a frozen, graded eval set.

Which means we don't grade our own homework. We handed the whole layer to a frontier model — Claude's Fable 5 — and told it to assess us cold, with evidence: live index stats, the health scores, the ops timers, real retrieval probes. Here's the card it gave back.

Fleet Memory + RAG — Report Card · graded live by Claude (Fable 5) · 2026-07-04
Overall A− "Earned, not inflated."
Ops disciplineA+"Almost nobody evals their memory. This layer beats most funded teams." Frontier-drain latency 0.4h against a 168h target.
ArchitectureAFour tiers, two-pass lifecycle retrieval, gated writes, offline replica. Archive contamination ~70% → 0%.
FreshnessAFive-minute transcript indexing. A doc written that evening was the #1 hit the same night.
Memory→weightsACanon baked into model weights, in production — a June-2026 paper running three weeks after it hit arXiv. "Ahead of the field."
RetrievalB+Right doc #1 on every probe — but bi-encoder only, no reranker yet. One generation behind best practice. (We're fixing it.)
EvaluationB+Harness is excellent; the bench is saturated near the ceiling — time for harder queries.

The line I keep, in its words — measured against Letta-, Mem0-, and Zep-class systems: "architecture at parity, ops/eval layer above most, retrieval the one component behind." We're publishing the B+ right alongside the A+. That's what building in the open means.

I'll take an A- I didn't write myself. And notice what scored highest — not the model, the operations. Almost nobody grades their memory at all; ours gets graded every Monday and pages us when the number slips. It's also consistent across every agent and every harness — the big-lab models and our local ones all read and write the same memory, so the experience of the AI stays continuous no matter which brain is driving. That cross-agent continuity is the exact thing the commercial "memory" features are only now reaching for. And when we audited how it's actually used, the finding was sharp: retrieval is basically solved — the fact you need is almost always the top result — so the real frontier is invocation: getting the agent to look before it leaps.

We didn't need a bigger model. We needed a memory — and the nerve to have it graded.

Era Five
The month local models got real

For most of this journey, local models were a toy. A wonderful, instructive toy — you learned the shape of the thing, the taste of the stew — but you wouldn't put them on a customer's phone line. They were how you learned; they weren't the meal.

That changed in the last couple of months, and it changed fast. Nine-billion-parameter models with multi-token prediction stacked on. Speculative decoding — DFlash, the DeepSeek "DSpark" work — pushing a single small GPU past ~120 tokens/second. Thank you, DeepSeek. Thank you, Zhipu and z.ai. The open-weights world did in a season what looked like years of runway.

The result: local inference on a laptop now feels the way ChatGPT felt six months ago. Genuinely. And it's running on hardware you own, on data that never leaves the building — which, when you answer real calls for real people, isn't a nice-to-have. It's the whole point.

An opinion, since it's my blog

Era Six
The computer isn't the computer anymore

"AI integration" used to be a slide. Now it's literal. The machine on your desk is not the machine you had two years ago, and the one in your pocket isn't either.

Which is why the loudest anti-AI argument — the water and energy panic — needs to grow up a little. Recirculation, heat capture, carbon footprint: real, worth engineering, I'm all for it. But the person posting "AI wastes water" from a phone that ships with a three-billion-parameter model baked into the silicon — Android or Apple, doesn't matter — is already using the thing they're mad about. You didn't opt out. There's a model in your pocket running whether you like it or not.

Nobody needs a buggy whip when there aren't any buggies.

That's what this reads like from inside the workshop. The people insisting it's all waste and hype are the buggy-whip makers watching the first cars sputter past — not wrong that the cars are loud and leak oil, just wrong about which way the road goes. The world is changing every second. Better to have your hands on it.


So that's the arc: quit fighting the OS, build a real platform, give the model hands, give it a memory worth grading, and wait for local hardware to catch up to the ambition — which it finally has. None of it was a master plan. It was a lot of wrong versions, written down, until our hands knew the shape.

We build in the open. If you want the receipts: the Fleet Log is a month of this read back from the fleet's own memory, and there's more in The Log — including how we rebuilt our video-understanding pipeline (/watch) to beat a trending open-source tool at frame selection while keeping instant, timestamped transcripts. More as it happens.

ScrappyLabs · built in the open · Asheville, NC · [email protected]