Every number I've published so far came from rented GPUs in the cloud. Mostly a single 48 GB card, rented by the hour, running an open model. That's a great way to start. You pay for what you use, you can grab a bigger card for an afternoon, and you hand it back when you're done.
But the GPU was never what held me back. My laptop was.
#The real bottleneck
The rented GPU does the thinking. Everything around it ran on my MacBook: the sandboxes, the copies of the code, the test runs, the builds, the grading. Every agent drags all of that along with it.
One agent is nothing. Five is fine. Then you get greedy, because more agents at once means more ideas tested by morning, and the laptop starts to sweat. It has 36 GB of memory. I have run it out of memory and killed it seven times now. Not slowed it down. Killed it. Every lane I had running stopped dead in its tracks, including countless sub-agents that couldn't be resumed.
So the ceiling I'm actually hitting isn't tokens or model size. It's room to run more agents side by side. StateFrame already keeps each agent's prompt inside a fixed budget, so the model side stays predictable however long a run goes. What grows is how many agents I want running at once, and that takes memory and cores.
#Why not just use the cloud?
Fair question. I already run Mission Control and my other production workloads on Kubernetes in AWS. I could have pointed the agents there and called it a day.
I didn't, on purpose. The foundry is where experiments happen, and experiments are messy. They eat memory, they run hot for hours, and some of them fall over in creative ways. None of that belongs anywhere near production, and none of it should show up on a cloud bill that scales with how curious I got this week.
The wall works in both directions, too. Production data never leaves its secure production environment. No customer data ends up on these machines. The foundry works on code, test tasks and benchmarks, not on anyone's real data.
So production stays boring and stable in the cloud, and the foundry gets to be loud and break things in my house. Which meant I needed something to put in my house.
Turns out I already had it. Down in my basement there's a mini-rack left over from an old home lab I haven't touched in a long time, with a few servers sitting in it collecting dust. So it's time to dust them off and bring them back to life.
#Meet the big server (Foundry-1)
First one out of retirement is a Dell PowerEdge R720XD, and I'm standing it up today. On Monday it slides into the mini-rack.
- System memory
- ~300 GB
- DDR3, blazing fast by 2012 standards
- GPU
- 24 GB
- NVIDIA Tesla P40, passively cooled
- Spinning storage
- 20+ TB
- Mechanical hard drives, almost every bay full
- CPU cores
- Plenty
- Two Xeon sockets, more threads than I can count
Let me be clear about what this is. The R720XD came out around 2012. The DDR3 in it is not fast. There's just a ridiculous amount of it. The Xeons won't win any single-thread contests, but there are so many cores that I've stopped counting them. And the 20+ TB of storage is the old-fashioned kind, with platters that actually spin.
The star is the Tesla P40, a 2016 datacenter card with 24 GB of memory and no fans at all. It counts on the server's own fans blowing air straight through it, which is exactly why it lives in a server and not on my desk. It also wants an 8-pin CPU-style power cable instead of a normal GPU one. It's a little bit of a project, and I love it.
Old as it is, it's exactly what the laptop wasn't:
- All that memory is room for far more agents at once, each with its own sandbox, without the whole machine falling over when I start one too many.
- All those cores handle the busy work every agent generates: test runs, builds, grading. More cores means more of that happening in parallel instead of in a queue.
- All that storage holds what I keep. StateFrame saves every request so you can see exactly what the model saw on any turn, and with this many agents that adds up fast.
- The P40 gives the foundry a GPU of its own, so not every experiment has to wait on a rented one.
This is where the foundry lives for now. It's mine, once it's up it stays on, and nobody is going to reclaim it halfway through a run.
#And it has company coming
The R720XD won't be alone for long. Two more machines from that same rack are next in line for a dusting and should be back online soon:
- A Hyve server with two CPUs, 128 GB of DDR3 and an NVIDIA Tesla P4. Same vintage energy, more cores and memory for more agents.
- A Dell R640 with 128 GB of DDR4, which is practically futuristic around here, and two more P4s.
The P4 is the P40's little sibling from the same 2016 generation: 8 GB each, small, low power, and happy to live in a 1U box.
That's three servers, four GPUs and about 550 GB of memory between them, none of it rented. I'm calling it a fleet. The electric bill is calling it something else.
To be clear, this is not a multi-million dollar GPU cluster. Nobody is going to mistake my basement for a datacenter. But for now, it will do.
#What comes next
Simple: put it to work. More agents running side by side, more ideas tested by morning, and hopefully no eighth crash. My laptop's answer to "how many agents?" was "fewer than I wanted," seven separate times. The foundry gets to answer that question now.
If you're running a pile of agents on your own hardware, I'd love to hear where your ceiling is. Open an issue on GitHub with what you ran and what happened. Results from setups I don't own are worth more than anything I can test myself.
So that's the plan for world domination: a laptop I keep crashing, a loud server from 2012 with a GPU from 2016, and two more from the same dusty basement rack that aren't much younger. Big tech should be very worried.