Field Notes
Jay's blog.
You will almost certainly build them
When I first started working in Big Tech, I was surprised at how much of their development tooling was house-made. Version control wasn’t Github, internal messaging wasn’t Slack — except at Salesforce, which notably bought Slack the company — and project tracking wasn’t Jira. Instead the big companies used custom tools they made themselves.
At first I thought this was just because the big guys have so many resources (they do) and so much hubris (and who wouldn’t) that they preferred their own solutions as better than packaged ones. Although using these tools they didn’t necessarily feel like better products than the packaged ones, and they certainly weren’t as polished.
Over time, though, I started to see how the custom tools did exactly what the companies wanted done, in the exact way they wanted to do it. And, at scale, that made a huge difference. More than anything, this allowed the companies to create their own processes and culture, and know that the tools would serve that. Reinforce it, even. And given how unwieldy processes and culture will get across tens of thousands of people, this was worth the investment.
At smaller companies, of course, custom tooling was a luxury that rarely justified the ROI. The tradeoff was that, in return, the company subtly loses a little control of its contours. You use Slack for collaboration and you become a Slack-shaped org. You use Jira for tracking and, sure enough, over time you become a Jira-shaped org. Ultimately everybody thinks in terms that work well for Github, Jira, Slack, whatever the tools are. And it’s fine, it’s better than having your engineers tied up writing and maintaining code for a custom version control system.
Until coding is automated and cheap.
Suddenly the agentic engineering org has made two big shifts: 1) it is producing way more output, so the effects of tools on its culture is more pronounced. And 2) the cost of building custom tooling is way lower.
I believe that this will result in many more small-to-medium size orgs building a lot more tools. For instance, Slack is, at first glance, a convenient place to wire up your agents and collaborate with them. But Slack is probably not the best surface at every engineering org for humans to operate agents, and if your org would do better with something else, it’s a lot easier than it used to be to build and maintain what you want. In fact, it’s probably easier to build and maintain your own system than it is to constantly try to shoehorn what you need through Slack - particularly when engineers are producing many multiples of PRs, reviews, and discussion more than they used to.
I don’t think this shift will be monolithic or universal. Packaged tooling makes sense when a need is both very generalized across an industry, and also very complex. Slack proved to be able to address the specifics of team collaboration across many companies, in a deep and reliable way that made them almost impossible to beat in terms of functionality for many companies.
Similarly, it’s easy to imagine that certain needs of the agentic engineering org — perhaps the knowledge base or the memory/context system — will prove well-suited to complex yet general solutions.
That said, in the meantime I think you should just plan to build a lot of what you want. It’ll be cheaper, and most importantly, more conducive to creating the company you want and the culture you believe in.
Big is the new small
The ramifications of scaling up coding via agents are pretty interesting.
I do think individual coders can attain massive increases in output of greenfield product work agentically. A single dev can manage, let's say, 5-10 agentic crews working 24 hours a day in separate areas of a codebase. This makes a single engineer into a leader of ~25-50 coders, with output increases to match.
Any experienced dev reading the previous paragraph will point out that this creates an orchestration nightmare -- a single person coordinating the work of 25-50 devs who work at different times and can't possibly know what all the others are doing, is a recipe for disaster.
(It's true.)
And then imagine an org of, say, 25 engineers, each of whom starts working agentically. At 25 engineers, you've got three teams, each team's lead can coordinate with the others, that's not a big deal. At 25 engineers working 50 coders each, that's 1,250 coders to coordinate, which is a completely different type of problem.
The fun thing is, this problem has known solutions -- because there are plenty of engineering organizations with a thousand devs. There's even at least a handful of companies with ~10,000 devs. I've worked at 3 of them, and it's interesting seeing the processes there suddenly become relevant for teams that number in the teens.
Big tech companies have whole groups dedicated to the problem of: how do we get engineers the documentation and information they need, narrowly scoped to exactly what they're working on, at the time they need it, with minimal effort required by the engineer.
Of course you will recognize this as the exact challenge agentic systems face: help LLMs to work well without drowning them in cognitive overload, by surfacing only the context/tools/skills they need, at the moment they are materialized.
And there's all these systems to do this. Mostly it's a paper trail that looks like bureaucracy run amok: architecture documents, decision records, runbooks, everything versioned and logged and kept up to date. And these are the tools preferred by the LLMs when you start building software with them, which makes sense since they of course trained their systems from human ones.
What I've found from playing around with these, is that it works really well. It's so important in agentic development to keep the robots on track, and nothing does a better job than a mountain of documentation. Of course, the robots also write the docs and maintain the docs (which, to be honest, they do a much better job at than people do). They don't mind that job and I think it's important to make sure they do it. Just as with people, when the project has hundreds of thousands or millions of lines of code, it's sure nice to have all the old decisions and obscure architecture documented, when a significant change is needed or some unexpected outcome happens.
A New Problem Space
I mentioned in an earlier post that, as a developer working in their spare time to make production-grade software, agentic engineering increases my effectiveness around 50x over hand-coding.
Mature engineering organizations aren't seeing those types of gains for a multitude of reasons which are mostly easy to suss out. I think the current rule of thumb is that mature organizations are seeing gains on the order of 25% as a first step if they can effectively introduce agentic development. Which seems like it might not be worth it, when you're also, to some extent, giving up control of the your code to a robot.
Meanwhile, all sorts of new concerns arise.
LLMs are, in many instances, unreliable. Give them too much context, too much information, too many tools, and they bog down with cognitive overload. Give them too little, and they hallucinate the information they need. The technical requirements of a system that addresses this conundrum effectively can be extremely complex.
They're also expensive, which the Claude subscription model tends to hide (these subscriptions are super subsidized). With a good tracking system like what we use in our dashboards, you can see what the API-equivalent costs are for subscription calls and it can be eye-opening -- a $200/month subscription can easily absorb $10,000 worth of API calls in that month.
I like building things, and I've always taken pride in architecting elegant code. There's something extremely satisfying about getting an object just right, where it maps perfectly to the problem domain, where it's clearly reusable and maintainable and will serve the system quietly and beautifully. With agentic engineering, honestly, that all goes away.
Instead, the problem space shifts, from engineering software to engineering a software factory. How do we build this system -- even a system for a single developer on a single computer -- that runs more efficently, producing higer quality outputs at a lower cost? The good news is that it is real engineering, with a lot of levers that we can bring our expetise to, even though its a totally new problem space from where we recently spent our focus.
The Politics of Competence
A colleague recently told me that they find it hard to learn about the state of agentic engineering through talking with friends in the industry, because many are reluctant to open such a converstaion for fear of being seen as an AI enthusiast or more extremely, a self-styled techno-futurist edgelord.
It's a reasonable concern.
Personally, I'm pretty convinced that Generative AI as it exists at this moment is destructive to our ecology, harmful to our social fabric, and totally unnecessary for us as people to thrive. Of course I also think the same is true about cars. And for many folks, it is essential they know how to drive and have a basic understanding of how cars work.
My sense is that this is the state of software engineering right now, and will be for a while into the future. I think these tools are landing in our industry with some weight, and the time is coming when we will need to be skilled with them in order to do what we do. Acknowledging that doesn't require losing sight of the real costs of this technology. If anything, an assessment of those costs has more credibility when it comes from someone who demonstrably understands the tools and how to use them.
Greetings
Welcome to our little neighborhood software lab.
What you see here was inspired by a conundrum I faced in my day job: we were working to chart a course forward into the world of AI-augmented software development, but my personal experience with these tools was limited and I was relying on what I could find online and in newsletters. And online content, in particular, was overwhelmingly just AI slop and/or clearly unsubstantiated hype and/or possibly unsubstantiated anti-hype.
I started contacting friends in the industry to see what they had found to read and learn from, until one friend told me something that was the big unlock. "Dude", he said, "just ask Claude." It has access to a world of information that would take you weeks to sort through, it can even put together a lesson plan for you if you want to learn something of deep complexity.
So I asked Claude. One thing led to another and I built a whole agentic software lab in my spare hours. In the course of that I learned a lot of basics about agentic development and finally started to get my mind around the issues we were working on.
The follow-on problem at my day job was that, as we each learned more about agentic engineering, we weren't learning from the same corpus. Without a shared understanding of what terms meant, how a system might work, and what was acheivable, we found oursleves talking past each other and spending a lot of time reconciling our basic ideas before we could build on them.
So I built another software lab, as a demonstration, on my employer's systems, to show what my working mental model was. But this lab was necessarily very limited -- I wasn't going to use the codebase of an active production SaaS to work out, for instance, the pros and cons of overnight autonomous agents and different cost-containment methods.
Finally, I came up with what you see here. I opened up my original, private experimental development system to public view so anyone can check it out. The specifics of the software beign developed are largely redacted, to help generalize the system and keep focus on the process. But you can see how the system tracks itself in dashboards, how costs are observed and optimized, how we manage our knowledge base, and so on.
Additionally, I packaged the basics into a starter kit so anyone can download it onto their own machine and begin down the path of agentic engineering with just a few minutes of setup and a Claude subscription. (And no, I don't have any relationship with Anthropic, it's just the system I happened to be using when I did this, so it's the one I know how to use the best.)
Why would you want to do this? Well, I can't say that you do, but here's the first thing I noticed when I did: it increased my personal velocity as a software developer by something like 50x.
Your mileage will clearly vary; I was only doing development in my spare time so the baseline was really low. I'd been working on a multitenant SaaS as a side project, whenever I could make large enough blocks of time to focus, for a couple years, and realistically was no more than 10% of the way to having a shippable product.
Once I got facile in agentic development, I found it productive and satisfying to dip into development for ten minutes, set up and launch a process and return many hours later when I had time and mindspace to process the output. As a result, I was able to create a demo-able SaaS from scratch -- it took me just a short while to decide it was better to greenfield the product agentically than to try to adapt what I had already developed -- in about 3 months. I estimate the software will be production ready in another 2-3 months. And this is spending less time on it than I previously had been.
Meanwhile, if I encounter a need for a piece of software in my day-to-day life, at this point it's easier for me to build exactly what I want than it is to try to see what's available commercially and adapt it to my needs.
What I'm getting at here is this: I think these tools are demonstrating power to a degree that it is inevitable they will completely reshape the profession of software development. I don't claim to know what the end shape will be, but if you want to explore some of the possibilities, I hope you find this site useful.