Earlier this year, OpenAI’s CEO Sam Altman admitted something that should make every engineering leader pause: they’re “currently losing money” on ChatGPT Pro subscriptions, which run $200 per month.
Let that sink in. A company charging two hundred dollars a month for AI access — 10 times what most SaaS products dare to ask — is still bleeding cash on every user.
This isn’t a pricing problem. It’s a physics problem. And it perfectly illustrates why AI’s productivity promises are crashing into the same constraints that have governed software development since someone first uttered “works on my machine.”
The Paradox of AI Productivity
Here’s what nobody wants to admit: AI makes individual contributors more productive — which makes everything else worse. It’s like giving everyone a Ferrari in a city built for horses. Sure, you can go 200 mph, but you’re still stuck behind the same old cart.
Think about it. When a developer can generate 10x more code with AI assistance, what happens? That code still needs to be:
- Reviewed by humans who read at human speed (for legal and compliance – e.g. SOC)
- Tested by systems designed for human-scale output
- Deployed through pipelines built for human-paced development (we’ll need GitHub Actions FastPasses™️)
- Run on infrastructure budgeted for human productivity levels
The Theory of Constraints tells us that every system is limited by its narrowest bottleneck. AI doesn’t eliminate these bottlenecks — it moves them around and makes them more expensive. Eliyahu Goldratt, the author of the theory, is somewhere laughing at us for thinking we could hack our way out of systems thinking.
Constraint 1: The Code
In 2024, GitClear tracked an 8-fold increase in duplicated code blocks, with redundancy levels now 10 times higher than in 2022. That’s not a typo. AI is turning our codebases into digital hoarder houses, filled with almost-but-not-quite-identical implementations of the same functionality.
The code constraint has evolved from “how fast can we write it?” to a more complex set of problems:
Quality chaos: AI doesn’t know your coding standards. It learned from the entire internet, including that Stack Overflow answer from 2012 that you know is wrong but keeps getting upvoted. The result? Code that works but makes your senior engineers cry. (Yes — you can use your own code as training and context; however, most of the code AI was trained on is not yours.)
Attribution anxiety: When your AI assistant suggests code, where did it come from? Three out of four low-hallucination users don’t fully trust what AI generates. We’re shipping code we don’t fully understand, from sources we can’t verify.
The Mythical Man-Month lives: In his book, The Mythical Man-Month: Essays on Software Engineering, first published 50 (!) years ago, Frederick Brooks warned us that adding developers to a late project makes it later. Now we’re adding AI developers that can write code 24/7. Guess what? The communication overhead didn’t disappear — it got worse. Your AI can’t attend standup.
Constraint 2: The Server
Remember when your biggest infrastructure worry was whether to use t2.micro or t2.small? Those were simpler times. Now we’re dealing with increased Cloud Computing Costs, and AI is the accelerant on this fire.
The server constraint has mutated into something more sinister:
GPU hunger games: Your AI models don’t run on hopes and dreams. They need GPUs, and lots of them. Sam Altman has complained about a lack of GPUs and pressure on OpenAI’s servers multiple times, including lamenting that their “GPUs [were] melting.” If OpenAI can’t keep their servers cool, what chance do the rest of us have?
Deployment bottlenecks: Your carefully crafted CI/CD pipeline was designed for code that humans write at human speed. Now it’s trying to process AI-generated pull requests that come faster than your build servers can compile. It’s like upgrading from a garden hose to a fire hose but keeping the same bucket.
The new shared services: AI agents, model serving endpoints, vector databases—every team wants their own AI infrastructure. Soon you’ll need a service mesh just to keep track of your service meshes. Maybe even a Trace Buster Buster?
Local vs. remote chaos: Developers want to run AI locally for speed. Ops wants everything centralized for governance. Security wants nothing to run anywhere. Meanwhile, your laptop fans sound like they’re preparing for takeoff every time you open your IDE.
Constraint 3: The Wallet
Ah, money. My dad’s universal advice, “Everybody talks to Mr. Green.” The universal constraint that makes CFOs wake up in cold sweats. If you thought cloud costs were bad, wait until you see your AI bills.
In one report, “Every executive [reports] the cancellation or postponement of at least one generative AI initiative due to cost concerns”. Every. Single. One. This isn’t a “some companies” problem — it’s an everyone problem.
The wallet constraint is where theory meets reality:
Stochastic spending: Traditional software has predictable costs. You know how much a server costs per hour. But AI? A single query on the company’s most advanced models can cost a staggering $1,000. One query! That’s not a service — that’s a mortgage payment.
Hidden infrastructure costs: Sure, the API call costs $0.02. But what about the vector database to store embeddings? The GPU cluster for fine-tuning? The logging infrastructure to debug why your AI just told a customer to eat rocks? These costs hide in the shadows until the monthly bill arrives.
Unit economics from hell: How do you price a product if you can’t attribute costs at all levels of the SLDC — from implementation to running workloads?
The redundancy tax: Every team wants their own AI experiments. Every developer needs their own API keys. Every project spins up its own infrastructure. Without governance, you’re not just paying for AI — you’re paying for AI waste.
The Critical Thinking Imperative
I’m a fan of the “How I AI” podcast. Successful AI adoption isn’t about the tools — it’s about the thinking. Every guest who’s genuinely succeeding with AI has one thing in common: they think critically about systems, constraints, and tradeoffs.
Also: as Ryan Holiday discussed with Rick Rubin on The Daily Stoic podcast, “AI Isn’t Stealing Creativity, It’s Supercharging It” — but only for those who bring creativity to the table. AI amplifies what you already have. If you have systems thinking, it amplifies that. If you have chaos, well… enjoy your amplified chaos.
The same applies to DevOps. AI doesn’t replace the need for:
- Thoughtful architecture that anticipates bottlenecks
- Governance that balances innovation with control
- Financial discipline that treats compute like the scarce resource it is
- Human judgment about what’s worth automating
So what can and should AI do?
AI excels at amplifying human creativity, automating repetitive tasks at scale, and surfacing patterns we might miss — but only when we’ve built the systems to harness it. It should augment our decision-making and accelerate our best practices, not replace the critical thinking that makes those practices work in the first place.
We cannot offload critical thinking in the age of AI. In fact, we need more of it than ever.
Recognizing Your Constraints
So how do you know which constraint is killing you? Here’s your diagnostic:
Code Constraint Symptoms:
- PR review queues growing exponentially
- “Nobody understands this code” becoming a daily standup item
- Git blame increasingly points to “AI Assistant”
- Your senior engineers are spending more time reviewing than building
Server Constraint Symptoms:
- Build times measured in geological epochs
- “It works locally” requires a 64GB MacBook Pro
- Your AWS bill has more digits than your phone number
- Deploy freezes because “the GPUs are busy”
Wallet Constraint Symptoms:
- Finance asks for a meeting. Again.
- “Can we use the smaller model?” becomes your new mantra
- You start pricing everything in terms of GPU-hours
- The phrase “burn rate” takes on new meaning
The irony? These constraints are interconnected. Solve one, and the others get worse. Optimize your code generation, and your review bottleneck explodes. Speed up deployment, and your infrastructure costs skyrocket. Cut costs by using smaller models, and your code quality plummets.
The Path Forward
The Theory of Constraints isn’t just relevant in the AI age — it’s essential. Every system has a bottleneck. Find it, fix it, and prepare for the next one. But remember: in AI-augmented development, the bottlenecks move faster than ever.
The winners won’t be those who generate the most code or deploy the most models. They’ll be the ones who think systematically about constraints, who build governance before they need it, and who remember that AI is a tool, not a strategy.
Because here’s the truth: AI will supercharge your development. The question is whether it supercharges your productivity or your problems. Without systems thinking, you’re just adding rocket fuel to a dumpster fire.
And at $1,000 per query, that’s one expensive fire.
The Cloud Cost Playbook
The step-by-step guide to cost maturity
