The Workaround Is the Warning Sign

Nearly every successful CI/CD adoption story starts the same way.

Someone got frustrated enough to just deploy something. Not through a formal process. Not after a committee approved a tooling recommendation. Someone looked at how the team was deploying software, manually, inconsistently, slowly, nervously, and decided they’d had enough, and they stood up Jenkins on a spare VM and showed everyone what was possible.

This happened so many times across so many organizations that it’s practically a genre. The frustrated engineer, the spare VM, the thing that shouldn’t have worked as well as it did. Jenkins was the canonical version for years. I dropped Jenkins into more engagements than I can count, had a whole playbook for it, could stand up a working CI/CD pipeline in an afternoon. Did terrible things with Jenkins. It was so open and flexible that you could make it do almost anything, which also meant you could make it do almost anything badly.

But it worked. And working, in an environment where the alternative was manual deployments and crossed fingers, was enough to change everything.

That’s not a Jenkins story. That’s a workaround story. And workaround stories almost always end the same way.

---

What the Workaround Is Actually Saying

When your best engineers build workarounds, they’re sending a signal.

Not a complaint. Not a request. A signal that the system they’re working in isn’t meeting the need, and that the person who could fix it has instead found a way around it. The workaround is evidence of both the problem and the capability that exists to solve it. It just went sideways.

The frustrated engineer who deployed Jenkins wasn’t wrong. The deployment process was broken. The workaround was better than what existed. The signal was real.

The problem is what happens next. The workaround works. Other people start using it. It gets more sophisticated. It accumulates configuration. It develops dependencies. It becomes the thing people rely on. And at some point it stops being a workaround and starts being infrastructure, without ever being designed as infrastructure, without ever being evaluated as infrastructure, without anyone making a deliberate decision that this is the right tool for this role at this scale.

Years pass. The person who built it has moved on. The organization has built pipelines on top of pipelines. The Jenkins instance is running on hardware that was supposed to be replaced two years ago. Nobody knows why certain jobs are structured the way they are. The documentation, if it exists, describes what the system does but not the thinking that produced it.

The workaround became load-bearing. And load-bearing things are very hard to replace.

---

The Inventory Problem

The first thing I do when I walk into an organization with a significant workaround problem is try to inventory them.

This is harder than it sounds. Workarounds don’t announce themselves. They don’t have a ticket number or an architecture diagram. They exist in scripts on shared drives, in Jenkins jobs with names that made sense in 2017, in Slack bots that do things nobody fully understands, in deployment processes that work if you do them in the right order but nobody wrote the order down.

The inventory question that surfaces them: what are the things your best engineers do that nobody else knows how to do?

The answers are almost always workarounds. The engineer who knows which server to SSH into before running the deployment. The one who wrote the script that fixes the thing that breaks every Tuesday. The one whose laptop has the credentials that make the integration work. When the knowledge lives in a person rather than the system, it’s almost always because someone solved a problem informally and the solution never made it back into the system.

This is where the warning sign becomes expensive. The workaround is now a single point of failure with a human face. When that person leaves, or gets sick, or is just unavailable at the wrong moment, the thing they built becomes the thing that blocks everyone else.

---

The Engineers Aren’t the Villains

Before going further, something worth saying directly: the engineers who built the workarounds are not the problem.

The instinct in most organizations is to treat workarounds as compliance failures. Someone used unsanctioned tooling. Someone went outside the approved process. Someone did the wrong thing and now we have a mess to clean up. That framing makes the engineer the villain. It’s exactly backwards.

The engineer who deployed Jenkins on a spare VM wasn’t violating policy for sport. They were solving a real problem with the tools available to them because the official system wasn’t solving it. They cared enough to fix something. They had the skills to fix it. They looked at the gap between what the team needed and what the system provided, and they closed it.

That’s not a failure of compliance. That’s engineering. The workaround exists because someone gave enough of a damn to build it.

Treating that as a violation is how organizations train their best people to stop caring. The engineer who gets told “you shouldn’t have deployed that without approval” learns that initiative is punished. The next time there’s a gap, they wait for someone else to fix it. The gap stays.

The villain in the workaround story, if there is one, is the system that created the gap wide enough to route around. The engineer is the canary. The workaround is the data. The gap is the problem.

---

Why the Workaround Beats the System

The workaround wins in the short term for the same reason the Jenkins deployment won in the short term: it’s working right now, in the real environment, serving the actual need.

The right solution is usually slower to arrive. It has to go through a process. It has to get approved. It has to be evaluated against requirements that were written by someone who hasn’t talked to the engineers who do the actual work. By the time it arrives, the workaround has been running for six months and everyone has built their workflow around it.

This is not a failure of the engineers who built the workarounds. It’s a failure of the system to respond to real needs at the speed those needs exist. When the gap between “I have a problem” and “the system provides a solution” is measured in quarters, the engineers who solve problems in hours will always route around the system.

The workaround is the engineering team voting with their hands. Every informal script, every spare VM, every Jenkins job built outside the official tooling is a data point about where the official system isn’t meeting the need. Most organizations treat these as compliance problems. They’re actually product feedback.

---

What Good Looks Like

The organizations that handle this well don’t eliminate workarounds. They treat them as the starting point for what comes next.

When someone builds a workaround that works, the right response isn’t to shut it down for using unapproved tooling. The right response is to understand what problem it solved, evaluate whether the official system should solve that problem, and either absorb the workaround into the platform or build the capability properly so the workaround isn’t necessary.

This requires the platform team to treat the engineers who built the workarounds as the experts they are. The Jenkins deployment worked because the person who built it understood the actual problem. Replacing it with something worse, even if the something worse is officially sanctioned, doesn’t serve anyone.

The best platform teams I’ve worked with run a regular process of workaround archaeology. They find the informal systems, they talk to the people who built them, they understand what need was being met, and they make a deliberate decision about what to do next. Some workarounds get formalized. Some get replaced with something better. Some get documented and left alone because the cost of replacing them exceeds the cost of maintaining them.

The inventory is the starting point. The conversation with the engineer who built it is the second step. Everything else follows from that.

---

The Jenkins Lesson

Jenkins taught the industry something important, even if most organizations didn’t learn it cleanly.

The lesson isn’t that Jenkins was good. It was good enough, which is different. The lesson is that a working solution deployed today beats a correct solution deployed in six months, and that engineers who are frustrated enough will deploy something regardless of whether the organization has sanctioned it.

The organizations that learned this built better platforms. They got faster at responding to real needs. They created paths for good workarounds to become official capabilities instead of accumulating as technical debt. They treated the frustrated engineer with the spare VM as a signal about where the platform needed to go.

The organizations that didn’t learn it are still running Jenkins jobs from 2016 that nobody will touch because everyone is afraid of what will break.

Your best engineers are telling you something every time they route around the system instead of through it. The workaround is the warning sign. The question is whether you’re reading it.