We Need to Sound the Alarm on Technical Debt. Here’s How I Do It.
By Doug Needham, DataOps.live
Technical debt is a challenge for any digital team. Suggestions to avoid it get overridden or ignored, and it’s a rare treat to get to fix the things we know need fixing. If only our PMs and data owners understood what we know—how terrifying technical debt really is—maybe they’d push back less when we fight to build things the right way.
I think I’ve found the way to tell them.
Fairy tales, parables, and stories our grandparents tell may not be factual, but they are “true.” They hold a kernel of truth that we remember when the time is right. The following tale I tell in that same spirit.
The Dangerous Race
You wanted your car built in a hurry, and here it is — oh, but twelve bolts were left over after we put it together.
The project managers have insisted that these particular bolts can wait for the next maintenance window.
The sales team assures you these bolts are not necessary.
The engineering team knows what these bolts will hold together. They recommend taking the time to apply the bolts.
The race will begin soon.
The clock is ticking.
You are the driver.
You will be making life-or-death decisions at breakneck speeds on a racetrack designed for demonstrating your car’s capabilities.
There are thirty-nine other cars on the starting line, each with their own engineering, sales, and project management teams.
Do they have all of the parts of their cars held together?
Which parts of your car are not held together in the best way possible?
By using this car and pushing it to the limit, will you win the race, or end up a puddle on turn 3?
Moral of the story
This is technical debt: a high-risk situation that could have easily been avoided by listening to the experts.
If you need a quicker analogy, it’s like Russian roulette, only you do not know the number of chambers, the caliber of the bullet, how many bullets are loaded, or which way the gun is facing.
Non-technical stakeholders sometimes confuse refactoring for technical debt. It is certainly true that there are times when architects and engineers learn better ways to build something after it’s been built. This is not technical debt. This is version efficiency.
In either case, those people actually doing the implementations should be able to decide what needs to be done. Having non-technical users make technical decisions about how to build a tool is a recipe for disaster.
When your PM or data owner asks you to cut corners, remind them of this story. Ask, “Do you trust your crew to make the right decisions?
If so, then let them.”
Duty to warn
Users don’t understand the risks when they tell us to cut corners. They don’t know the importance of the right driver, structured SQL, updating the subroutines, adding a node to the cluster, or updating to the current patch level and getting a clean reboot.
They may understand driving a car they wanted built. You built them a wonderful car, but they dictated a few shortcuts that you knew needed to be fixed later, only later never came.
Since we do know the risks of technical debt, we have a responsibility to raise the alarm. Tell them the story of the race. Remind them that they are sitting in that car. They are the ones who told you not to fix the thing you know needed fixing.
I hope this story becomes an arrow in your quiver when a business user or project manager tries to overrule you architects and engineers who know something needs to be done, and it needs to be done the right way. Share this story widely, and let me know if it helps convey your message.
Technical debt is something we all live with. Choosing the tools we use to mitigate technical debt risk is a decision still in our hands. A tool like DataOps.live provides a codified architecture that ensures data management is governed by a deterministic set of rules. You can stand on the shoulders of this team and rest easy at night, or take turn 3 at full speed!
