The Balancing Act: How Business Risk and Engineering Excellence Shape Software Companies
Navigating the tension between business priorities and engineering excellence in modern software companies
I took a few weeks' break from my Substack to focus on other projects. With limited time available, I realized that while I enjoy writing, I also want to experiment with different technologies and frameworks. Since these technical explorations require time, I've decided to reduce my posting frequency. This will allow me to dive deeper into other interests while continuing my Substack. Today's post explores why business priorities so often conflict with engineering departments. Something I've recently had many conversations about, so I wanted to get my thoughts down on paper.
Understanding Company Risk Profiles
Let's look at different operational modes of companies to better understand what we mean.
Consider these 3 fictional companies:
Company A - Its leadership aggressively pursues every potential customer and business case.
Company B - Has a stable product but occasionally takes on risky business cases that its current portfolio cannot fully support.
Company C - Focuses on operational excellence with rigid processes and timelines. However, I believe this approach is flawed—a healthy company needs occasional disruption to avoid being overtaken by smaller competitors.
These companies often reflect typical business stages (startup, scale-up, and enterprise), though real-world examples don't always fit this pattern.
The Risk-Reward Dynamic: Why would a company pursue risky business cases that require operational, process, and technical shortcuts? Often, it's because substantial revenue or market entry opportunities are at stake, opportunities that might not come again for a long time.
Businesses need revenue to fund their growth or attract investor money. While some companies choose to grow slowly, this path can be challenging, especially when facing tough financial conditions.
In recent discussions with various people, I've noticed something interesting about risk perception. While most departments understand both the risks and benefits of aggressive business decisions, engineering teams consistently mention the challenges of working under such circumstances. Interestingly, other departments often viewed engineering as a roadblock to any risky business case.
This sentiment was expressed across the board, which raises the question: why?
Balancing Engineering Excellence with Business Speed
As I mentioned in a previous post, engineers take pride in what we build. We value creating robust, performant, and modular systems. Like furniture makers who take pride in their craftsmanship, we strive for quality in our software. However, this commitment to excellence can create friction when facing business opportunities.
These business opportunities are typically fast-moving with tight deadlines that don't allow time for our usual quality-focused processes. Yet they're a business reality. Successful companies don't need to search for them; they naturally arise from customer needs.
Engineering departments often clash with these decisions because experienced engineers can foresee the future implications. They understand how shortcuts might force a system rewrite later when the product succeeds and data volumes increase.
But this forward-thinking mindset can impede our business perspective.
The product we build might never need that rewrite to handle more data, or it might not become wildly successful. However, by delivering it, we show customers they can count on us, which may lead to other opportunities. The product itself isn't always the end goal. Business outcomes are more nuanced. Yes, there's a risk the product might become wildly successful and require a rewrite, but what if we acknowledge and accept that risk together?
Building Trust Through Shared Risk and Responsibility
Throughout their careers, engineers often find themselves in situations where they're initially open to "hacking" quick solutions. However, this willingness tends to decrease over time. This isn't universally true; it depends on individual risk tolerance, and engineers generally tend to be risk-averse. Don't believe it? Just compare how many engineers invest in high-risk assets compared to employees in other departments.
Why do engineers become more cautious over time? Often, it's because their goodwill gets tested. Consider our earlier example:
We pursue a business case and build a prototype
The project succeeds, but performance issues arise
This is precisely the risk we warned about when taking those necessary shortcuts
When leadership hears customer complaints, they turn on engineering
This kind of experience makes engineers more reluctant to proceed without proper implementation.
What's the lesson here? Having engineers who support business goals is vital for any company. However, it's crucial to document risks together and share ownership. When everyone commits to moving faster, everyone should also share responsibility for the consequences. This culture enables continued collaboration on business cases and shared risk-taking.
While it's easy to point fingers at other departments, especially in larger companies, it's far more challenging and valuable to own decisions collectively, even years after they're made. Regardless of your position, avoiding blame is essential. The only result of blame is increased reluctance to take future risks.
Takeaways
In a capitalistic world, businesses survive by gaining market share, earning investor trust, and achieving growth. While business decisions often involve well-understood financial risks, technical risks, though frequently seen as a burden by leadership, must be taken seriously. Success depends on having an engineering department that shares ownership of these risks.
Will everyone be happy with these decisions? No. However, when technical teams help explain the risks and leadership avoids blame when things go wrong, it becomes a working best practice.
You need engineers willing to take risks, but you must also take their concerns seriously and avoid blame games. Engineers have long memories, and damaged trust makes future risk-taking much harder. This often leads to leadership complaining, "I could do so much more business if only my engineering department would support it." But perhaps the responsibility doesn't lie solely with engineering.
History provides many examples where business priorities overruled engineering concerns, leading to negative outcomes. In my next post, I'll examine two such cases, exploring:
The leadership culture at the time
Why the relationship between business and engineering plays such a crucial role in software company success