Sunk Cost Fallacy Engineering

Reading Time: 4 minutes

While working on permanent.link this week, I ran into the same issue twice. I had built something that was working great, only to scrap it for another solution a few days after. The first time this occurred, I had just finished my infrastructure for allowing custom domains to be used for permanent links. The other time was when I had built a serverless solution for my background URL monitoring.

In each case, I had spent the whole day devising and building something really cool and technically challenging and figured out shortly after that it wouldn’t be enough. It took me a lot of mental work both times to accept that I had been wrong and needed to rethink my approach.

Today, I want to talk about the sunk cost fallacy and its particularly devious impact on bootstrapped businesses.

Imagine going to the cinema, paying for the expensive seats, only to watch a movie that is a total waste of your time. Do you stay? Or do you leave early? Even though people are in complete control of how they spend their time, many choose to stay and watch until the end. No matter if they leave within the first few minutes or sit through hours of sub-par cinematography, they won’t get their tickets reimbursed. Still, they stay and watch a movie they don’t like instead of actually doing something nice. Just because they paid for it. That’s the sunk cost fallacy.

The story is always the same for founders as well: we work on something for a long time, pouring all our efforts and skills into a project. Then, we suddenly learn something new, and it shines a new light on our work, making it less attractive as a solution to a problem. We don’t want our efforts to go to waste, even when they allowed us to learn something important in the first place. We cling to choices we made with less knowledge. Often, that leads to us self-sabotaging our own efforts to build something that people need.

Validating our assumptions, as much as we claim to want it, is often less enjoyable than we want it to be. It definitely was in my case. Thankfully, I have become more aware of my own reactions to new information. Whenever I notice myself thinking, “this shouldn’t be,” I try to rephrase it to “this is something that I didn’t want to happen—why is that?” My automatic response to information that invalidates my prior efforts is to doubt it. That’s the sunk cost fallacy in action. Right then, when new facts are staring you in the face, it is a chance to grow. I have to remind myself of this every single time, and the awareness of me clinging to what I had built before really helps with that.

In both cases this week, this perspective has allowed me to move forward without regretting my prior actions.

My custom domain integration — and I talked about that at length on my podcast last week — was really exciting to build. It involved LetsEncrypt and a lot of cool JavaScript. The day after I implemented this, I talked about it at length on Twitter and had a really nice DM conversation with another founder who’s interested in building something for that. A day or two later, he shared a link to a Tweet by the maintainer of the LetsEncrypt JavaScript library. He mentioned that the version that I was using would stop working on October 31st, 2020, due to a LetsEncrypt API change. I looked at the calendar, and it was October 29th. After a minute of frantic facepalming, I took a deep breath and explored alternatives. My Twitter friend eventually recommended a different solution, which I spent a day implementing, and that one turned out to be much faster, more reliable, and easier to maintain. If I hadn’t built the JS-based solution in the first place, I would never have talked about it and never learned of a better way. What looked like a sunk cost was an investment. It often is.

My serverless monitoring infrastructure was not long for this world, either. Within a day, I learned how to build and deploy a little pinging script into 14 different AWS Lambda regions. I needed geo-diversity for my monitoring scripts because some websites like Amazon and YouTube actively fight scrapers, even when they’re just issuing HEAD requests to see if pages respond to HTTP requests. I thought that by using the AWS Lambda regions, I could reliably circumvent those bans and blocks. Well, it turns out that was not the case. Just a few hours after pushing the new serverless monitoring service to production, Amazon started blocking those endpoints as well. My grand plan foiled, I looked at alternatives, and just like with the custom domains, a Twitter conversation with a fellow German developer helped me out. I found an affordable and reliable scraping API that leveraged a global network of servers and IPs, avoiding blocks and captchas. I implemented that for the offending URLs only at first, but when my serverless functions turned out to be significantly slower and less reliable even for the servers that didn’t block them, I moved over to the API.

So, what did I get out of this? A new technology and a new dependency. But most importantly, I found better solutions. Better solutions to problems that are at the core of permanent.link. My monitoring is now incredibly stable, and custom domains are redirected super fast and reliably.

And I feel good about it. The things that felt like wasted effort at the time were just steps towards something more significant. Once I got past my pride in figuring things out myself, using other solutions lead to superior results.

There is an entrepreneurial lesson in this, too. Fred Destin recently said on Twitter that “every startup is a portfolio of one.” For me, that means that I am the CEO of permanent.link as well as the CTO. Even more importantly, I am the only shareholder of the business. It was painful to see both my custom domain and monitoring systems being invalidated, but only the CTO felt that pain. For the CEO and the business owner, this was not a setback, but something that couldn’t have come fast enough: it was a necessary learning.

When you run into a technical sunk cost fallacy situation yourself in the future, try envisioning it as a lesson that you couldn’t have learned otherwise. Try to perceive it as something that moves your business forward. In high school, my math teacher quoted this saying whenever we ran into a dead end with a proof or theorem: “Vorwärts, Kameraden, wir müssen zurück!” — “Onwards, comrades, we’ve gotta head back!”

Sometimes, taking a step backward will allow you to leap so much further.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.