I was working on a side project this week, and I ran into an all-too-familiar pattern: the more I thought about the technical requirements for this project, the bigger it grew in my mind.
I only want to build this for myself to see if I could be Customer Zero. But it derailed quickly. What started as “What is the core functionality I need to solve my problem?” turned into “I probably need a multizonal cloud database with zero-downtime guarantees ten years down the road…”
Why does this happen all the time? Why do we over-engineer projects even before we write the first line of code?
As a software engineer, I’m trained to see problems and edge cases. Either by education or experience, I live more in the future than I am living in the present. I see a choice of technology not just as an “enabler of something right now” but also as a “source of technical debt later.” A commitment to a specific dependency today could mean a lot of work a year from now when I will need to switch it out for something that can handle future demand.
And that’s when I paused. Future demand. I was already counting on future demand — before I had even validated the need for myself! My mind was already looking at the problems of a full-grown business, while I hadn’t even thought about how I would implement the features I’d need for myself.
Clearly, I was projecting too much.
But how much is too much?
Combatting Initial Analysis Paralysis
Once I understood that I was projecting fictional technical debt into an uncertain future, I started to look for the things I could be sure of.
First of all, when you build a prototype, either for yourself or for beta customers, there is a clear and limited amount of stakeholders. Right there and then, the people who have something to lose are you and, when present, your potential beta testers.
And that’s it. There is no one else. There are no employees, no future co-founders, no investors, no acquirers. Just you, right now, and a handful of interested prospects.
An MVP is a validation machine. It’s not your future product. Scalability is irrelevant for most validation prototypes. The whole point is to build something simple that can do the job “well enough.” You could likely validate most ideas with your test data hard-coded into your application.
Consider your MVP to be something you will likely scrap. A test version of the real thing. A fake version of your final product.
Long-term technical considerations for an MVP will lead to analysis paralysis. If you assume you will shape your codebase from the initial prototype to the final product without ever rewriting it, you are front-loading a lot of decision-making that lacks all kinds of data. What if you learn that your chosen data model doesn’t work for your use case? What if your customers have certain requirements that your initial database choice can’t satisfy? What if you learn that an adjacent problem is much more critical to solve for your audience?
So many what-if’s. All of which can only be answered by actually “leaving the building” and validating your assumptions in the real world—validations from which you will need to iterate and improve.
It is at that point that the other stakeholders enter the arena. Your future self as an owner of a functional business should only appear on your consideration horizon at the point where actual business is forming. Not when you are conceptualizing your MVP.
The thing I build today is not the product I will sell in five years. It’s a validation step, not the goal. If you understand your initial choices to be limited in time and scope, you can severely reduce your levels of analysis paralysis.
Ask yourself:
- Will this allow me to build something that is good enough for a few dozen people to test?
- Will this be reliable enough for what I want to show as a “prototype?”
- Is the MVP tech I choose cheap enough to be easily built?
- Can I quickly scrap this prototype and take the “lessons learned” with me?
These questions are practical and actionable.
Save the questions about multizonal high-availability and scalability for when you have validated that you’re building something that deserves those considerations.
I want to share a story about how we approached this question over time.
When we started building FeedbackPanda, we made the choice of going with a DBaaS provider that I had worked with before—Compose. I chose that one because it offered very reliable databases without much fluff. It definitely wasn’t the cheapest option, but it was very convenient. The lowest tier was great for initial development and extremely cheap, under $20. While the pricing model could get quite expensive, my reasoning was that during building, running, and operating FeedbackPanda, I would never want to do anything related to database maintenance. I really don’t enjoy that, as it’s like open-heart surgery. It touches the source of truth for all our data, and I’d rather have the experts do that job.
Our little database didn’t need much scaling for the longest time. We really didn’t produce too much data with FeedbackPanda; most of it was text templates and generated student feedback. At some point, a few clever database vacuum cronjobs and finding unused indices allowed me to significantly being down the RAM usage of our database, keeping costs down as well.
Of course, data grows over time. At the point when we sold our business, our database expensed accounted for almost half of our monthly non-salary spend for the business. But it was a choice. We paid a four-figure sum to Compose every month so that we didn’t have to employ a database administrator. And they provided superb service over the years.
AND STILL, at some point, and this is why I am telling this story, I considered changing our database completely. Our servers were located in Frankfurt, and our American customers occasionally complained about latency issues. Half of those were due to their own slow computers and connections, but for many, being halfway around the world from our data center made a noticeable difference. Also, Compose would occasionally schedule maintenance windows right when the usage of FeedbackPanda was highest. Customers really don’t like your product to be offline right when they need it.
So I considered building FeedbackPanda into a multizonal, globally distributed application. No more lag, no more downtime. How hard could it be?
I looked into Google Spanner, a globally available cloud database with amazing distribution capabilities. My initial calculation got me to a monthly 16.000 dollars for the smallest available instance. I had a good laugh and scrapped my global distribution plans. Just wasn’t worth it.
What I did instead was looking into building more reliability into the existing system. We had had a few maintenance windows that surprised us before, where the database would be restarted or down for a few minutes.
Now that I knew we’d stick with Compose for a while longer, I dug into their infrastructure and found that they used something called portals, all proxies to the database. Different portal, different connection string, but the same database. Whenever they had maintenance, it would only affect a certain group of portals, always leaving the other groups available. While this wouldn’t fix the transatlantic lag, it would definitely fix the downtime issues.
All I needed to do was to register multiple portals for our database and build a small piece of connection logic that would randomly select one of those connection strings when the server started. Eventually, every running instance of our backend would find a database portal that worked and get connected within seconds.
This was a twenty-line code change and a double-digit monthly expense for the additional portals. It immediately removed the threat of lengthy maintenance windows that we had no control over. I spent maybe half a day working on this.
Compared to the $16.000 monthly fee for Spanner, this was something that made a lot of sense, was quickly done, and made the system better without adding complexity.
Your initial choices can go a long way if you want them to. There will be a point in time where it makes sense to consider alternatives and upgrades, and even then, they might not be the best choice—no need to rush these decisions.
