A few years ago I had a big challenge at work. With two or three other programmers we spent almost 8 hours browsing through code, drawing diagrams, swearing and thinking to the point our heads started to hurt. The conclusion was clear – we can’t do it. It’s impossible.

A few months before that day our batch job started processing too much data to fit a single processing window. Not a big deal. Business adapted. They started the job a few days early each month, to make sure all reports were ready before the deadline.

But now they came up with a new product. One requirement didn’t fit our model. From then on information that used to be always in one piece, could be spread across multiple records. What’s worse, related records didn’t have to be located next to each other and could arrive in  a random order.

We started analyzing edge cases. What if the processing window ends before all related records are processed? We’ll have incorrect results. We KNEW that was unacceptable. So we were thinking and thinking and thinking…

Eventually we realized that ensuring correctness at the end of each processing window is either impossible or extremely expensive. So on the very same day we set up the meeting and broke the bad news to the business. We felt uneasy about admitting that our system can’t accomodate their business idea. We were sure they’ll be extremely angry and unhappy.

Then somebody from the business asked whether we can ensure that results are correct when the whole batch job completes. We could. So no big deal, they said. We never look at this data in the middle of the job anyway. The call ended at that and we just looked at each other in disbelief.

That day I realized that business indeed copes with eventual consistency better than most programmers. I also learned that if we try to disguise our own technical biases as business requirements we might at best waste some time. If we’re less lucky we can lose a lot of money.

It might seem like an isolated example, but the world is full of similar stories.

Amazon decided they can deal with an occassional “absolutely unacceptable” situation that resulted in selling a single paper book to two different customers. Fixing that and making customers happy costs them some money, but they make much, much more thanks to the design that allows for that situation to happen.

Gojko Adzic in one of his books mentioned a betting company that was discussing key examples for their specifications (i.e. test cases). They realized that allowing customers to spend more money than they had is indeed a great idea. Even though it was an “absolutely unacceptable” scenario and would make every programmer cringe. Turns out that the customer that spent some money was more likely to come back and make another transaction. In order to do that they would have to pay off their debt first, so the company would get their money back eventually… But even if not, it’s better to make 5 dollars instead of 10 than make 0 instead of 10. Or at least that’s the logic business people apparently follow.

I try to keep those stories in mind every time I feel like saying that something is just unacceptable. Somebody else might already make money on proving me wrong.