Category Archives: Uncategorized

Code is the easy part

When I shared what I’ve learned about DDD from a “boring” domain¬†some people wanted to see the code. Somebody even said “Show me the code or it didn’t happen!” ūüôā

On the one hand I understand that for programmers code is the tool of work, this is how we solve problems. On the other I was dissapointed, because in my mind code wasn’t the most interesting aspect of that project and in the post I intentionaly focused on those other aspects – how even in trivial projects learning more about the domain matters, how that knowledge is deepened and distilled over time, how business and technology interact.

But some time later I realized I was the same. For years I had been waiting for a “really interesting project”, for my holy grail of complex domain. Of course, I worked on interesting projects, occasionally solved really complicated problems… but nothing came close to the holy domain of cargo shipping.

Just looking at those large ships and how much stuff they move from one place to another, imagine planning routes, tracking where things are, troubleshooting when things get lost…

I wished I could work on a project like this.

cargo shipping

However, at some point I realized that most software is probably the “boring line of business” kind. Valuable, with the potential to make somebody’s work life hard or easy, but generally not very exciting all the time.

Seems that part of the challenge is that as programmers we think about complexity in terms of number of if statements and nested conditions (cyclomatic complexity). After getting some experience we learn that complexity is also in coupling and scale. After a few failed projects you also learn about the complexity of having too many moving parts and getting too excited about new technologies ūüėČ

But if you pay close attention then you will eventually notice, that in the end technology is the easy part. I often noticed that technology was made¬†the hard part even though it wasn’t necessary, because of overengineering, implementing features that nobody used or premature optimizations. But the real complexity was elsewhere.

Take the “boring” domain of personal finance. There’re literally hundreds of books on personal finance, thousands of blogs, and thousands of experts. Each saying different things and claiming they know the best, all the other guys are wrong. They make up their own vocabulary. They argue and challenge the “common wisdom”. They rarely agree with each other on anything. It’s easy to get overwhelmed and lost.

money

But after reading a few books you start noticing common themes, you distill the knowledge. You pick things that make sense for you, for your lifestyle, that fit¬†your personal situation. At this stage you can start making decisions on what is important and should be explicitly modelled, and¬†what can be safely ignored. You decide how optimal your solution should be, what are the trade-offs you’re willing to accept.

Often you ask experts for clarifications, but you know what you’re after, more or less. You’re not just a passive listener¬†that waits for directions, you understand the assumptions and constraints, become an active participant in defining requirements and designing solutions.

In many projects the Domain is the hard part. If you solve a problem that has never been solved, or that hasn’t been solved in that specific way, then nobody is the expert in the tradidional sense of the word. You have to come up the “right” approach together with the domain experts, you need to collaborate. There are no “ready” requirements that you can just write down, it’s a very creative work.

Even more interesting is the situation when you have multiple experts or stakeholders, who can’t agree with each other on the priorities. Yet, you need an agreement in order to write any code. As many observed before me, most people don’t really know what they want, but they’re absolutely sure it’s not the thing you created for them. Or you might discover that the “official” reason for a project is not what you thought, and you¬†optimized the wrong criteria or got hurt by office politics. That’s the People complexity.

I could go on and on and on… but you get the idea. I agree that technology is important, and always will be. We need to learn our craft and be fluent with the tools. But that’s simply not enough. There’re other kinds of complexity – domain, people, and more.

If you want to create great software that matters you need to take into account all those kinds of complexity. To me this is what DDD is about. It’s a¬†tool for tackling the ultimate complexity in the heart of software. Technology is just one aspect, and often¬†the (relatively) easy one.

real complexity

If you’re interested in hearing more, that’s one of the things I want to cover in my upcoming talk at DDD Europe.

Business or technical decision – which one is that?

It’s been a long, long time since my colleague said that “if the business can’t give us better requirements, it’s their problem, I’ll just do what I think they had in mind”. But I get frustrated every day with systems that apparently¬†were created using that approach.

I think the essential problem is that we, programmers/designers/makers, very often don’t have¬†enough “business” knowledge around the system we’re building.

I’m not talking about having more user stories or adding more detail to them.

I’m talking about the fact that it’s impossible to build a great¬†product if you don’t understand the business around it.

It’s not just hard. In the long run¬†it’s simply impossible. Because sooner or later you’ll make something that will ruin your system, that’s inevitable. Maybe you’ll be lucky and not many people will be affected. But if you don’t have enough context, you’ll keep making¬†bad decisions over and over and over again. Interestingly enough, you might not even realize you make any decisions, because you make assumptions without noticing.

After a while your only hope will be lock-in and competition that is even worse, but who wants that?

If you don’t know how your target audience thinks, what they value, what mindset they have, then how can you make sensible technical trade-offs? Is speed the priority? Or rather ease of use? Or accuracy? Or pleasant design? If you do know that, then do you actually keep that in mind when you develop every new feature or fix every bug?

If you don’t know what’s the relative importance of the specific qualities of the system you’re building, then how can you judge if the design you selected is optimal? What criteria do you use to dismiss alternative approaches? Do you even bother to come up with a few alternatives?

Last but not least, how often and for how long do you think about business consequences of technical decisions you make?


On the one hand we brag¬†that technology is making the world a better place, on the other it seems we don’t fully realize how big impact it has on daily lives of many people when it doesn’t deliver on the promises it makes.

A few small examples from my personal experience as a user:

For a long time I looked for an app to learn German vocabulary. Finally, I found something that looked like what I wanted. The content was interesting, I had lessons to every video, it was self-paced and had revisions. I was thrilled and right after the trial I bought 1-year access… and soon afterwards I stopped using the app completely.

Turns out the “learning” part is very rigid and I have to work around it. The content is not ideal, so I have to learn words that I don’t even know in English after learning it for 10 years. That’s¬†on a newbie level (sic!). I have to misuse the app to dismiss the words I’m not interested in learning, because apparently nobody thought that ability to customize what I learn would be useful. Then on some days¬†suddenly I have (literally!) hunderds of words for revision, even though I didn’t¬†fall behind the schedule.

Talk about demotivating and overwhelming.

So my idea was to keep using the content (as it is¬†really interesting) and use another app with more sensible revision part for learning vocabulary (by manually entering vocabulary¬†I want to learn). Great idea, but I¬†thought¬†about doing it 2 months ago and still haven’t¬†even started. I’m scared of opening the app, because probably all my 1000+ words are due for revision.


As a language learner I want an app that doesn’t force me to learn useless vocabulary, doesn’t torture me with boring content and doesn’t push me to work at a too fast pace, so that I don’t end up abandoning it and feeling miserable about not learning anything (again!).


Another example is generating invoices from the freelancing portal. For whatever reason (maybe local law?) the invoices are generated on a weekly basis, but the Polish tax authority requires accounting everything on a monthly basis.

Of course, I can manually reverse-engineer all the costs, all the data is there, no problem. Only it takes time and costs extra money for the accounting services, not to mention the “fun” of doing currency conversions and corrections 5 times more times than necessary every month (as I have to make that conversion for every invoice I submit, plus correct when I transfer¬†money to¬†my account). On top of that, I can’t be sure how¬†the tax inspector will judge my creativity when they analyze my documents.


As a freelancer I don’t want to spend hours manually crafting¬†reports that could easily be generated automatically (or semi-automatically) from the data you already give me, in a format that my tax authority¬†demands, so that I can sleep well and not look for the first opportunity to stop using your portal as soon as a better opportunity arises.


Of course, those are just two tiny examples, there are many, many, many more. Once you start noticing such issues, it’s impossible to unsee them. And it’s all too easy to put myself¬†in the shoes of the people¬†that created those features and wonder if I’d¬†even think about scenarios that now drive me nuts.

It’s not about the need for better user stories or being a more responsible programmer.

It’s realizing that nowadays many businesses are shaped by technology. For better or worse.

It’s about considering our work not only in the nice, sterile, non-existing box of technical requirements, but also in the context of how it impacts business and users. It’s about considering the business impact of decisions disguised as purely technical.

Because¬†it might be more efficient to generate reports on a weekly basis and your task might be to optimize DB performance, but tax law specialists might tell you that’s problematic in some countries. If you ask them.

Because the acurracy of the vocabulary revision algorithm may be ideal and that might be one of your app’s selling points, but the expert teacher might tell you that encouraging regularity and persistence is more important than precision.¬†If you ask them.

At the end of the day, it all comes down to asking constantly How this technical decision will impact business and users? Who should decide if that impact is OK?

I know that many people think PM/BA/everybody-but-them should worry about such things. In my experience assinging such responsibility to a specific person or group is only useful when you¬†look for somebody to blame for yet another disaster.¬†If the “makers” don’t think about it, then they’ll make a lot of decisions that weren’t really their to make.

For better or worse.

Beware of proxy domain experts

Today I’ve watched Greg Young’s keynote¬†from DDD eXchange 2016. The talk is really awesome, though-provoking, turning everything upside down… I really recommend.

However, the point that caught my attention most was the answer to the question from the audience (at the very end). Greg mentioned that we need to be very careful with who we consider domain experts. He even said that on one project their end-users hated the app, because their “domain experts” actually had no idea about the work that end-users were doing.

I’ve experienced that mistake myself, in one of my summer jobs. For a couple months I worked in a call-centre for a mobile operator.

When you call somebody in the middle of the day to talk about their soon-expiring-contract with a mobile operator, what do you think is the most common answer?

I need to think about it/I can’t talk¬†right now, please call me at XYZ time.

Pretty obvious, right?

The only problem was that it was impossible to do, because the calls were assigned randomly to people and it was impossible to make sure the call was assigned at a specified time! You could specify the preferred time range, but more often than not it didn’t work as you’d expect.

The result?

People entered random information and date in the system, wrote everything down in their paper calendars and handled the whole case outside the system. Some time later another unlucky operator who got that call assigned had a very bad day (you won’t believe how angry people are when you have no idea what they told your colleage a few months earlier and what they asked them to do back then).

In the end the system was an expensive phone book, that pissed off a lot of people, both working there and those that got the calls. The work was hard, stressful and annoying enough in itself, and the software we had to use only made it much worse instead of helping.

That’s what happens when you don’t realize you’re dealing with proxy domain experts.

Master paradigms, not syntax

Every now and then a programming newbie asks this question: what language should I learn? That of course is then followed by a long discussion with lots of strong opinions and no consensus.

That newbie decides to learn one of the suggested languages and gets the first job. Then they ask: what X frameworks should I learn? (where X refers to their chosen primary technology). A very similar discussion starts…

Then around an intermediate level the typical programmer starts freaking out on the point of becoming obsolete. Somebody suggests to learn a new language every year.

And it goes on and on and on…

I’ve asked those questions myself, multiple times, various people. But at some point I’ve realized that they don’t make much sense, and the answers even less so.

I think what matters is really just two things: to grow our skills over time and to not lose passion among daily struggles.

For the latter reason, I’d say it’s reasonable to just learn whatever you like and find interesting. Sometimes we have a particularly boring or depressing (think uber-legacy) project¬†at work, or our coworkers drive us crazy. Having some pet project, reading an interesting book, watching a mind-bending presentation reminds us why we still keep doing it, despite all the challenges. I don’t think the importance of it can be stressed enough. In that case it doesn’t really matter if and how soon you’d be able to apply what you learnt at work. The purpose is different.

But there’s that other thing, more difficult, especially given that in most jobs we don’t have much¬†support in planning our career. Heck, most people in IT can’t¬†even believe there are people that keep programming for 30 years, we ask where do the old programmers¬†go, so why would you plan for staying that long?

While I think coming up with a 30-year learning plan is insane (or even a 5-year for that matter), it makes sense to consciously think what¬†paradigms and principles we’d like to learn. They don’t change that often, they get refined over time, but the underlying ideas are relevant for years and it’s quite easy to catch up with the newest developments. Mastering the syntax of the new tool is relatively easy, if you understand what problem it solves and how it works on a high-level. Of course, there’ll be quirks, gotchas and edge-cases, but those you learn by doing, very often on a project you’re paid for (let’s be honest, most problems don’t manifest in pet-projects).

To make the learning efficient it makes sense to use the technology that is restrictive and will guide our learning in terms of principles. Taking languages as an example, hybrid languages are not the best way¬†to learn functional programming. ¬†I’ve learned from experience that it also makes sense to invest money and learn from the authorities in the given subject. They spent a lot of time distilling their knowledge and often package it in a easy to digest way. If they’re good teachers they will guide you instead of just talking (so you still get all the joy of discovery ;)) and will cover most important principles, so you can continue learning on your own.

But to the point, for me the biggest jumps in skills as far was learning about clean code, testing, requirements, DDD, functional programming, messaging, actors and event-driven architectures. Even when I didn’t use the relevant tools at work right away, my way of thinking changed. My designs got better. I was able to come with a few completely different approaches to solving the same problem, and thus pick the better solution. I could foresee some challenges and prevent them.

To me the biggest benefits of learning about various paradigms is¬†having more options and making better decisions. Because picking the best tool for the job shouldn’t be considered on the syntax level, it’s a paradigm-level decision.

Mechanical sympathy – not as low-level as you think

Last week I had a pleasure to attend a 1 day¬†workshop on “Understanding Mechanical Sympathy” by Martin Thompson.

The special thing about this workshop, and Martin’s work in general is that he convinced me that performance optimizations are not black magic, assembly tricks. If you think otherwise, I encourage you to watch his¬†“95% of performance is about clean representative models”¬†presentation.

Besides, he’s a great teacher – the knowledge is distilled, complex things reduced to simple basics, explained in a straightforward way. Last but not least, even though the exercises and presentations are using¬†Java, the lessons can be applied to any other programming language. For the workshop the basic knowledge of Java was sufficient.

So what did I learn?

Clean code leads to good performance

I did learn that at the uni, I’ve seen that before, even in Martin’s presentations, but only now it finally clicked for me. Martin spent a few hours (literally!) explaining how CPUs work and how they evolved across a few models. Before the workshop I believed him that clean code results in a good performance, now I feel I also understand why.

It comes down to a few simple rules. For example, one¬†must realize that CPU is a mini-distributed system. Every level down the architecture diagram (so registry, L1 cache, L2, L3, etc.) is more expensive with regards to the communication in terms of latency. On the other hand , every level up can store less data. If your class doesn’t fit into that limited space,¬†then you might be wasting a few cycles for getting necessary data to the registers¬†when its needed. Also typically¬†processors have a higher number of ALUs than for example JMP units. They also have dedicated units for performing operations on matrices and vectors. All that comes down to the fact that CPUs are better in arithmetic than in evaluating logical statements. All¬†conditionals (loops, ifs, etc.)¬†are¬†expensive.

Another important point is that¬†CPUs try to optimize the execution of our code and make a few bets: they assume that things which¬†are close to each other will be used together, things which¬†have been recently used will be used again, and that memory access will follow some kind of a pattern. That improves the performance of a “typical” (or should I rather say¬†well-written?) code, but if you’re code doesn’t follow those rules¬†you’ll pay for it. To be fair, the¬†CPUs¬†don’t expect¬†anything wicked.¬†On a higher level the hardware-friendly code can be translated to a few basic rules of clean code: small classes, high cohesion, short functions, low cyclomatic complexity, keeping loops very short and simple, etc.

I’ve never heard anybody else making the¬†argument that clean code leads to good performance, and I think it’s a great shame. I regret now I didn’t use this argument in my previous job when advocating for improving code quality. And that I didn’t know how to measure it and show¬†its impact. A few things I want to learn now after the workshop.

Things can go wrong when you measure performance

This is one of the areas I’d like to learn more about. Martin mentioned a few things to consider when measuring performance. Micro-benchmarks are generally hard to get right, even harder is to measure things that actually matter.

He suggested focusing on a higher-level tests, kind of end-to-end ones, which use realistic data and realistic use cases. The last two are very important, because compilers, etc.¬†tend to be smart and will optimize our code (e.g. remove paths that will never get executed for our crappy data, even though they’ll be for realistic one).

Gil Tene talked more about such things in his “Priming Java for Speed at Market Open” presentation. Even if you don’t use Java it’s worth watching to get an idea what can be really happening when your code is executed, especially if you use JIT compilation.

Locks are evil and concurrency is hard

I knew it before, but I didn’t realize that communication between threads¬†using locks might be slower than over the network. Apart from examples, we’ve heard a few “tales from the trenches” which were both funny and scary at the same time. Realizing you got your basic data structure¬†wrong after over a year in production is not a place anybody would like to be one day.

Pro-tip for work: if somebody suggests using concurrency and claims it’s easy, then it’s time to¬†run! They’re dangerous.

I think it’s worth a deeper thought that one of the best known experts on performance optimizations and concurrency in Java is saying that. If LMAX¬†could do¬†with just a single thread for business logic, then 90%+ systems in this world could also probably manage.

Martin ends his “The Quest for Low-latency with Concurrent Java” with Albert Einstain’s quote:
“Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage, to move in the opposite direction.” I let you guess where concurrency fits in it.

It’s all about the basics… and great¬†teachers

We have lots of hypes in programming. Every now and then somebody writes that everything has changed and people get worried how they can ever keep up with the pace of change.

After Martin’s workshop I have one more example that hypes come and go. Performance seems to be an ever-green hype, every year we have more data and want to process it faster, or we have more users and want to give them even better experience. Luckily, it seems¬†that¬†principles don’t change that often. If you understand the underlying principles then you can learn the syntax of a new language, master 95% of API of a new framework, or understand impact of new hardware features very quickly.

At the same time a lot of the complexity we deal with in our daily work is unnecessary and generated by ourselves. Because we overcomplicate things and copy-and-paste without understanding what happens under the hoods, maybe¬†we know what and how to do something, but don’t really understand why.

To be honest for me hardware and all the “low-level stuff” was one of such things that I never could fully grasp. It just didn’t seem relevant or as important as other things I could spend time on. Martin convinced me otherwise.

I’m not sure if performance will become my great passion, but even if I stop at the level of “20% of knowledge that gives 80% results” that will be great. At least now I understand where to start and “hardware and all this low-level stuff” seems less scary than ever before.

Summary

If you ever have opportunity to attend Martin’s workshop, go! It doesn’t matter what language you use or how high is the level of abstraction you operate on – it all runs on the same hardware. Learning a little bit about it will make you a better developer.

To DDD or not to DDD – Is my model right?

  • “I’m new to DDD, please, help! Can you review my design?”
  • “Can you recommend some DDD code samples?”‘
  • “Why the’re not more DDD samples out there?”

Have you heard any of those before? Maybe it was you asking?

I know how it feels when you are trying to learn something that is really difficult (hello DDD!) and all you hear is “it depends” or “you’re asking a wrong question” (thanks for helping).

The challenge is that both of those answers are true. It’s impossible to give a good answer to any of those questions without having the full context. It’s easy to notice when something is obviously wrong (and you probably know that already anyway), but how do you decide if something is good? How do you know that there’s no better model out there? How do you even know your model is good enough? Is there something like “good enough”? Does DDD come in grades or is it all or nothing thing – either you do it right or not at all?

In that respect DDD is very much like investing.

Some time ago I attended a conference for real-estate investors (landlords). There were many interesting workshops, but my biggest lesson was lunch conversation.

I sat at a table with 3 investors.¬†The first one introduced herself and said she invested in large flats for students and young professionals (flat sharing). The other said it is a really stupid idea. It’s obvious students would ruin your flat sooner or later (he ignored the first investor claiming it never happened to her in the last 10 years). So he invested only in small condos and studios for young working couples. The third observed that small flats tend to have low ROI, so it’s better to buy bigger flats for families. Another advantage is that families¬†change rented flats less often and you don’t have to look for new tenants every few months.

And so it went… For two hours!

It was fascinating. Each investor was making money. Each was happy (and comfortable) with their investment strategy. Each thought that not only is their strategy THE BEST ONE, but that it’s THE ONLY reasonable option. And it was true. Their strategy was the best and only option FOR THEMSELVES.

If you’ve asked them what you should do with your $100 000, each would give you a different answer.¬†All of them would be wrong. Because if you can’t make this decision for yourself then the only thing you should invest in is education. Blindly following¬†other’s people orders won’t make you money¬†in the long term.

Don’t get me wrong. Inspiration is important. Learning from other people’s mistakes is invaluable. It’s OK to ask for advice and consider different viewpoints. The more various options you consider, the better! I’ve learned a lot just by listening to those investors.

But in the end there’s no single best investment strategy and there’s no “right” domain model. You can stop looking now.

The truth is that there are countless strategies and models available. Each has their advantages and disadvantages. Some might be completely wrong, but most would be quite good or good enough for the time being or maybe just satisfactory given the constraints we have. In the end it all depends on your context. And the very important (and mostly ignored) part of that context is… you.

You and everything you know about the business, company and people in there. Your context.

Somebody said that investments are not risky. Investors are risky. ¬†It matters how much you know, what are your past experiences, how well do you get along with your team and stakeholders, how well you deal with pressure, how much risk and overtime are you willing to accept before you talk about it with your PM…

Even if some DDD uber-expert comes to your team for a project and literally tells you what to type for a couple of months, after they leave you’re on your own. Back to square one. Or even worse, because he probably didn’t realize that the “business expert” he talked to is an ignorant know-it-all who has no idea what he’s talking about. He really should’ve talked to that shy, quiet guy in the darkest corner of the open-space. Everybody goes to him when something goes wrong or when they encounter a new edge case.

So don’t fret and start where you are. You already have everything you need at the moment. The rest you can learn by doing and analyzing results. By all means, ask for advice but don’t mistake it with outsourcing thinking.

After doing it 10, 50, 100 times you’d be surprised how much progress you’ve made.

Trust me. Good investors know that there always will be another great opportunity. Good modelers know that there will always be some other good models they could come up with. Don’t get stuck at looking for “perfect solution”. It doesn’t exist.

The better you become, the better will be your results.

How to lose a client with a simple form

Banks in Poland are fully embracing the power of technology. You¬†can open a new bank account, take a loan or do¬†lots of other things without leaving home. You don’t need to send¬†a photocopy¬†of your ID or passport, so your personal data is safe. All you need to confirm your identity is an existing bank account.

The system is based on trust. One bank trusts the other that they checked your documents when you opened the first account. So all you have to do now is to fill in the form online, then make a small payment (~30 cents) to a given account. That transfer is used for confirmation of your identity and then money is returned. The bank checks personal details, including your address, date of birth, etc.

That’s really brilliant. I’ve used this system a few times before¬†and it worked like a charm.

But not this time.

I come from a small village. Streets in my lovely village don’t have names. My¬†village doesn’t have its¬†own¬†post code. That’s common in Poland. The¬†national post has closed many branches a few years ago. We use post code of a nearby town, just like other villages in the area.

Depending on the design of a particular form the name of my village becomes “Street”, “City” or “Name of the place”. The name of the town with the postcode is either “City” or “Post”.

About ~40% Polish people live in villages, so I’m not alone.¬†Yet, it seems like many¬†IT specialists¬†have no idea about all of this. Maybe they all live in¬†cities.

Found on Flickr

About a week ago I decided to exchange currency using an online platform. I filled the form to open the account. But my identity can’t be confirmed.

Even though I provided the same values as for my other account, they¬†are in different fields. Because of how the forms were designed I couldn’t possibly fill them in the same way.

I understand it, help-desk people¬†understand it, but system doesn’t. Apparently, humans¬†can’t override the automatic match. Probably that’s some¬†super-duper-smarter-than-human¬†security policy. Oh, did I mention both services (online banking¬†and currency exchange)¬†are provided¬†by¬†the same¬†company?

I’ve been trying to untangle this mess for the last week and lost hope. I switched to their competitors.

All their wonderful support people couldn’t make up for the¬†wrong model. For a stupid address.¬†I wonder how much money they’ve lost by this.

How is address represented in your model? Would this model work for me?

Choosing the right tool is not enough

As a programmer I’ve been taught that picking the right tool for the job is¬†THE THING. I’m thinking¬†here about the kind of decisions that¬†are¬†based on combination of extensive knowledge (not only tools, but also paradigms behind them and their limitations) and lots of thinking (most likely inspired by¬†solving¬†challenging, real-world¬†problems).

But looking at projects I participated in I realized that picking the tool is only one tiny step in a very long journey.

Here are a few challenges that I observed as far.

The right tool won’t help you if you use it in a¬†wrong way

I have very little experience with NoSQL databases as far, but a few things surprised me already. In SQL we usually start modelling by thinking what data we have and what would be the best way to store it. But in NoSQL databases we need to start by thinking how we want to access this data and optimize structure for that goal. What is considered a dubious practice in SQL world might be perfectly acceptable in NoSQL one (e.g. data duplication is very common).

If we don’t know how we’re going to use the¬†data then I don’t think it’s possible to come up with a good model. You can’t just make it up as you go, because you’ll end up trying to build SQL-like model for querying on top of NoSQL database. That means you’ll have great performance problems. For example, document databases work best if you access documents by ids and if each document is an independent aggregate (meaning it contains all the data you need at the moment). If you create lots of tiny documents and top it up with¬†many¬†indexes, with each index touching the majority of the documents to check a¬†single¬†field on them…¬†Well, probably it won’t work¬†well in the long run.

I’ve came across similar¬†situation a few times. Usually the problem was that the person who picked the tool understood it really well, but wasn’t involved in implementation or was involved in a limited capacity. Think typical architects, typical “enterprisy” software development process, or just not enough code reviews.

take_a_horse_they_said

Image from http://www.omniref.com/blog/blog/page/2/

The right tool won’t help you if your problem changes

… and you don’t¬†notice that the once perfect tool doesn’t fit the problem any more. Or worse, you notice but can’t switch the tool because of other factors (like cost or time limits).

That change might be the result of new requirements, growing understanding of the system, or using the system in a new context, e.g. when you find access to a new type of clients.

That situation can lead¬†to the previous point. It’s hard to decide when the line is crossed and it’s time time to stop abusing¬†our perfect tool to fit¬†the¬†new problem. It’s even harder to do it when we have limited resources and are expected to keep delivering.

The right tool won’t help you if¬†you solve a¬†problem you don’t have

I’ve seen two¬†types of situations when people solved problems they didn’t have. One is the infamous premature optimization. It’s very hard to resist the temptation to use your skills to the maximum and just aim for the simplest thing that works. It’s also very hard to accept¬†that you can’t predict the future requirements, and what seems like a great idea today might turn out to be a waste of time tomorrow.

The other situation is equally popular communication issues resulting in misunderstanding what the actual problem is.

Image from https://medium.com/interactive-mind/the-evolution-of-ux-f27dd1ac56b4

Image from https://medium.com/interactive-mind/the-evolution-of-ux-f27dd1ac56b4

The right tool won’t help you if¬†you solve a¬†problem¬†that is not worth solving

The great example would be adding lots of features to the system, knowing well that only a very tiny percentage of users will use them (or maybe none at all). It’s estimated that 30% of features in our systems are not used or rarely used, so don’t think your system is an exception.

Working on problems that are not worth solving¬†not only generates maintenance costs in the future, but also is a lost opportunity. We could’ve be working on something else instead.

Start at the end

Level 0: Commit message

Once upon a time somebody told me the commit message should be written before any code. Of course, I’ve ignored the rule many times since then. But every time¬†I follow it I’m amazed that such a simple thing actually helps with focus. Plus it makes me super guilty if I decide to batch multiple unrelated things in a single commit.

Tiny habit, but results add up quickly and actually make a difference.

Level 1: ATDD, BDD, Specification by example, etc.

Some time later I noticed that even though we, developers, can be very good at writing code the right way, quite often we work on the wrong things. We make¬†wrong assumptions that break everything in the last moment, we keep making business decisions disguised as technical ones, or we simply don’t care about solving the business problem as much as about playing with all the cool technology.

I was fascinated by the stories where asking a few extra questions saved hundreds of hours of work, made existing model completely unsuitable or led to a complete change in approach.

But after putting the ideas into practice I lost some of the early enthusiasm. Those methods work and they certainly brought a lot of benefit to a few projects I worked on. Even though the actual acronyms were not always used, we found a few serious problems or misunderstandings early in the process with those methods. So I know they work.

The issue I have with them though, is that there are so many tools and processes around them now that it’s hard to not get distracted. It’s so¬†tempting to play with a new framework or argue which tool is better instead of talking to the human being on the other side of our program. And there’s a lot¬†one can play with.

Level 2: Sell it before you build it

At my current job we have a¬†process that tries to force us to focus on users and business aspects before we even think about writing any code. We make so called “impact analysis” to determine what is the expected benefit of building a specific feature. Sometimes in the process we realize that in fact there are better things we could be working on at the moment and the idea is dropped.¬†Then we prepare announcement and documentation drafts to think how will we communicate this feature to the users. If we can’t explain it or it doesn’t sound very attractive¬†then it’s probably not worth building.

To be honest it’s not easy to work that way. We’re developers, not marketers after all. It really takes some effort. It’s easier to just code or cheat the process.

The obvious benefit of that approach is not wasting time for building things nobody would use. But there’s another one. A few times by going back to the announcement we realized we had made mistakes. Last time it happened to me yesterday. Another time we came up with¬†a few extra things to discuss, noticed holes in the initial approach to solving problem¬†or identified new¬†edge-case scenarios.

The main benefit of starting with customer communication¬†over BDD is that it doesn’t involve any fancy technology. There are not as¬†many powerful¬†distractions.

Next level: ???

I’m not sure what the next level is, I guess that it¬†would have something to do with metrics and verifying assumptions we made using numbers. But it might be something completely different.

I hope to find out soon.

That’s just unacceptable

A¬†few years ago I had a big challenge at work. With two or three other programmers we spent almost 8 hours browsing through code, drawing diagrams, swearing and thinking to the point our heads started to hurt.¬†The conclusion was clear – we can’t do it. It’s impossible.

A few months before that day our batch job started processing too much data to fit a single processing window. Not a big deal. Business adapted. They started the job a few days early each month, to make sure all reports were ready before the deadline.

But now they came up with a new product. One requirement didn’t fit our model. From then¬†on information that used to be always in one piece, could be spread across multiple records. What’s worse, related records didn’t have to be¬†located next to each other and could arrive in ¬†a random order.

We started analyzing edge cases. What if the processing window ends before all related records are processed? We’ll have incorrect results. We KNEW that was unacceptable.¬†So we were thinking and thinking and thinking…

Eventually we realized that ensuring correctness at the end of each processing window is either impossible or extremely expensive. So on the very same day we set up the meeting and broke the bad news to the business. We felt uneasy about admitting¬†that our system can’t accomodate their business idea. We were sure they’ll be extremely angry and unhappy.

Then somebody from the business asked whether we can ensure that results are correct when the whole batch job completes. We could. So no big deal, they said. We never look at this data in the middle of the job anyway. The call ended at that and we just looked at each other in disbelief.

That day I realized that business indeed copes with eventual consistency better than most programmers. I also learned that if we try to disguise our own technical biases as business requirements we might at best waste some time. If we’re less lucky we can lose a lot of money.

It might seem like an isolated example, but the world is full of similar stories.

Amazon decided they can deal with an occassional “absolutely unacceptable” situation¬†that resulted¬†in selling a single paper book to two different customers. Fixing that and making customers happy costs them some money, but they make much, much more thanks to the design that allows for that situation to happen.

Gojko Adzic in one of his books mentioned a betting company that was discussing key examples for their specifications (i.e. test cases). They realized that allowing customers to spend more money than they had is indeed a great idea. Even though it was an¬†“absolutely unacceptable” scenario and would make every programmer cringe. Turns out that the customer that¬†spent some money was more likely to come back and make another transaction. In order to do that they would have to pay off their debt first, so the company would¬†get their money back eventually… But even if not, it’s better to make 5 dollars instead of 10 than make 0 instead of 10. Or at least that’s the logic business people apparently follow.

I try to keep those stories in mind every time I feel like saying that something is just unacceptable. Somebody else might already make money on proving me wrong.