Working with your Complexity-dar

Managing complexity is what we do as programmers, right?

Whether you’re architecting a new application from scratch, or deep into writing a component or class or test method, complexity is the elephant in the room, and our job is to ignore it, or at least keep it in the corner, right?

I mean, complexity in software development is inevitable. Beyond “Hello World”, building and maintaining systems that solve interesting problems is like being in a boat full of holes. Every new thought the client has is another leak and we’re the poor sods standing in the middle with a bucket, scooping up the complexity and flinging it back into the sea as fast as possible, right?

You can tell by now that I’m about to say “think again”, can’t you?

Listen to your complexity-dar

Your brain is an amazing thing. Tiny fluctuations in relative levels of substances such as dopamine alert you to changes in patterns that you’re not even aware of observing. The difference between a great fighter pilot and a terrible freeway driver is partly the level of attunement to those chemical changes. Fighter pilots don’t think they need to take evasive action, they feel it – and they act upon that feeling.

Cognitive dissonance is a feeling you’ll recognise even if you’ve never heard the term before. When your internal (or external) dialogue is saying one thing, and your subconscious is telling you something else, you feel uneasy.

Your particular form of unease might be different from mine. I tend to feel mentally heavy or hesitant. I go to type the code and something makes me pause. I can gather myself and force myself to type, but it feels different. I used to think I had to overcome that feeling, but now I know that it’s useful, and instead I pay attention to it.

It tells me that something doesn’t ‘fit’. And I think that’s a damn good definition of Complexity – the stuff that doesn’t ‘fit’ the patterns, and thus has to be carried in your mind in its own special compartment because you have to make allowances for the differences.

Humans are smart, we can manage complexity (but not in programming)

As @debreuil posted this week, human beings are smart – our problem-solving-monkey-brains can do interpersonal communication, combining language and gestures and facial expressions and body language to perform the incredibly complex task of achieving shared understanding with another human being. (Something I do less well than most people, at least when presented with neuro-typical people to communicate with).

Computer programming is actually for dumb people. All you have to do is figure out the rules and apply them consistently using your dog-brain. And that consistently part is the catch. At run-time, any human being is a sea of memory leaks…

… ok, so if the message carries a response ID then the server is expecting … I think I might have sausages for dinner … a response ID means the server is expecting a response … hey, I’m not sure my sister ever responded to that email I sent asking for half the money for Dad’s birthday present … the server is expecting a response, and we’ve got an ID, and – oh, is that response a String or a uint? Boy, sometimes I think this was simpler in as1 when everything was just soft typed… is it soft typed? or dynamic typed? I guess I should learn the difference one day. And the banjo. That would be awesome. … ok, it’s a String…

No matter how intelligent you are, you have to assume that most of your brain is occupied with a bunch of other human-being stuff, and there is only limited (and frequently interrupted) threading available for thinking about your code. Which is why patterns are so useful. And why complexity that sits outside of those patterns is a bad thing.

If you have 100 things and they all fit 3 patterns then you only have to carry those 3 patterns and the ability to categorise the things at mental-run-time. Whether you have 101 things, or 4 things, if one of them doesn’t fit the first 3 patterns then your cognitive load just went up by a big chunk. And your chance of a MRTE (mental-run-time error) just escalated even more, because coming up with the rules that sort the things into the patterns is a more robust process with multiple test subjects. This is the impact of complexity – every time you think you’ve got it down, you find another uncaptured corner case.

MRTE: A Mental Run Time Error – the programmer’s comprehension of the system was compromised. Uncaught, these are the most catastrophic errors in any programming language.

Complexity: avoid or embrace?

I see three very different types of complexity:

  • complexity that we can’t influence
  • complexity that can be avoided by changing course
  • complexity that can be leveraged to discover simplicity

Working with Complexity that can’t be influenced

Complexity that can’t be influenced (yet) should be boxed off. Always. Find it, mark it toxic and double-decouple-it with the Bridge or Facade pattern. Wait, you thought Bridge / Facade were only for working with legacy code? I apply Bridge or Facade to anything that I feel my application would only want to poke with a very long stick. Occasionally that includes a part of my own code.

Stick it behind an interface and have the concrete implementation use it only through composition. Imagine what it would be like if it wasn’t complex, and that’s your interface or interfaces – keeping it on a need-to-know basis. Dream your simple solution and use the internal code of whatever monstrous implementation you have to use to make sure that complexity stays in one place. Do not confuse one-place with one-class.

Even when you’re working with a monstrosity, keep the code that deals with it clean.

Code is like a public bathroom, once it starts to get dirty it goes straight to hell.

With the right interfaces, as far as every other actor in your software is concerned, this monster is a cute and cuddly teddy bear.

Avoid Complexity where you can

Could you avoid this complexity by taking a different course? Then do it. When you sense that you’ve invented a complex solution, ask yourself – specifically ask your ego – whether you’ve just created yourself 18 hours of interesting work in order to avoid 3 hours of grunt work. Nobody enjoys programming-101 but you will not lose hours chasing an extra static constant on a custom event. If you resent making classes then it’s time to sharpen your tools or crack open your Mavis Beacon. And check out my previous post on how a 200 day project actually takes a week to type.

And, sometimes, the right thing to do with complexity is to climb right through it until you find the simplicity underneath.

Leveraging complexity to find simplicity

This week one of my tasks was to add a new kind of puzzle to our learning-puzzles set.  I’ve named this puzzle a Combination Game.

A Combination Game is one in which you have to choose a valid combination from a set of options. It’s your mc-lunch.

In the specific game that I am building as the first example, you have to put together a team of skilled workers. A valid selection is one worker from each of the required professions, and none of the un-required professions.

A combination game where you have to assemble a team

Pick a winning combination

Our flash lessons are produced by designers and animators who have a basic grasp of using AS3. So I need to provide them with an API that they can use to hook up this kind of game with different content.

I like to design simple things like this by imagining that I’m using a Fluent Builder – in the end it might be that I do give the animators a Fluent Builder to use, but it’s a useful approach anyway because it encourages you to design by composition.

I try to use a plain text file or pencil and paper rather than my actual code environment, it reduces the temptation to start coding before you’ve done enough thinking. It’s somewhat the opposite of TDD-as-if-you-mean-it, which I also like a great deal.

Through this process, I recognised that I would need to define a game in terms of a combination of requirements for a valid game, which I thought of as ‘rule sets’:

    new CombinationGameBuilder()
        .requireOneOf ( someItems )
        .requireNoneOf ( someOtherItems )

        // more rules as required

        .build();

And after about 10 minutes I had come up with four different types of rule set:

  • one_or_more_of
  • only_one_of
  • any_or_none_of
  • none_of

Four rules is not too bad, and would certainly allow me to build the specific game I had in front of me. In fact it allowed me to do more than that; any_or_none_of and one_or_more_of are both beyond the scope of this game.

I had decided I was going to implement each rule set as a different class – probably with a base CombinationRuleSet that handled most things. Each different function on the builder would instantiate a different concrete type of rule set. At least that way I wouldn’t have conditionals in my base rule set.

But, as I went to write the code, I had that hesitation. I just had this instinct that something was missing. It’s possible that it’s obvious to you already what’s missing, but I didn’t have any specific ideas – I just felt that it was somehow not right. So I asked for some prompts:

In ‘pick a valid combination’ game, option sets can be oneOrMoreOf / onlyOneOf / anyOrNoneOf / noneOf. Can it be done w. less/is anything missing?
[@stray_and_ruby, Jan 12, 1:18pm GMT]

And then I ate some lunch and replied to some emails, and then suddenly the internets had worked their magic:

@stray_and_ruby I may be a bit late, but what about moreThanOneOf or moreThanXOf?
[@mark_star, Jan 12, 2:47pm GMT]

He was right – they were both sensible rules, so now I had six rules:

  • one_or_more_of
  • only_one_of
  • any_or_none_of
  • none_of
  • more_than_one_of
  • more_than_x_of

But more_than_x_of was annoying initially, because it required a different method signature. All of the rest just required an Array or Vector of the ‘things’ in that set; more_than_x_of required another parameter – the minimum number required.

Then I noticed that more_than_one_of is a special case of more_than_x_of, with the parameter value 1, so that made me kind of happy.

In fact, I realised that I could use this to group my rules into exactly and at_least. Suddenly I had just two rules with the same signature:

  • exactly_n_of   (requiredNumber:uint, options:Array)
  • at_least_n_of  (requiredNumber:uint, options:Array)

Nice!

And then I had this horrible niggly thought that if you’ve got an at_least, then maybe there’s a case for the opposite: not_more_than.  I nearly ignored it. My two rules felt neat and ‘good enough’. But anyway, now I had:

  • exactly_n_of   (requiredNumber:uint, options:Array)
  • at_least_n_of  (requiredNumber:uint, options:Array)
  • not_more_than_n_of (requiredNumber:uint, options:Array)

Ah, hmm… what about cases where you need at_least some number and not_more_than another number. Shoot.

  • exactly_n_of (requiredNumber:uint, options:Array)
  • at_least_n_of (requiredNumber:uint, options:Array)
  • not_more_than_n_of (requiredNumber:uint, options:Array)
  • at_least_and_not_more_than_of (atLeast:uint, notMoreThan:uint, options:Array)

How annoying, now my method signatures are different.

And then I had the moment of realisation – that some of you will have had much earlier – that there is only one rule:

  • at_least_and_not_more_than_of (atLeast:uint, notMoreThan:uint, options:Array)

That’s it. Every other rule is a specific (note, not special) case of that rule varying only by the values passed for atLeast and notMoreThan.

So, my builder does present sugar methods for noneOf and atLeastOneOf and so on, but they all just wrap the only required rule. There is only one CombinationRuleSet – with the 3 parameters – which has no conditionals and requires no subclassing.

Yes, it’s less code, but the win is that it’s less to carry in my head. And had I rejected the complexity as it started to unfold, with a “That’ll do, they can work within this” I would have been stuck with a more complex approach which would have taken longer and been more prone to bugs. Of course I TDD’d it but your tests only cover what you think to test.

I actually think this exercise would potentially be a nice teaching tool or even a kata.

So – that’s how I eat my complexity – with the help of the ever-generous programming community. How do you eat yours?

About the Author

I'm an actionscript programmer living and working in a tiny village in the Yorkshire Dales, UK. I used to be a TV reporter, but my inner (and often outer) geek won. I also write stuff. Most recently Head First 2D Geometry.

Visit Stray's Website

Share the post

Delicious It Digg this! Stumble this! Share on Reddit Share on Buzz Share on FriendFeed
  • http://twitter.com/wrobel221 Michał Wróblewski

    Awesome article. Great quotes I posted earlier on my twitter. I also try not to code at first, but it’s a bit harder to think about everything when there’s no specification or at least a FULL list of needed features. I’m happy you described that subconscious alert that I also feel.
    Many thanks for that post.
    Michal

    • http://www.xxcoder.net Stray

      Thanks Michał,

      It’s really great to hear that you experience the subconscious alert in a similar way!

      In terms of requirements vs complexity, sometimes I think there’s more simplicity in expanding the specific requirements to the more general situation, but it really does depend on your client / budget / schedule.

      It doesn’t always work out, but it’s always worth exploring on paper (I think), what your system would be like if you made it more flexible to try to anticipate things the end user might reasonably want.

      One of the interesting things is that complexity under the hood often also comes out as complexity for the end user. We had one particular client who had a ‘templated’ website, but every new page they wanted was a different ‘special’ template. We tried to encourage them to stick to the established patterns, in order to make their site more usable, but they didn’t care what we think, or what their users think.

      We ditched them as a client as soon as we could :)

      Thanks for commenting,

      Stray