IF YOU BUILD IT

... Will they Come?

November 2007

Jeff Atwood (of Coding Horror) writes about Brian Foote and Joseph Yoder's Big Ball of Mud paper, but completely misses the ball (pardon the pun). Jeff says that the paper describes "classic architectural mistakes in software development" while the paper describes the exact opposite.

With the exception of the Big Ball of Mud pattern itself which can be seen as an anti-pattern (as, by the way, the authors explain) the rest contain exceptionally good advice on how to prevent problems. Let's look at them one by one:

• Throwaway Code. When you just want to make sure something works (prototype, spike, etc.) or even a quick fix, you don't write elaborate and heavy code; instead you write something simple to solve the problem. Throwaway code only becomes a problem if you don't actually throw it away....
• Piecemeal Growth. The waterfallish "big plan" in advance has failed numerous times. So instead we should build incrementally, don't over plan or over design. Build things into libraries only when it is proven to be recurrent problem, refactor, etc.
• Keep it working. This pattern is basically about building often and keeping iterations short, having tests that always prove your code still works as you make changes to it.
• Shearing Layers . This pattern is about identifying related components. We all know change is inevitable. However things don't change at the same rate. For instance, we can probably get interfaces to be a little more stable than the implementations behind them. Frameworks evolve at a different rate than the business they support etc. The pattern is about dividing the system so that things that change at similar rates are together i.e., in the same package, CSCI or whatever you use to partition your system.
• Sweeping It Under the Rug. When you find that you do have badly designed or badly implemented code -- the pattern suggest you localize and isolate it to a fixed area (e.g., behind a facade) to prevent it from propagating into the rest of the system.
• Reconstruction. This pattern is about understanding when the code base is so bad that it is better to start over and rebuild from scratch rather then try to patch it. In many ways, reconstruction is not a good or easy thing to do however the point here is to identify when it is the lesser evil.

Oh, and what about Big Ball of Mud itself? Essentially from the architectural perspective it is indeed an anti-pattern -- you have something that is not very maintainable, hard to understand, and the like. However keep in mind that the idea behind designing an architecture is not to get the best, cleanest architecture. The idea is to make the right tradeoffs so that you'd be able to deliver the best overall solution under the constraints you face (budget, time, the team's skill and what not). If you're biggest constraint is time-to-market and your architect spends all eternity planning the 8th wonder of the world, fire his butt. I'd rather live with a Big Ball of Mud for the first release than not ever make a release....

Big Ball of Mud can be considered a pattern for pragmatic approach to building working software. This is probably not acceptable in the long term, but it can be a good option for short term if you are aware that that's what you are doing and willing to treat what you get as "Throwaway code".

So again, except maybe big ball of mud, these patterns are not "project pathologies" as Jeff calls them -- these are very good ways to keep delivering business value and working software.

Posted by Arnon Rotem-Gal-Oz at 01:35 PM  Permalink |

A few weeks ago I posted a reaction to a post by Pete Lacey that asked what is SOA. In a comment to my post, Pete said that my definition isn't good since:

...even according to your definition, an architectural style contains constraints, and to date neither SOA nor web services have been shown to exhibit any constraints

The idea behind this series of posts is to try to take a little more formal view at what I think SOA is. It is based on my thinking for the past few weeks but it is also still a work in progress (so any comments are welcome).

The way I see it SOA is an architectural style which is derived from the following architectural styles:

1. Client/Server
2. Layered System
3. Pipe and Filters
4. Distributed Agents

Note that if you add to the above statelessness, uniformed pipe and filters and a cache you can get a RESTful SOA. This is not REST as REST itself does not require distributed agent or even pipes and filters (but it does build on client/server and layered system). In other words not all RESTful systems are SOA, you can build SOAs which are not RESTful and you can build RESTful SOAs.

The main components of SOA are Service, Message, Contracts, and Consumers. Policies also exists but now I tend to think they are optional. The four architectural styles mentioned above affect the definitions of the different components and the way they interact together.

In the following posts on this subject I'll first take a look at each of the contributing architectural styles and how they affect SOA and later try to provide a definition that builds on them.

Posted by Arnon Rotem-Gal-Oz at 12:52 PM  Permalink |

While everybody and their sister is busy celebrating the release of VS2008, a more interesting* release happened -- the first community release of Ruby.NET (version 0.9). This is another step in the languages trend I discussed here a few weeks ago.

The release is said to have a lot of improvements yet to come, however. For instance, Ruby.NET isn't yet running Rails. Hopefully we'd have that soon. Another thing I would love to have is to be able to use Ruby's testing frameworks like Mocha to test .NET classes (which already works for Java and JRuby). Well I am off to test that now :) -- you can do that too, if you download the bits.

By the way, you may also want to read the paper discussing the design of Ruby.NET (by Wayne Kelly and John Gough who started this project).

Posted by Arnon Rotem-Gal-Oz at 06:09 PM  Permalink |

In the post Ignorance vs. Negligence, Ayende blows some steam off in regards to so-called "professionals" he's met along the way. You know -- those people with fancy titles who don't know jack and design some of the nightmares we see from time to time (go read his post, I'll wait -- I promise). I see this all the time:

• The senior security expert who recommended something which isn't supported by the platform.
• The senior architect who throws the system to hell by basing all the system on a clunky asynchronous solutions that should only be used by a tiny portion of the application.
• The geniuses who built this wonderful code generator that generated code with so many dependencies and singletons that made the solution unusable.
• The chief architect who created this wonderful performance hog, then kept poking around to make sure we don't fix it too much.
• The architect who partitioned a distributed solution based on functions -- so that each and every business process has visit go through all the tiers and components. The solution made the everything more complicated by few orders of magnitude (scale, synchronization, availability, performance what not).
• The architect who designed his own distributed transaction mechanism (basically duplicating COM+) ---naturally with less than satisfactory results...
• Etc.

Ayende says:

They all have a few things in common, they represent themselves as experts, senior, knowledgeable people. In all those cases, they have actively acted to harm the business they were working for, by action, inaction or misaction

I have no issue with people not knowing any better, but I do expect people that ought to know better to... actually do know better.

I don't think that this is negligence involved here -- I think all of these people want to do the right thing, they probably believe they are right. They were probably also pretty good at their jobs which is what got them to their current position.

What they didn't learn is to "know that they don't know". This is a hard lesson to learn. I think hope I learned my lesson after the first time I tried to distribute a (naive) solution I was so proud of. At least, in my case, I stayed around for enough time to both see the results and learn how to fix the problem.

Not staying around for enough time is one of the problems that causes this ignorance -- since starting out things usually look good enough and if by the time the problem rears its ugly head you are already in a new shiny job, they you don't know better.

Another problem that causes ignorance is not looking around and learning only from your experience. For instance, I am now interviewing a lot of people, and when I ask a question like "tell me about something interesting you recently read -- a book, an article, a blog uanything" -- I usually get blank stares. Few people tell me about an article they read that is related to a problem they had, and fewer yet tell me about something without a direct relation to their work. If you don't look beyond the keyboard you will never know better. Learning only from your mistakes can be problematic -- especially if we also consider the previous point (people don't stay around).

Ignorance is bliss they say, but ignorance has a lot to do with the crappy systems we see all around us and its one of the reasons writing software stays more of an art than a science or craft.

Posted by Arnon Rotem-Gal-Oz at 07:19 PM  Permalink |

Jeremy Miller writes that he prefers user stories over use cases--and I basically agree with him. Mainly because user stories are more fine-grained, which leads to making them more "estimateable" and manageable than use cases.

However, having used both, there's one thing use cases do better than user stories -- eliciting related scenarios. When you write a use case that covers a business scenario, the use case template makes you think about variations and exceptions. If you're not careful to proceed in a structured manner when you try to think of a theme of user stories, you might miss some.

One important point regarding use cases is that you don't have to treat the use case as a single monolithic block. When you work with use cases it doesn't have to mean that you sign-off use cases in a Waterfall-ish manner -- that is, something that has to do with your development methodology.

You can develop use cases incrementally; that is, only do some of the use cases per "release". You can also elaborate use cases iteratively by elaborating/identifying some of the scenarios in each iteration (in a manner similar to user stories). I have
successfully used both incremental and iterative use case elaborations on several projects and I find this approach useful. I even wrote a ( paper summarizing my use cases experience a few years ago. (Warning! More than 60 pages.)

In fact, even today I find it useful to use both tools when I would usually start with identifying a user story and writing that down. Then map it to a use case (either an existing or new one) -- where a use case roughly equals a user story theme. As I mentioned previously, I can them identify more user stories and add them to the product backlog. The use case stays as a skeleton with links to the user stories and helps provide context to individual use stories. (Okay, so that isn't a use case in the traditional sense, but I find it useful anyway.)

Generally speaking, it is important to note that while it is tempting to equate a user story with a scenario in a use case, that isn't always correct. Sometimes a user story would be a step in a use case, and sometimes a story is shared by several use cases -- since a fine-grained feature that delivers business value can be used in many business processes, which is what use cases are oriented to.

As an aside, if you have a project where you do have to write all the requirements up-front (like they won't change anyway), then I find that use cases are superior to IEEE 830 style requirements ("The system shall..."). But hey, that's another story.

Posted by Arnon Rotem-Gal-Oz at 04:05 PM  Permalink |

Back in April I wrote how Adobe AIR (then called "Apollo") marks the beginning of the invasion of Web clients onto the desktop.

Later I wrote about the Java and .NET counterattacks (JavaFX and Silverlight) and then I wrote about Google's answer when Google Gears was announced.

Now Mozilla's Prism shows that even simple steps can help make this transition.

The main idea behind Prism is to "integrate web applications into the user desktop experience". Behind this fancy statement we have a simple solution -- the ability to add a a desktop/start/quicklaunch shortcut to any web application (or page for that matter)and have that show in a window that is configurable so that it doesn't waste pixels on irrelevant stuff for the applications (like navigation buttons, address bar etc.). This makes it better then just adding a shortcut yourself. Simple and elegant. Here's what my Google reader looks like with Prism:

If you want to start using it, you can just download the prototype for Mac OS X, Linux, and Windows.

Posted by Arnon Rotem-Gal-Oz at 08:15 AM  Permalink |

Let's assume I convinced you that some projects need architects (see Part I). Convinced, you go and hire an architect. Now what?

Let's start by looking at "architectural decisions" -- which is sure sounds like something we'd want an architect to do. I read once (I think that was something Martin Fowler wrote) that an architectural decision is a decision that in hindsight you wished you made right. if we look at a formal definition of software architecture (say from IEEE 1471) we see that the architecture embodies the fundamental decisions about the system its components, their relations and their properties. Using this definition an architectural decision is a fundamental decision about the system (which pretty much explain why we want to make them right etc.)

Well, here are two observations on what I've said thus far.

• One is that we would want to postpone architectural decision as much as we can, since changing them will cause us a lot of headache. The problem is that in order to postpone an architectural decision we need to build flexibility into the system which is an architectural quality in itself -- which might not be the top of the list if we prioritize it vs. other architectural qualities we need.

• The second observation is that if we "refactor" the pretty language out from both of these definition -- we can see that an architectural decision is basically a guess, hopefully that's an educated guess but it is a guess nonetheless. And as Albert Einstein once said it is hard to make predictions -- especially about the future.
  

This is why architect's breadth of knowledge -- which helps explain the architect training program I posted about a few weeks ago (see Architect training program Part I and Architect training program Part II).

Another aspect is experience. And to get a wider perspective it can be helpful if this experience includes other roles besides developer such as project manager or business analyst etc. Another important component is domain knowledge and understanding of the business.

Using all these you (as an architect) may come up with a reasonable architectural decision (e.g. use MVC pattern) and a design to match it and that's it.

Well, actually, not quite since as I said earlier it is still a guess. Remember an architectural decision (and any design for that matter) is a mirage no matter how beautiful the power point slide looks (or white board or UML sketch etc.).

Alas, power point compilers are still in the making. Which means that as an architect, you must be able to prove your point in writing -- that is coding. While you are at it, you also need to know a thing or two about the technology you are using because it too has an architecture, features etc. which can have a significant effect on the end result. (You can read a little bit more on this in the paper I published a while ago.)

The result of trying to postpone architectural decisions, ever changing requirements along with adding details as we unfold the architectural abstraction level to a working system, is that the architect can't just appear at the inception of a project and disappear afterwards -- they need to stick around for the game. This is especially true if you want to have an evolving architecture.

An architect needs to do more than "architectural decisions". There are also additional reasons why the architect should have continuous interaction with the rest of the development team. However that will
have to wait for Part III. :)

Posted by Arnon Rotem-Gal-Oz at 05:58 PM  Permalink |