IF YOU BUILD IT

... Will they Come?

April 2007

This is the fourth installment on the topic of architects and project managers. In this part, I examine "how do you balance and divide the responsibilities between the project manager and the software architect?"

One observation I have is that we are talking about roles, not necessarily people. For instance, in a single project there might be a person with a project manager title who also performs architect roles and vice versa. This is also true for architects vs. developers. For example, on a small enough project, one of the developers can also perform the architect's roles.

On the surface the division seems clear. The project manager is responsible for the schedule and budget and the architect for the technical integrity of the solution. This is looks fine -- until you consider that every technical decision has budget and schedule implications. So now what?

When the project has dedicated architect, I would say that the responsibility of the architect is to provide the the technical and technological aspects for major decisions. But it is up to the project manager to make the decisions. The architect should try to get the best possible solution under the constraints the project manager sets.

What happens when you have a single architect that works with multiple projects?

The way I see it there are two scenarios.

• One is that the architect is sort of like a consultant for multiple project. In this case (which, by the way, I would try to avoid), the architect should be treated like a consultant. In this case, I wouldn't even let the external architect make the final decisions within the constrains set by the project manager. The team that builds the project, which hopefully also includes a dedicated architect, should do that. I guess this type of architect is what is known as "Ivory tower architect."

• The other scenario for an architect who works with multiple projects is to have a product line architect.
  In this case, the architect needs to consider constrains and requirements beyond the scope of any single project. However, for this to be successful, this kind of architect should also come with the budget, schedule, and resources to make these decisions happen. If an organization is serious about a product line, it should finance that extra work which doesn't bring any direct benefit to the project where the initial work takes place. If this precondition is satisfied it is viable for the architect to override the project's manager constraints to get wider scope requirements prioritized. You can think of the architect in this scenario as a technical product owner (in the SCRUM meaning of product owner.
  

There can be variations on this (e.g. a large project where the architect is in charge of sub-projects) but the idea is the same the architect should either be part of the team -- working for the project manager, a consultant that you can opt listening to (but don't have to), or a "technical manager". That's someone who can override the project manger. But to do that, the architect should be backed up with the resources (or at least budget/schedule) to make this change.

What do you think?

Posted by Arnon Rotem-Gal-Oz at 04:28 PM  Permalink |

Another short post (I hope to be up and well next week.)

Back in January, I participated in an architect panel that Microsoft Israel organized. The panel was led by Ron Jacobs who turned it into a podcast in his Arcast series.

The panel's focus was on lessons learned from past mistakes. Ego maniac as I may be :) -- even though you don't get to hear me much in the final edited version -- I think the podcast is worth listening to, as the panel raised some interesting points. You can download the podcast here.

(The podcast is in English. Also, you might recognize Udi Dahan who also participated in the panel. Lastly, I am the first speaker after the introduction, in case you are wondering.)

Posted by Arnon Rotem-Gal-Oz at 12:30 PM  Permalink |

I am feeling a little under the weather so I'll keep this one brief.

I mentioned Architecture & Design World 2007 a little while ago, but I just found out the conference's web site and registration are now open. I'd be very happy to meet with any of you who happen to read this blog and attend the conference. Once there, I hope you'll be able to find some time among the many great presentation there and attend my 90-minute class on SOA Patterns.

See you there :)

Posted by Arnon Rotem-Gal-Oz at 01:31 PM  Permalink |

I've seen the following question on one of the forums I follow:

I have studied up on the SOA approach and it all sounds good.But most articles stop at the theory.

Let's say I sell things. I have a CustomerProfileService. The application does CRUD through this service to a back end database. Its autonomous and isolated. I have anther service, InventoryItemProfileService. Again, the application does CRUD through this service to a back end database. It is autonomous from the CustomerProfileService. Not only may it live on a different DB from the CustomerProfileService, it might exist on a different platform.

Now let's get to the InvoiceService. From the client side, I would guess that I would have a CreateInvoice(custID,itemID[] ) method. The InvoiceService would then call out to the CustomerProfileService for a profile that meets the needs of the invoice, then another call out to the InventoryItemProfileService for the item descriptions and such.

So here is the question: It would seem like in the back end (the db) of the InvoiceService there would be tables to support the customer info and the item info from the invoice. Where prior to SOA, when everything was in the same db, these requirements would be largely satisfied by joins. Now a logical join across services just seems radically expensive (everytime you touch the invoice). Hence the need for the customer and item tables local to the invoice service.

Does this sound right? Just how often does the InvoiceService have to go back to these other supporting services?

I also got a comment with a similar theme on my Cross Service Transactions post.

I see a few problems with the way the services in the question are modeled (like CRUDy interface) but in the end it all boils down to the root cause -- and the real problem: granularity of the services.

Sure when "a service" is too small it doesn't make sense to separate its tables from those of other services. it doesn't make sense to have transactions that span only what's internal to the service. It doesn't make sense to pay the price to make a service autonomous (like caching reference data from other services). When the granularity is too small you will often find that you need to make a loot of interactions with other so called services. you are more likely to have CRUDy interfaces.

You are also more likely to have slow performing solution and suffer fromlow-availability. Using services in a granularity mentioned above is, in my opinion, a nightmare that would probably make you work very hard to maintain the SOA principles in place -- or the more likely option, that you would circumvent the principles so that you can get something maintainable, usable and performing (and flip the bozo bit on this all SOA thing)

So what is the right granularity. Well, it is not a one-size-fits-all kind of thing, but as a rule of thumb I would say anything just shy of a sub-system and up. A service has to have enough meat so that it would make sense having it autonomous; that the transactions would fit nicely inside its boundaries; that it would be worthwile making it highly available; that you can pass a complete task/document to it; and it won't have to talk to a gazillion other services to complete processing it, etc.

If your application's idea of invoices is a two tables, one with a header and one with invoice details, then don't make that a service. If invoicing is a sub-system with complex business rules, a lot of options, and what-not, then it can be a good candidate

Think about it next time you design a service :)

Posted by Arnon Rotem-Gal-Oz at 06:10 PM  Permalink |

I received a few interesting comments following my post "Transactions Between Services? No, No, No! ". So here are a few clarifications.

The post was triggered by an article that promoted flowing transactions (i.e., you perform a transaction against one or two services and then one of the services calls an additional service and it joins the transaction). It is important to say that I think transactions between services should be discouraged regardless of automating extension of transactions. Transaction flowing just makes the matters worse.

There might still be some edge case where you have to have an atomic transaction from a service consumer to the service. I think that in the vast majority of SOA implementations you shouldn't do that and I would think real hard about the other options before allowing it in my architecture.

One of the comments I received began with the following:

"Cross-service transactions are a sure way to introduce coupling and performance problems into your SOA." I'm not sure I agree with that thought. Logically speaking, cross service transactions are a must. The question is how to implement them. There are two mechanisms we can use for implementing TXs: (1) ACID TXs; (2) Long-running TXs. The latter is preferable for the cases Arnon is talking about (large geographical distances, multiple trust authorities, and distinct execution environments). ACID TXs are more suitable for what Guy has mentioned (DeleteCustomer service invokes the DeleteCustomerOrder service internally). I agree with Arnon the a-synchronicity is preferable, but we all have encountered use-cases where ACID-ness is required from a business requirement level... [snipped]

One minor point in regard to this comment is that I don't like the term "long running transaction". There is a long running interactions between services and I think the term "SAGA" describes them better. SAGAs are made of a series of business activities that flow back and forth between services to realize a larger business process. Note that these interactions doesn't necessarily have transaction-like behavior.

Which brings me to the more important point of looking at the statement "Logically speaking, cross service transactions are a must." I don't think so. Consider, for instance, if a service that manages the inventory in a warehouse receives a request for some items and later a cancelation of that request. The first request can trigger the inventory service to order some more items from a supplier. Whether the cancellation would cause a cancellation of the order of the supplier depends on the business rules of the inventory service for inventory levels for the items ordered. it might also depend on whether or not the items have already been received etc. The cancellation (the "abort") of the original request does not have to translate to an abort (or compensation) on the request receiver. Furthermore if the service communications model is based on the push model (e.g., using EDA with SOA) the cancellation notice would just be propagated without regard to the inventory service. It is the inventory service's responsibility to understand the ramifications of this event and act accordingly. Even the example given in the comment "DeleteCustomer service invokes the DeleteCustomerOrder service internally" is not a good candidate from ACID transactions (there's also a problem of service granularity here; I'll talk about it later). Since when the customer service decides to delete a comment and request the Orders service to delete orders, there's a reasonable chance that some of the orders are already paid for but not delivered. In this case the customer cannot really delete the customer until all the paid orders are resolved. Or maybe the order service is a facade to a night batch that does the actual deletion. I know I am just fantasizing with these examples but the point is that the customer service has no knowledge on the order service or the inventory service above except the messages supported in their contract. To assume something about the internal behavior is problematic. Even if you know about the internal structure on the onset, the whole idea of SOA is that the services can evolve independently from each other.

Another thought triggered by the example in the comment originated by the granularity of the services (DeleteCustomer service vs. a Customer Service that also supports deleting customers) is that we should be really conscious to the difference between other architectures like 3-tier client/server and SOA. SOA is actually more distributed than 3-tier. We cross a distribution boundary every time we pass a message from a service to a service and not just when we move a massage from a client-tier to an application server. We add this distribution to gain advantages in flexibility and agility. However, we should note that this is a weakness of SOA (considering, for example, that Martin Fowler's first law of distributed object design is "Don't distribute your objects!") and we should really pay attention to the way services interact with each other.

• The granularity of services. Having a lot of fine grained services means there will be a lot of interactions over the wire (even if you don't go out to the network you still have to serialize/deserialize, follow the security policy etc.) rather than internal interactions that much faster.
• The Granularity of messages. The same considerations should also guide us to try to create larger and fewer messages. for the example above . Instead of a DeleteCustomerOrder message maybe something like an UpdateCustomersOrders message that can hold a list of customers and orders and the status changes or,by the way this would also support off-line clients better since they can accumulate changes.
• The assumptions we can make on the other service's availability, performance, internal structure, the trust we have for it etc. We should try to minimize the assumptions we make and concentrate on what can be inferred from the contract. Remember that policies can change externally so the business logic within a service cannot count on them being constant. this brings us back to the issue of transaction. every cross-wire interaction increases the chances of failure -- in transactions one failure invalidates all the transaction is invalidate. every cross-wire interaction within a transaction increases the length of time we lock internal resources (even if we do trust all the involved parties) -- especially if that transaction can extend itself automatically. Also as I've mentioned in the previous post the transactions also open the door for denial of service attacks.

If we want to reap the benefits that are sold under the SOA moniker, like flexibility and agility, we really have to pay attention to this extra distribution and design our services differently than we would omponents in a 3-tier architecture -- but hey, that's why they pay us the big bucks, right ? :)

I should probably also say here that building SOAs is not a goal in itself. We can build perfectly good solutions using other architectures, but if we find that we do need SOA (or any other architecture for that matter) we have to pay attention to the way we implement it to both keep its benefits and not harm other quality attributes like performance, security etc..

One last point (and, I guess, a word from our sponsor :) ) regarding the transactionless solution of eBay. I've got another comment that pointed out the it is most likely that eBay just don't use distributed transactions across their databases but do use local transactions. This sounds reasonable and I also believe that , though it isn't completely clear from looking at the presentation's slides. I hope I'll be able to find out the reply in July when I'll hear Dan Pritchett's presentation in Architecture & Design World 2007 :)

Posted by Arnon Rotem-Gal-Oz at 08:23 AM  Permalink |

I've just read Juval Lowy's article entitled "WCF Transaction Propagation" in the May 2007 issue of MSDN magazine.

Juval opens the article by saying:

Transactions are the key to building robust, high-quality service-oriented applications. Windows^® Communication Foundation provides simple, declarative transaction support for service developers, enabling you to configure parameters such as transaction flow and voting, all outside the scope of your service. In addition, Windows Communication Foundation allows client applications to create transactions and to propagate them across service boundaries. [Emphasis added by me]

So you've spent all that time toiling and sweating to create loosley coupled and autonomous services and there comes a propagated transaction and invalidates all your efforts. I don't think that WCF transaction propagation is necessarily a bad thing in itself. I am sure it has its uses -- but not in the context of SOAs.

Consider, for example, what Don Box said back in 2004:

Boundaries are explicit. A service-oriented application often consists of services that are spread over large geographical distances, multiple trust authorities, and distinct execution environments. The cost of traversing these various boundaries is nontrivial in terms of complexity and performance. Service-oriented designs acknowledge these costs by putting a premium on boundary crossings. Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation. Compared to distributed objects, the service-oriented model views cross-service method invocation as a private implementation technique, not as a primitive construct—the fact that a given interaction may be implemented as a method call is a private implementation detail that is not visible outside the service boundary.

Do you really want to flow transactions between "services that are spread over large geographical distances, multiple trust authorities, and distinct execution environments"? I know I don't. Things get even worse when we consider that service consumers can be from external parties? Do you want to lock resources within your organization on behalf of a request coming from a someone else? As a competitor, I've just found a way to introduce denial of service to all your systems -- I'll just open a transactional call and never close it.

Cross service transactions are a sure way to introduce coupling and performance problems into your SOA.

By the way, using distributed transactions is not something you should use lightly even if your architecture is not SOA. As Martin Fowler said (commenting on Ebay's architecture which is transactionless):

The rationale for not using transactions was that they harm performance at the sort of scale that eBay deals with. This effect is exacerbated by the fact that eBay heavily partitions its data into many, many physical databases. As a result using transactions would mean using distributed transactions, which is a common thing to be wary of.

I guess that making it easy to flow transactions between WCF web services is another perfect example for what I meant when I said "Microsoft has too often given us enough rope to hang ourselves by providing ways for the inexperienced to easily build solutions that will make them suffer down the road".

Finally, note that transactions can be put to a good use within SOA -- only if you use them inside the services as a way to handle a message in a reliable manner (a pattern I call "transactional service").

Posted by Arnon Rotem-Gal-Oz at 05:12 AM  Permalink |

Back in January I opined that moving to web applications was not the optimal solution to the real problem we have/had with desktop applications. The real problem is installation woes. What we ended up with is a poor UI without installation problems, so we (the software industry) started to resolve problems like we had when we moved from terminals to graphical UIs etc. So now we have Rich Internet Applications (RIA) using technologies like AJAX. But they suffer from other problems which again we've already been through.

Well that was the topic of the post in January. Now I've stumbled upon an interesting/amusing twist -- Adobe Apollo.

Apollo let's you -- yes you've guessed it -- take your RIA applications and deploy them as desktop applications. You can now take your HTML, CSSs , AJAX scripts pack them up as a single file (AIR) and lo and behold deploy them on the desktop. You even get these nifty start menu and desktop shortcuts :).

The reason not to dismiss this as a complete waste of time is that what we actually see here is another example of a trend to convergence web and desktop UI architectures and programming models. I say "another" because coming from the desktop direction Microsoft is doing pretty much the same thing. WPF brings the web-programming model with its markup (XAML) and "code-behind" concepts to the desktop as well as pushing the same model to the browser with WPF/E. You can find more on WPF in Properties, Dependency Properties, WPF, Sizing Up .NET 3.0's WPF, and Windows Presentation Foundation Interoperability.

The difference between Microsoft's and Adobe's solutions is that Adobe is coming from the web-side and Microsoft from the desktop side. Both companies are striding toward the same goal, and what we are left with is yet another technology war.

Posted by Arnon Rotem-Gal-Oz at 08:22 AM  Permalink |

In the previous post on this subject, I looked at what an architect can do to avoid crossing the line between steering the project into the right technical direction and "destroying the developers' jot of creation and professionalism".

As Osiris commented, this only looked at the problem from the architect's perspective. This post looks at the situation from the PM perspective and answers the second question raised in Part 1 :

Assuming this PM is right, what's the best way to defuse the situation?

Granted, this question is a little bit off-topic for this blog (Architecture and Design) -- but not that far. In my experience as a manager, the best ways to deal with issues is to talk them out and the sooner the better. Wait too long and the architect and team will build enough antagonism to render future cooperation very problematic. Remember that developers have enough reasons as it is to circumvent the architecture, and as a PM (assuming the architect is good) it should be a priority for you that the architect be heard and the architecture that emerges (by the architect and the team -- but that's a topic for another post) will be followed. The best way to get there is cooperation.

To talk this out, I prefer to first try to approach alone the person I'm having conflict with. However, it sometimes might be better to have an impartial facilitator (mutual boss?). The interests of the architects and the program managers should be the success of the project (even if it the reasons behind it is justpersonal glorification). Try to focus on that and avoid making the conversation personal. I would probably start with asking the architects if everything is okay and do they see any problems with the team. If they don't, then try to present the problems you are seeing. And don't forget to listen and try to understand the interests and not just the positions.

Possible solutions in the particular conflict we are talking about is to try to steer the architect to a mentoring, rather than a command-and-control, position. This has several benefits. It will get the team and the architect to work closely and get better working relations. Assuming the architect is good, it can help make the developers better. Also it will help make the architects\ feel better about the team and give them more slack.

In any event, try to figure out your Best Alternative To a Negotiated Agreement (BANTA) and try to think if you can live with it. I personally think that good working relations between architect-PM-team are crucial to a success of a projectbut if things gets worse than your BANTA go with that instead.

Two recommended books on negotiations areGetting to Yes and Getting past No. In the next part I'll try to look at the the third question, which to me seems like the most interesting

How do you balance and divide the responsibilities between the project manager and the software architect?

Posted by Arnon Rotem-Gal-Oz at 05:23 PM  Permalink |

I've been buried under Chapter 4 of my book for the last week or so -- you know, deadlines and stuff . Anyway, during this time I missed the onset of an interesting discussion among Andres, Udi, Jesse, Ayende, and Frans.

It all started when Andres posted that in Orcas there is no disconnected mode for Entity framework, Microsoft's version of O/R mappings. You can read the post, hopefully in the right order, at Andres, Udi, Andres, Jesse, Ayende Udi, Frans, Ayende).

I guess that coming late into the discussion there's not as much that hasn't been said by one of the parties, so I'll just mention that I share most of Udi's views on the matter and the way I see it Microsoft has too often given us enough rope to hang ourselves by providing ways for the inexperienced to easily build solutions that will make them suffer down the road (e.g. result in unsalable solutions, performance problems, etc.) so its good to see one occasion where it will be harder.

Anyway, what I do want to mention is a related architectural dilemma which is: When/if should Entities travel from the business layer(s) to the UI. The way I see it, this is the root cause for the above mentioned question of whether we need disconnected entities.

As I did with the previous dilemma (O/R mapping), this will be a multi-part discussion -- and I'll start by introducing the context.

Over the years we've seen a lot of architectures for distributed computing, starting with 2-tiers client/server with emphasis on the server side, 2-tiers with emphasis on the client side, 3-tiers, n-tiers, and SOA to name a few.

On the client side we have a range of client styles that can be used with the different architectures (not all styles are compatible with all server architectures) from Connected Systems (web and smart clients), applications that can tolerate communication failures which I'll call "Occasionally Disconnected Systems" and "Occasionally Connected Systems," which are meant to work off-line as part of their normal operations.

Moving Entities to the UI makes sense in some of the combinations, but doesn't make a lot of sense in some of the others -- which is what I'll start discussing in the next post on this series.

Posted by Arnon Rotem-Gal-Oz at 05:58 PM  Permalink |