I find myself surprised. I guess I had assumed that everyone has already read books like Enterprise Integration Patterns by Hohpe and Woolf and Patterns of Enterprise Application Architecture by Fowler. I guess I was being optimistic.
SOA in an architecture for Enterprise Application Integration. If integration isn’t part of the goal, SOA is sound and fury signifying nothing. (my apologies to the bard).
In SOA, there is a very specific principle that intermediation gives us: substitutability.
A little Context: the Factory Pattern. When an interesting article was published in DDJ a number of years ago titled “Use of Java New considered harmful” it raised a good bit of noise. After all, how could it be ‘bad’ to use the ‘new’ keyword to create an object. The author correctly pointed out that creating an object binds the code in the module to the type itself. It is impossible, if you use the language keyword, to create a subtype based on configuration if you use ‘new’. This was part of the general move towards patterns, a move that brought us dependency injection and the Spring framework, to name a few. The value of these innovations has been tremendous. In systems where they are used, dependency injection patterns lead to tremendous gains in productivity in the code.
The principle that supported this: Liskov Substitution Principle.
In order to make this change, factory patterns became commonplace. While I do NOT recommend that every object be created with a factory, I do recommend that any object that is to be considered reusable or extendable be created with a factory. Otherwise, reuse or extensibility is a fiction (a chimera, as one of my respondants mentioned on a prior post). Over the years, we changed the practice of OO development. “Decouple fiercely” is a good way to describe the change. You decouple the code every time unless you can justify NOT doing it. Big change.
At the architectural level, however, we didn’t look at the substitutability principle in an OO manner. Perhaps that is because many of the principles of OO, including inheritance, are quite a bit harder to grapple with when you are dealing with components. Hiding a component is not the most positive thing to do when you have to deploy it anyway. 🙂
Instead, we took the same notion: “decouple fiercely,” and we created the concept of services. They are decoupled… fiercely decoupled. We use one protocol to communicate. We can intermediate anything. We have one mechanism to monitor and manage. This led to the growth of ESB patterns and entire product lines that move information using messaging. It is a sea change.
But no good deed will go unpunished, and no good idea will be remembered if it is not justified every now and then… so here goes.
The ability to add intermediation gives me some fairly specific advantages. Just like factories give me the benefits of the Liskov Substitution Principle in OO design, intermediation gives me the benefits of substitution in messaging design.
Intermediation is the ability to observe a message passing from one trading partner to another and to take action based on the contents of that message (assuming I have been given proper access to it).
Those actions include
- changing the destination to another system,
- changing the destination to two systems (useful when transitioning from one destination to another),
- allowing events to be published from multiple sources,
- adding event handling and orchestration as the result of the business event,
- repurposing or repackaging the event for exposure on a different protocol or externally to a public interface like SaaS would require.
These are the tactical changes. Consider the actual abilities this introduces. For example, #4: adding orchestration.
Let’s say that system 1 generates an invoice. It sends an event to the world saying “invoice here” and system 2 captures that message. System 2 asks for details about the invoice… perhaps it will place the information on a web site for internal support teams.
Let’s say that we are moving to a CRM solution in our internal support groups. We want to create the information in the CRM system related to the invoices that specific customers have been issued. We need to integrate these two systems. The existing web app needs to have a link to the CRM system’s data, to allow the user to move across easily.
We can intercept the request for further information from the web app to the publisher. When the publisher responds with information about the invoice, we can insert the invoice in the CRM system, add a link to the CRM record for that invoice to the data structure, and resume our response to the web app. Assuming that our canonical schema has a field for ‘foreign key’, we have just integrated our CRM and web information portal… without changing either one.
Clearly, we’d still want to add a front end bit of functionality to allow the link to be transversed, but the data integration is done… without either the publisher or the subscriber having to change.
This is just one example of intermediation in practice. I’m sure that MANY folks can present other examples.
I would argue that the title of this post should be the benefits of intermediation. You can’t arrive at the value of intermediation if you don’t take into the consideration the costs (which you have ignored here).
I’ll note a few of the costs I’ve seen. Requiring intermediation encourages messages to carry more contextual information. Because the sender can’t make any assumptions about the receiver, the messages need to explicit and self-contained That doesn’t sound too bad, but, given that you often don’t know anything about receiving systems (they may not even exist yet), it is easy for messages to become bloated.
Intermediation greatly complicates any message exchange pattern other than request-response and pub-sub. In those situations, it’s usually best to have a protocol for other systems that intervene. Imagine dancers doing the tango. If you’re going to cut in, it’s considered polite to ask permission.
Finally, intermediation requires the intermediating system to understand at least some of the semantics of the messages it intercepts. Otherwise, it risks breaking a functioning system. Unfortunately, those semantics are never contained completely within the messages themselves.
I want to be clear. I’m not arguing that intermediation is a bad thing. I don’t think anyone would claim that intermediation doesn’t have the manifold benefits that you articulate here, but that is not the same as saying that intermediation is required for SOA. For that, you would need to explain why an architecture MUST always allow intermediation to be considered service-oriented. That’s the step I’m not willing to take with you.
I think you can apply LSP for SOA – esp. when you think about contract versioning:
When changing the internal behavior of a service, you don’t need to create a new version of the contract if for each operation defined in the contract the preconditions are the same or weaker and the postconditions (i.e. the outcome of the request) are the same or stronger or in other words the to retain the same contract version, the new version of the service should meet the expectations that consumers of the service have come to expect from the old version’s observable behavior
Hello John,
I find your response interesting. You raised three issues, and I will blog a seperate article to respond to them.
Needless to say, you have not convinced me, but it is worth the discussion to carry this out.
— Nick
@John
I would argue that self-contained messages should be one of the foundations of a service-oriented architecture. There will always be edge cases (ie.. the 10 MB message), which will have to be treated differently, but as a rule of thumb. Self-contained messages are the antithesis of chatty, distributed object protocols. I will give you the fact that bloat is easy to promote, but if you design each message with a specific business intent, it becomes easier to hold the line.
As for this particular post, I agree with Nick’s points. The two words that come to mind are "location independence". Services should not care who they are talking to (within the parameters of trust and security).
There are two types of intermediations. One is explicit where the sender sets the path along which the message is to travel and be subject to intermediaries.
The other is implicit where neither the sender nor the receiver is aware of any intermediaries. The intermediaries are placed for use case specific policy enforcement/monitoring.
I would also argue that there is no such thing as a self contained message. This is the reason we have a protocol. A well defined protocol can limit the number of exchanges between the sender and receiver. About the only thing a message needs to contain (beside the actual data) is source/destination and protocol in use.