If we want to decouple a SOA system, we must get away from the notion of the remote procedure call.  In other words, our services need to have as few “command” messages as we can get away with.  This is a design philosophy but it is easier said than done.

According to Hohpe and Wolfe, there are three basic message patterns.  Excerpt from their classic work Enterprise Integration Patterns:

Message intent — Messages are ultimately just bundles of data, but the sender can have different intentions for what it expects the receiver to do with the message. It can send a Command Message, specifying a function or method on the receiver that the sender wishes to invoke. The sender is telling the receiver what code to run. It can send a Document Message, enabling the sender to transmit one of its data structures to the receiver. The sender is passing the data to the receiver, but not specifying what the receiver should necessarily do with it. Or it can send an Event Message, notifying the receiver of a change in the sender. The sender is not telling the receiver how to react, just providing notification.

If you look carefully, it isn’t hard to see that the command message is sent with the understanding that something will happen on the receiving end.  More importantly, the sending KNOWS what will happen on the receiver’s end.  This is a particularly insidious form of coupling.  It is also really simple.

All kinds of things creep into a SOA message as a result of this knowledge.  If I am asking a service to “Create an Invoice” and I send data that describes an invoice to another system, I am making a lot of assumptions. 

1. I am assuming that the receiver will succeed.  The receiver has to be present. I have not said “I have an invoice for you.”  I have said “I need you to do this for me.”  If the receiver isn’t present, then what?  Sure, I can use durable messaging, but if a message is stuck in a queue, it isn’t in ANY part of the distributed system.  It vanishes from existence until it turns up at the other end.  The invoice isn’t created… it doesn’t exist in any form… for an indefinite period of time.  NOT GOOD.
    
2. I am assuming that the sender should, ultimately, have the right to decide if an invoice should be created.  That’s odd.  Why do I even need the receiver to create the invoice?  Answer: because the sender cannot.  Implication: the sender does not have the “right” to create an invoice.  The receiver is the “system of record,” not the sender.
    
   But if someone wants to add a new validation rule, that says we won’t create invoices for customers in Germany because of a new import law that went into effect, where does that restriction go?  If we put it into the system of record, (the receiver), then we have to put a rule into the sender as well, to allow it to return an error message or handle the refusal.  In effect, the sender still has intimate knowledge of the workings of the receiver. 
    
3. I am assuming that the invoice isn’t already there and the sender is the first system to notice!  This flies in the face of reality.  It is entirely normal for the CRM system, the billing system, the shipping system, and perhaps even the portal sytem to be part of a “new invoice” process.  If I make the statement that “It is I, Sender of the Magic Message, Master of Invoice Creation, who has the right to demand an invoice into existence!” then what… people have to call the sender to create an invoice?  What if they don’t!  What if they call the system of record and I discover the sale later.  Will I mistakenly generate another invoice?  Or will I have to put complex rules into my code to insure that I only generate some of the invoices that I am aware of, but not others because I know that other systems have ordered the creation of the invoice.  Unmaintainable.  
    
4. I assume that creating the invoice should happen RIGHT NOW for both my system and the receiver.  That may be convenient for me, but not for the receiver.  In fact, it may be wildly unreliable for the receiver to receive messages as they come in the door.  Or perhaps some messages happen right away but others take too long.  This places an artifical constraint on the system of record: do what you want, as long as it doesn’t take more than 100 milliseconds.  This is seriously tight coupling. 

Each of these assumptions exist in a Remote Procedure Call.  They are forms of coupling, pure and simple.  They fly in the face of SOA.

So what to do?  How do you avoid making SOA endpoints that are commands?  If I want to offer the ability to create an invoice to the enterprise, what should my endpoint look like?

You have two choices: event driven and document driven

Event driven looks like this:

First of all, there is an event that you need to subscribe to.  It is not the event of “invoice created” because the sender is not allowed to create invoices.  Therefore, we need the system of record to subscribe to a different event… but what event?

What event occurs in a business that says “create an invoice.”  How about “we made a sale?”  Think of the subtle difference.  An invoice is a document.  We use it to track the sale.  We assign a number to it and we look up other sales etc.  But it isn’t the BUSINESS event.  The business didn’t make an invoice.  The business made a sale.  Operations people made an invoice to track the sale (long before computers came along).  One tidbit: the fact that the system of record can reject the transaction means that this is an “unapproved sale.”

Notice Steps 2 and 3.  The event message usually doesn’t contain sufficient information for  a system of record to fulfill it’s responsibilities.  It subscribed to the event, and therefore discovered it, but it needs to call back to the source system to get the actual data to act upon. 

Notice Step 4 above.  There are two subscriptions back to the sending system.  This handles the case where the sale wasn’t allowed.  It is the system of record that denies the sale.  

There is an interesting bit of coupling still going on here.  The source of our sale came from a system that is not aware of the rules surrounding a sale.  It has a simple task: collect data and start a sale transaction.  However, we had to subscribe to two kinds of events, didn’t we?  We had to subscribe to both “Invoice” and “Sale Denied”.  That means that we had to tell that source system that there were two possible status values for a sale, and we had to subscribe to each. What if a third comes along?  What if the business wants to change the rules to allow a new kind of sale… one that doesn’t generate an invoice OR a sale-denied message?  We’d have to change BOTH systems.

Both systems are coupled on the business process itself… the business process isn’t in the diagram but it definitely affects the design.  So how do you decouple from that?  Let’s look at Document based messaging.

First thing to notice: the number of ports and channels is a LOT simpler and we don’t spend nearly the same amount of time “chatting” about things.  However, in this model, the responsibilities of both systems are VERY different.  This is an architectural design change.  This kind of change CAN be added to a system later, but it is more expensive than if you add it up front.

In this model, we don’t send events at all.  Notice that.  We send documents and the documents have a transaction id that is carried from point to point.  As the document goes from describing an unapproved sale to describing an invoice (and later to a shipment), you carry one transaction id along the way.  This is your correlation identifier.  This allows each of the systems to perform activities based on their own business processes, without needing to know anything about the business process implied in the other system.

Notice that we are down to one response subscription, and it isn’t even a specific subscription.  It basically says “For any transaction that started with me, or that I touched, please send me back any documents related to it so I can update my status.”  It is very simple.

The simplicity is a bit deceiving.  If we need to trigger another event on the transaction source, we need to put in some logic for that, but it is not a substantial change over the event-driven approach.  The logic is simply contained in the app instead of the messaging system.

Conclusion

So which method is better?  Should we use commands, events, or documents? 

I’m a big fan of simplicity.  And the simplest method, in the end, is document driven.  Unfortunately, due to the architectural changes needed to make it happen, we often cannot start there. 

So, for a migration plan, take legacy systems and make them event driven.  If you are building new systems: make them document driven.  Either way… kill the command message!  Avoid that mess.