I’ve been thinking a lot lately about the gap between “what we have” and “what we need” in the Enterprise SOA space. I think I have a need that is not yet filled by software. (that I’m aware of).
I put up a post back in June about the difficulty in creating a common information model in a large enterprise, especially one with a federated environment like ours. (CISR coordination and diversification models). The feedback I got back was telling. The majority of respondants told me what I suspected: developing an enterprise-wide common information model was so difficult that many folks considered it unfeasable. (Actual words included things like “utopian” and “big design up front.” I’ll stick to “unfeasable”).
That said, I have also stated in public that I believe, firmly, that SOA at the enterprise level requires some levels of agreement on the way that information is understood and coordinated. Each business unit can own information that is specific to that unit, but in the areas of coordination, where there is value, the business needs to be able to communicate.
So, we have something that is hard to do (build a CIM and keep it up to date). That something is useful (to get the benefits of Enterprise SOA). So, why not take the software approach? After all, I do work at Microsoft.
What business scenario would this tool need to support?
Basically: each business unit would submit their information model to a common repository. The submitters are architects, and they MUST align the models in some way, even if it is just to show that there are differences. Services are posted to the same repository, and must be lined up under specific information models. In order to consume a service, the business has to agree to the information model. Multiple competing models are allowed to co-exist. What do we get? An economic model of self-governance that produces an optimum information model: one where agreement is reached only where agreement makes financial sense.
- A business unit architect can consume part of an information model without consuming the entire thing.
- A business unit architect can assert part of an information model without proposing the entire thing.
- A business unit architect can offer services tied to their information model.
- A business unit architect can assert that a portion of an information model is “standard” for their use
- A developer, writing software for that business unit, can easily find and adopt their information model.
- A project team can provide a report to a governance committee proving that they are conformant to local standards
- An enterprise architect can run reports to isolate “points of difference” between business unit information models.
- A system designer, intent upon consuming services from another business unit, can begin a workflow to insure that the consuming business unit agrees to the information model of the presenting business unit. (an “information adoption workflow”).
- The workflow needed for a consuming business unit to agree can be custom-tailored to the organization.
- Organizations that present a service have an automatic measurement mechanism built in allowing them to “charge back” the businesses that consume the service. Various financial models must be supported (one time fee, pay per use, annual licenses). This provides economic incentive for sharing of code, as well as the incentive to create services that align to commonly needed information models.
- Built in support for the versioning of information models over time (including both past and future versions) allows the business to change their minds, and even chart their course.
That’s what my gut tells me. This has some pretty interesting effects:
- Information architects have a clearly defined and critical role at the earliest stages of a project: get consensus on information model changes needed to allow the consumption of existing, lower cost services.
- Economics will drive good behavior. No need for an Enterprise Architect to “design” good behavior.
- Less emotion. There will not be consensus on everything, and this model doesn’t require it. Consensus will be reached surprisingly easily on some key areas, and it will happen without any architect looking to make it happen. This helps remove politics from the picture as well.
- It is easier to adopt existing code than build new code if services, offered by other groups, aligned with the information standards of the consuming business, are already clearly identifiied.
We may be closer than we think. With bits from various MS products, and with Oslo coming, this vision is getting closer to reality. It’s an end-to-end idea.
Something to consider. Comments?
4 thoughts on “Enterprise SOA needs a Federated Evolutionary Modeling Environment”
Good post… but I’m wondering if you’re overlooking the obvious. By the way, I’ve put a post up that got a few comments about how "challenging it would be to achieve [what I suggested] in a large organization." (http://bit.ly/YtX2c)
My point is… large organizations already have solved this problem. Whether HIPAA or FIX or SWIFT, each of this is a multi-organization data standard. Each achieved without dumping stuff in a registry.
Let’s look at what the differences are:
1. The organizations I mentioned are specifically motivated to deliver a common data model.
2. There is a clear business driver (more rapid communication that can be more easily understood between organizations).
I think the effort should be made to motivate properly and address the business drivers, rather than on technology (registry) adoption.
By the way, I admit, these data standards only address the data model, and not how it’s used between organizations, which is something you mention above as well, that I’m not addressing.
There are three levels of abstraction here, and for each, there is a solution.
For B-to-B: use information models like the standards you mention.
Pros: useful for interactions between organizations
Cons: not good for interactions that take place inside a company.
For standardized integration: some attempts at messaging standards have been made… especially with respect to process models.
Pros: simpler integration (at least, in theory)
Cons: proliferation of incompatible standards reduces the value of these dramatically.
For enterprise implementable integration: this suggestion plays here.
Pros: tailored to the unique value prop of the business
Cons: manpower. Even with the suggested tool, this isn’t free. If you don’t have a tool, it costs five times as much.
I was discussing the detailed level… where rubber meets the road in large enterprises. For the other two levels of abstraction, standards are a great addition (albeit too weak out of the box).
1) Although we can implement SOA without CIM (or canonical data model), we really need it for EDA. So your post has sense certainly…
2) What you model CIM (or CDM) in? I do not see any problem with splitting model to parts and sharing them with existing tools (if you have CASE with shared repository, most of your requirements can be implemented easily). I do not see any problem with functions as workflow (it can be implemented as add-on to CASE if needed).
3) But I see the problem with "adopting" local standards. What I tried to tell in my blog entry (http://it.toolbox.com/blogs/system-integration-theory/canonical-data-model-using-industry-standard-data-models-26635) was, that existing schema languages are not ready for "adoption" of industry standard model. Support for evolutionary modeling (from industry standard models to local standard models and from local standard models to business unit models) would be great… You work in Microsoft, you could solve it somehow 🙂
(I assume this is Peter Rajsky)
Yes, I work in MSFT. It’s an interesting problem. I don’t know if I’ll get the chance to solve it, but if I do, then you’ll know where I first documented the idea.