SOA in the Unification Model

By |2007-10-27T15:16:00+00:00October 27th, 2007|Enterprise Architecture|

This is third in a series on the impact of the business operating model on Service Oriented Architecture.  (see overview)

What can you get from this series?

My prior post raised a bit of ire with one of my readers, a fellow whom I respect.  He felt that my posts were not telling a positive story about SOA. 

I believe that SOA is a highly valuable paradigm for enterprise integration.  I also believe that the ability to apply SOA to a problem is not uniform.  Some problems can be solved with SOA.  Others cannot.  There is no magic bullet. 

I am hopeful that this series helps folks to

  1. identify which situations they are in,
  2. understand the challenges they will face when applying SOA to their situation, and
  3. improve their odds of successfully providing value to their business.

I am not attempting to sell SOA to anyone.  I make no money from “Selling SOA.”  I have no personal or professional stake in the success of the SOA paradigm.  That said, I believe strongly that SOA has a value, and if we do a good job of avoiding mistakes, we can demonstrate that value to our respective businesses.

The Unification Operating Model

“When organizational units are tightly integrated around a standardized set of processes, companies benefit from a Unification model.  Companies applying this model find little benefit in business unit autonomy.  They maximize efficiencies and customer services by presenting integrated data and driving variability out of business processes.” (Enterprise Architecture As Strategy, by Ross, Weill, and Robertson)

I illustrate this model as follows.  (I described how to read these images in a prior post).

operational models - Unification

Companies that use a Unification model work very hard to wring out every drop of waste from often-complex processes.   Some attributes:

  1. Highly centralized management environment
  2. Company grows through leveraging economies of scale
  3. Process standardization seen as a key element of corporate success
  4. Frequently found in commodity businesses

One example that Ross uses is the core chemicals manufacturing business of Dow Chemical. 

IT in the Unification Operating model

As I said before, the operating model is the single largest driver of decisions in your SOA.  The impact of the model starts with the business, extends through business funding of IT, and into the architecture, design, and complexity of the IT ecosystem.  In a company based on the unification model, the following situations are typical:

  • Shared enterprise systems – Large IT systems are often used to reinforce the centralized nature of these companies.  It is not uncommon to see an ERP system managing a wide array of core functions, including HR, manufacturing, customer relationship management, and financial management.  It is also common to see a best-of-breed approach, where large systems from different vendors are implemented to support each function separately.  Integration is key to success.  However, the data models for the information typically is defined by these applications themselves.
  • Process owners – In order to get process consistency among the various operating units, a single person is identified to own each key process and they, along with their team, is responsible for insuring that the process is efficient, cost-effective, and productive. 
  • Central IT decision making – Decisions in the IT organization are central.  Systems are purchased, and deployed, with the goal of improving the cost effectiveness and efficiency of the company’s core business processes.  These companies tend to have very tight IT budgets, as IT is frequently not seen as a core contributor to process innovation.  Data masters are often centrally mandated.

Note that, in the unification model, variation between business units is kept to a minimum.  It is common to see the CIO report to the Finance Director or Chief Financial Officer. 

SOA and BPM in the Unification Model

The Unification Model is interesting because (a) SOA is expensive and may take years to reap full benefits, and (b) this IT environment is very cost conscious.  That said, the environment is already familiar with purchasing large software systems and taking a long period of time to implement them.  If you work in an environment like this one, and there is no SOA in place, you may make some traction by framing SOA as a single software package.  (Not as large as ERP, but larger than a shipping management system, for example.)  On the other hand, you could also get traction by tying SOA to a large system upgrade (see below).

The common data model

Once again, the key to implementing Enterprise SOA is the common data model.  While the cost of creating a common data model is far less in this model, the benefit may be difficult to explain.  That is because the data model is driven by the central systems that produce or consume the data.  The urgency for creating a common data model may not be high.

In this model, SOA provides two benefits.  Both require the common data model:

1) Protection from Vendor Lock-in: If the company has taken a “best of breed” approach to technology acquisition, then they have likely purchased different systems from different vendors for key business functions.  Integrating these systems is expensive, and there are scars to prove it.  This creates a “high barrier to entry and high barrier to exit” for these companies.  It costs a lot to put in a system, but even more to leave.  SOA can help lower those barriers by creating an abstract layer that processes abstract transactions in a standardized way. 

This allows a company to purchase a new Financial system (Microsoft Dynamics AX, for example), and instead of integrating each of the other systems to it, the IT department would simply write adapters to connect the new system to the abstract services layer.  Other systems can now directly consume the new system without substantial modification  (in theory, anyway.  Nothing is perfect.)

2) Process Composition across systems:  It can be difficult to track a single transaction, through a complex process, across many systems.  The customer doesn’t care.  You can directly impact customer satisfaction if it is difficult to figure out which system has the information that your customer needs to know, or to make it difficult to retrieve that information readily. 

So, if customer satisfaction is important, process composition across many systems is a key capability of your IT infrastructure.  To solve for this problem, you need four things: SOA as an integration mechanism, a common information model as a unifier, an enterprise identifier scheme to allow the transaction to be traced from start to finish, and tools to inspect each system for the information related to a unique transaction (using SOA, of course).

Funding SOA

If you get your information model in place, you could implement SOA as a combination of a package install and software adapters that is tied to a larger project, like an ERP replacement.  On the other hand, with this operating model, it is possible to build a bottom-up SOA, where the services are produced as part of IT projects without architectural coordination, primarily in areas where the information model is well-understood. 

Direct Impacts of the Unification Model on SOA Operations

The following effects would be typical for SOA+BPM in a Coordination model:

Centralized Process Management – Process owners manage a subset of the processes.  Processes are often coordinated.  Using the same BPM tool and repository is a best-practice, and one that will make immediate sense to the business.  The tool must be able to support a wide array of BPM needs, and must leverage standards.  

Centralized Governance Model – SOA Governance tools are quite useful in this model, and they should be used.  (SOA Software and Amberpoint are two Microsoft partners that I would suggest, for readers interested in this space.) 

SOA Service Adoption – Due to centralized decision-making, the decision to consume a service can be tied to project funding.  This bit of overhead should pay for itself quite easily, since you would end up with a greater amount of service reuse, and therefore, less code to maintain in the long run.  Some organizations have even gone all out, and implemented a set of core SOA services as a single project, then turning to governance to require all new projects to adopt them.  (Top-down SOA). 

Cross-system process concerns – Getting SOA benefits out of processes that look to a single enterprise system is a fairly quick win.  I like quick-wins.  You can build credibility for SOA by rolling out a few of these “low-hanging fruit” projects.  However, to get real enterprise benefit out of SOA, you need to be able to compose a process across many systems.  This can be easy, or this can be hard. 

To make it simple, repeatable, and adaptable, you need to create your common information model.  That model must contain not only information entities, but also a notion of what business documents you will communicate with, and what events occur on each document. 

SOA Readiness – While a central group may decide to implement SOA, each of the IT teams that surround the major systems will have different levels of understanding of the concept of event-driven services.  You will need to build a common understanding, and common standards, to make sure that these different groups, using different technologies, can reduce the friction that could occur when a process consumes many different services.

Centralized Service Catalog – You are likely to end up with a single catalog, but it may be a good idea to consider splitting the catalog into layers, with the upper layer of services oriented towards the business process areas that the organization cares about, and the lower layer of services to makes the information available.  Services in the upper layer consume services in the lower one. By separating services in this manner, you can simplify the SOA composition process. 


The Service Oriented Architecture for the unification operating model should take a cost-focused approach to delivering business value, orienting the services towards both process and information.

SOA in the Coordination Model

By |2007-10-19T21:12:00+00:00October 19th, 2007|Enterprise Architecture|

This is second in a series on the impact of the business operating model on Service Oriented Architecture.  (see overview)

Quick note about the CISR models

Before I go into a lot of detail about the operating models, I want to make one thing clear: The operating models from CISR follow the same laws as any other reference model.  To whit: “All models are wrong.  Some models are useful.” 

In other words, most enterprises will exhibit behaviors of each of the four CISR models in one aspect or another.  That said, most businesses will exhibit the behaviors of one model more than the rest.  This is usually apparent in the financial filings of the company (annual report, 10-Q) because it is difficult to hide the operating model of a company when you have to tell investors how you plan to deliver value in the coming year.

Also, as with all models, before you assume that your business is a Coordination model, make sure that you ask the business, not the IT team.  Best case: ask the executive leadership (CxO level) to debate the point.  It does no good for IT to have one view and the business to have another (path of pain).

The Coordination Operating Model

“The Coordination model provides integrated service to each key customer group.  The integration results from sharing key data across the business units to present a common fact to the customer.” (Enterprise Architecture As Strategy, by Ross, Weill, and Robertson)

operational models - coordination

The illustration that I use is above.  I have similar illustrations for the other models in the overview page. 

How to read this image: a single gray horizontal box represents commonality across business units.  In this case, a coordination model company will share customers and partners, even though each of the business units are managed seperately.  The business processes are distinct by each business, as is the operational data, but it is all brought together in the business intelligence as a way of understanding the customer from a 360-degree view. 

The example provided by Ross for a Coordination model company is Metlife, a financial products company that provides insurance and other financial services to millions of customers.  The customers themselves may qualify for, and should benefit from, the products of many different business groups.  However, each group may have their own “view” of the customer that is pertinent to their business. 

IT in the Coordination Operating model

The business model is the single largest driver of decisions in your SOA, whether you know it or not.  The impact of the model starts with the business, extends through business funding of IT, and into the architecture, design, and complexity of the IT ecosystem.  In a company based on the coordination model, the following situations are typical:

  • Shared data sources – in order to “bring together” the results of many different transactions into a rational Business Intelligence view, systems across the enterprise typically consume a set of core data tables, often from a small number of “source” systems.  If you agree on the ids for things like “country” and “operating unit” and “employee”, then business intelligence becomes possible. 
  • Coordinated Transactions – a business transaction in one business unit may affect the view of the customer in another unit.  For example, it would be rational to offer a discount on auto insurance to a customer who has purchased life insurance.  You may also have a situation where the revenue from a single customer has to be divided up among two or three business units for reporting. 
  • Integrated Business Intelligence on customers, products and suppliers – If you make no attempt to understand your customer in an all-up view, you are probably not in this business model.  The goal of this (frequently expensive) business activity is to provide insight into the patterns and behaviors of your customers, both in how those patterns work to the company’s advantage and how to root out sources of customer dissatisfaction. 

Note that, in the coordinated model, each of the business units is managed separately.  This allows business leaders a great deal of flexibility (and accountability) for making their part of the business successful.  It also means that the IT group tends to be divided up so that there are separate funding or reporting models to each of the different business units.  Centralization is infrequent, and coordination (of funding, of architecture, of standards) requires consensus.

SOA and BPM in the Coordination Model

The Coordination Model is interesting because (a) SOA can provide real benefits, and (b) this is the most difficult of the four models in which to create a SOA.  That is because of the impact of separate business management on IT funding, especially of a shared infrastructure like “common data models” or “common messaging standards” or even “common OS platforms.”  Anything ‘common’ is difficult, because the business units do not typically provide an incentive for their IT teams to create common models, and yet commonality is the key to getting value out of SOA.

The following effects would be typical for SOA+BPM in a Coordination model:

Distributed / Federated Process Management – Each business unit manages its own processes, using its own tools.  Processes are rarely coordinated across business units.  This is a fact of life for the BPM tools, and frequently there is little business value behind forcing all business units to use the same BPM tool, since they cannot share the primary value chain processes anyway.  Common corporate processes (like HR and Finance) are typically the only exception to this independent spirit.

Distributed / Federated Governance Model – A favorite topic among SOA folks is governance.  Governance has many meanings.  SOA Governance tools may prove ineffective if you use the tool to find a service, because you may not be able to tell which of your businesses paid for that service.  This is important because the businesses themselves are quite likely to affect the amount of governance they want to occur, and therefore the governance rules that “their” services are to be judged against.  

Semantic dissonance in the data – Making a service that is useful is nice.  Make a service that is reusable is great.  Getting two or more business units to adopt the service is darn near impossible.  That is because, in this model, you are expected to integrate the data, but not the business processes.  This usually creates a situation where the “semantic understanding” of the data is quite different between businesses. 

For example, in one business, a purchase order may be created by anyone but is followed by an approval process if it exceeds a threshold.  In another business, the purchase order cannot be created until the approvals have occurred, and perhaps, all purchase orders require approval. 

So, if you look in a database and you see a purchase order… has it been approved or not?  The answer depends on the business unit that created it.  While this example is overly simplistic, the goal is to show that, for some core business entities, there will inevitably be places where the underlying states for the information cannot easily be aligned.  This is semantic dissonance, and it makes Enterprise SOA a distant fantasy for those enterprises who are unprepared or unable to muster the consensus needed to find the commonalities in the various business processes. 

To bring together the full benefits of SOA will require a costly process to align the events that create your business documents and to create a shared understanding of “when the purchase order becomes a purchase order.”  That says nothing about “what data is in a purchase order” or “what system does a purchase order live in.”  All of these variables create complicating effects on Service Oriented Architecture.

Consensus processes for designing and deploying SOA – the decision to deploy Service Oriented Architecture cannot be made by one central group.  It must be made by many people, working together.  In a “coordination model” company, centralized groups are rare and often quite small.  They don’t have the clout to bring about the kind of change needed to deliver truly reusable services.  That requires “virtual teams” or “steering committees” that are often difficult to manage and can seriously delay deployments.

Service consumption is voluntary – Ever heard this fantasy: “If you build it, they will come.”  In a Coordination model, this idea, far fetched in any model, is downright silly.  IT projects, especially ones funded by different business units, are going to have a difficult time explaining to their business why they created a service that another business can use… especially if they don’t get some benefit out of it.  Reusable services are more expensive than point-solution services. 

Plus, reusable services force the IT team writing the consumer service to take a dependency on another team for delivery.  That nearly always causes problems.  Therefore, the only services you can easily reuse are services that (a) developed by the same business unit, or (b) are dictated from executive management, or (c) are so “thin” that they provide no real processing, and therefore, no real value.

Federated Service Catalog – Since processes may be different in each business unit, but some data elements are expected to be shared, it is typical to see one of two effects on the service catalog.  Either (a) each business unit has it’s own catalog, or (b) one common catalog exists, but each service is flagged with whether that services is “core” or if it is provided by a particular business.  The assumption is that non-core services will only be consumed by applications that are part of the same business unit as the service publisher. 


The Service Oriented Architecture for each of the business operating models will be quite different.  Those differences can determine not only how difficult it is to create a SOA in that business environment, and the amount of business value that can be generated by doing so.

SOA and the CISR Operating Models

By |2007-10-12T11:04:00+00:00October 12th, 2007|Enterprise Architecture|

I’m begining a new series on Integration.  Five parts.   Clear up some of the noise on SOA.  This series will refer to the set of four Operating Models described in the excellent book, “Enterprise Architecture as Strategy” from the great minds at MIT’s Center for Information Systems Research (CISR), Jeanne Ross and Peter Weill.  If you have not read the book, that’s OK.  Working Paper 359 on the CISR site will provide the right level of depth (there is a free registration on the site).

To quote from the working paper:

Companies make two important choices in the design of their operations: (1) How standardized their business processes should be across operational units (business units, region, function, market segment) and (2) how integrated their business processes should be across those units.

That’s two variables.  Assuming each has two values: Low and High, and you get four possible combinations.  Ross gave these four types of models different names.

  1. Low Integration, Low Standardization – the Diversification Operating Model
  2. High Integration, Low Standardization – the Coordination Operating Model
  3. Low Integration, High Standardization – the Replication Operating Model
  4. High Integration, High Standardization – the Unification Operating Model

I’ve illustrated these models in this way.

operational models

The book I mentioned above goes into excellent detail on the implications of these models, and what kinds of companies are well suited to each model.  Using success as a guide, you can quickly see how these two choices: Integration, and Standardization, drive huge effects across the organizations of these companies.  They also drive the kind of “Big Picture” Enterprise Architecture models that the organization should produce, because each one benefits differently from EA.

Many folks now believe that SOA and EA will gradually combine, so that they are two parts of the same message.  That story is interesting when you consider these four models, because SOA is radically different in each one.  I believe it is true, because the four models each benefit from both EA and SOA, but in different ways.  I believe there will be four types of Enterprise Architecture organization, and for each one, there will be a different set of methods, artifacts, and outputs… and a different SOA for each one.

And that is what this series will be about: the effects of the operating model on SOA.  I will cover each of the operating models and describe the effect on a high level SOA reference architecture for that type of business, as well as the technical implications with respect to governance, catalogs, standardized schema, and mechanisms for gaining unity.

Here are a list of links to follow-up blog posts that discuss the different aspects of SOA in each of the business operating models described by CISR.

SOA in the Diversification Operating Model
SOA in the Coordination Operating Model
SOA in the Replication Operating Model
SOA in the Unification Operating Model

Why do this?

I see arguments every day, in the blogosphere and in books and white papers, where thinkers disagree about the value of a catalog or the need for standards or the future of SOA in general.  The problem with these arguments is that most of the speakers are (a) correct, from their viewpoint, and (b) not changing their minds.  We have a set of entrenched voices saying different things.  So how can all of them be right?

I believe that there is more than one right answer, and that it depends on the operating model of the company.  In other words, the tools, standards, and approach that is useful for a Unification model are simply not optimal when applied to a company with a Coordination model.  That doesn’t mean that you won’t get progress, but you won’t get as much as you could without considering the operating model first.

For years, the IT thinkers have been saying “let’s get closer to the business… then IT becomes more useful.”  Guess what, folks!  There is no “THE” in business!  There are ten thousand ways to make money, and we need to serve all of them.  One size, or one approach, or one toolset, will not suit all businesses.  But chaos and cacaphony won’t suit our ability to be effective.

By using simple models like the Operating Model concept from CISR, we can create reference architectures that can be reused in different organizations to base not only their Enterprise Architecture program, but their SOA implementation, and with it, the toolset that they need to build an agile, responsive, efficient, and highly valuable IT organization.

Once the SOA leaders realize that there is more than one right answer, we can begin to have productive conversations, not about whether a catalog is needed, or not, for everyone, but what role a service catalog plays in a Coordination operating model SOA, and how that role is different in a Diversification model.  (it is, wildly).  Consensus can be reached if we understand the scope of the problems that we are trying to solve.

That is my goal. 

A Tale of Two Visions

By |2007-10-11T09:17:00+00:00October 11th, 2007|Enterprise Architecture|

As I am called upon, more and more, to present a clear “vision” for how SOA will occur, I realize that folks are using the same words for two completely different requests.  The trick is to provide both.

The question may be phrased as “Where are we going with SOA?” or “What is our integration story?” or “Why is it so hard to build shared services?”

There are two answers.  I need to provide them both, and if you are sharing vision, you do too.

Regardless of whether you are creating the vision for information quality, or application simplification, or a new way to manage the funding of projects, you need to provide two different vision statements.  In fact, you cannot possibly succeed unless you do.

You see, people like leadership.  They abhor pretentious noisemakers.  There’s a thin line sometimes.  And you can be the leader, instead of the noisemaker by putting yourself in their shoes.  Take the viewpoint of your audience when you share the vision(s).

First off: distant but attainable and highly valuable goal.  You have to show that there is not only light at the end of the tunnel, but that it is the light of nirvana.  You have to really sell a 3-5 year future where things that are broken today are not broken anymore, where folks do good work as a matter of course, and the business is successful. 

Second: the immediate and important first step.  Once people feel like you’ve painted your fantasy future, you need to show them what the first step is.  It has to be a valuable first step.  It has to have some use.  We must all benefit.  It also has to be a little bit painful, to make it clear that this is not a step that folks would take if it weren’t for that big-picture goal.  It has to be emminently feasable, politically attainable, and requires a small number of key changes, mostly affecting a single role.  That way, you have a small audience to influence, while you gain support from the greater community. 

Some people will be motivated more by the distant vision, others by the near term goal.  Regardless, you need both to make real changes occur.

Let’s say you want to bring in Agile development practices.  What would you need to do?

Well, you could convince a single development team to move forward with agile development.  That would work for a while.  But the other teams would not be interested in your success and it wouldn’t matter if it worked or not… after a while, the managers would move to different jobs and the department would revert to bad practices.

On the other hand, you could make your pitch to the CIO and the top level folks, convincing them of the amazing value they will get if they use Agile development practices.  What will you get?  A set of executives now expecting YOU to fix things, regardless of whether you believe you can.

You need to do both.  You need to take a pressing problem, one that is going to get solved, that is going to get funded.  With that problem, you need to convince key folks that the future of the solution is either “elegance” or “chaos” and you have the road to elegance.  However, you tell them, the entire organization cannot change at once, so you want “Just This Project” to adopt the new techniques and that will prove the value.

At the same time, you need to convince the team to take the first step.  Get training and coaching.  Start using some of the techniques.  Start using the terms in conversation.  Tell them that it is a new way and it will require unlearning some of the old habits, but that it will pay for itself.

At every step, you have to keep painting both visions: nirvana someday, effectiveness and value for now.

And then it must work.  If it doesn’t work, you will be set back, but if it does work, you will be ready to demonstrate it’s success to the people who want things to change.  And then, maybe, just maybe, they will sign up for “step 2.”

Two visions, near and far.  Both necessary.

If you don't like the senator's politics, attack his clothing

By |2007-10-10T15:55:00+00:00October 10th, 2007|Enterprise Architecture|

Pete Lacey started an interesting discussion a few days ago when he reopened the debate over the use of the term “SOA” in his blog post “What is SOA?”.  He certainly dove in to attack the idea of SOA, but did not come to any useful conclusions.

He starts by offering a humorous anecdote to explain, to himself apparently, why we need SOA.  The most interesting line is this, when describing how CORBA was a huge innovation:

“CORBA. Look, see. Interoperable remote procedure calls.”

Interoperable remote procedure calls.  Yep.  That was CORBA.  Jump down a little…

Worse, CORBA wasn’t quite as easy to use as it seemed. It was complex, nothing interoperated, it didn’t work through firewalls, and a bewildering number of specs were coming out of the OMG.

“Well, how about this new SOAP thing,” said the new new guy…

OK… so here comes SOAP to replace CORBA for what… interoperable remote procedure calls.  See where this is going?  Fast forward again.

We just need to give this idea a name.”

“I’ve heard it called service-oriented architecture,

So, according to Pete, SOA is a name for a technology, based on SOAP, used to give us interoperable remote procedure calls.  I shudder at the thought.

From there, Pete goes on to criticize SOA because it isn’t an architecture and the term ‘service’ doesn’t suit his mindset.  Heck, with that definition, he’d be right.

Give me a break!  Service Oriented Architecture is not RPC.  People who write RPC code to integrate applications have created a ball of rubber bands in their infrastructure… I don’t care if they used CORBA, or DCOM, or RPC, or Stored Procedures, or synchronous pixie dust!  When you follow an antipattern in your architecture, the technology cannot save you.

 The moment you criticize SOA on the basis of the technology, you’ve missed the point.  SOA is an architecture… a technology agnostic architecture.  It requires the basic things that are so difficult that folks are afraid to do them, and therefore the technical implementation fails and people blame the technologist… what are these so-important things that people are afraid to do?

Common Information Model

Common Business Event Model

Common Solution Model

Common Technology Model

I dare you to make any form of simplified, integrated, agile infrastructure work without these things. 

What is going to become of the ‘beloved SOA’ for those geeks who live and breathe the term?  Oh, who cares?  We will rename it and it will keep coming back, just as it has always done. Pete even suggests a new name, “Network Oriented Computing.” 

Oh… right… we renamed it.  THAT will make it work!

The concept is sound.  The concept works.  (Every implementation of SAP is based on the above elements.  SAP integrates extremely divergent systems into a single scalable platform.  There’s your proof of concept).

So stop arguing about the name, for goodness sakes.  Attacking the name of something you don’t understand is a tactic best reserved for pundits and political candidates.  If you don’t like the politics of your opponent, attack his (or her) clothing, hairstyle, automobile, whatever… something that has nothing to do with the point.

Just as Pete did.

The culture of art vs. the culture of engineering

By |2007-10-09T08:28:00+00:00October 9th, 2007|Enterprise Architecture|

You know what seperates an artist from an engineer?  Let’s say Joe designs something pretty cool.  Mary sees it and uses that design.  In Engineering, Mary is wise.  In Art, Mary is a thief.

For years, in many IT shops, and throughout the computing world, we have built a culture where it is better to write your own definition, your own design, your own snippet of code, than it is to use someone else’s.  We still reward this thinking. 

I work in the area of EA Frameworks.  There are quite a few frameworks emerging these days, and most of them are wildly similar.  The solution is right in front of us, and with some thinking, we are converging on the answer.  Yet, few people have taken the time to learn any of these frameworks (except the oldest and least comprehensive, Zachmann).  The number of people who have taken time to study two or more frameworks: geometrically smaller.

In Engineering, this would be foolish, but in Computer “Science” it appears to be simply acceptable for professionals to invent new definitions when existing ones will do, invent new frameworks when existing ones are ample, and invent new services when existing ones meet the stated needs. 

We need a broad array of people to take the time first to LEARN what is available, before inventing new things.  It is a culture of learning that is missing.  We have replaced learning with invention in Computer Science.  And as a result, we are a group of artists competing for space to show our artwork, instead of teams of engineers leveraging excellent ideas to build greater and taller and stronger constructions.

The lack of reuse is not a problem that we will solve by installing a SOA product or inventing a framework.  Building a culture of engineering requires that we recognize and reward the people who reuse other’s ideas and work, and that we remove the obstacles to their success. 

You can reuse code TODAY.  If your employer paid you to, and you took the time to learn, you could do it again tomorrow. 

The New Life of Joe – Part Three – The Users Are Coming!

By |2007-10-08T13:47:00+00:00October 8th, 2007|Enterprise Architecture|

Joe was in a good mood when he walked into the meeting with his Process Management leads for the first time.  Perhaps is was because Selina had just walked him through a consistent process for tracking and managing project information that didn’t keep everyone in “fire fighting mode” all month long.  “Orderly is good,” he used to say, and he believed it.  Ever since coming to this IT group, Joe has had a series of “rude awakenings” about their totally different way of delivering software.  This day was a good day.

That mood was about to change.

(Authors note: this is the third and final installment in a series of articles on Joe Freeflier, a new IT manager taking over a SOA+BPM driven IT group.  See Part 1 and Part 2 ).

When Joe sat down, he met his three process leads.  There was one for each business area that he worked with: Sales, Marketing, and Public Relations.  They were very different business areas, yet they maintained a very nicely integrated portal that all of their professionals, internal and external, used to do their work. 

After a round of introductions, Joe asked if they had project status available in the same portal that the shared services team used.  When they answered in the affirmative, Joe opened up the portal to look at the Marketing team’s project list, and stopped dead in his tracks.

There was one project.  The end date was the end of the fiscal year.  The budget was large enough to pay for the salary of every person in the team. 

Joe turned to Tom Rankle, head of the Marketing process team and asked for a rundown of project status and deliverables.

“Well, Joe, we don’t manage our team by project status and deliverables.  We manage by customer value points.  We have delivered 61 value points to the Marketing team this month through the updates to the advertising platform workflow and the new reporting interface for Marketing management.  That puts us right on target for delivery for the quarter.” 

“Customer Sat is running at 80% and our delivery backlog is sitting at an optimum level of six weeks of likely work.  Team morale is high, and our overtime ratio has been kept under 5% for the fifth month in a row.  Our current risks are that the new lead entry service update won’t ship in November, which would slip our updates to the event interface by two months.  Shared services has posted the risk as “yellow” so we are releasing the event update without the new service.  We will revisit it in the spring, after the big winter conferences.

I’d say we are pretty healthy.”  Tom finished by handing a small set of papers to Joe with all of these points in nice graphical format, showing steady values or steady trends. 

Joe was immediately lost. 

Where was the list of projects?  Where were the budgets and deliverable dates and change control processes?  The bug report triages, and the eternal fights over project scope? 

“Back up, Tom.”  Joe had to admit to his leaders that he wasn’t following the thread.  “I’m not sure I’m with you.  Can you walk me through the way your teams work?”

Marcie jumped in.  She was Tom’s peer, running the Sales process team.

“Let me give you some history…

“About two years ago, we were in a bind.  Our projects were always late.  Our customers were complaining all the time.  Every project was questioned, challenged, picked apart.  We would say something would cost 1 million, and the customer would say “you have half that.”  Most of the code was brittle, and team morale had gone below “low” to a kind of political viciousness… the most senior folks got to write new code, while the new guys always had to maintain the junk. 

“Then Michaela brought in the SOA and BPM guys.  We split IT up into shared services and process teams.  Our teams write the process bits.  We do workflows in the workflow management system, and we write the user interfaces.  The shared services team (or SST) provides data through services.  We own NONE of the data ourselves. 

“As a result, the stuff that changed most frequently, process and user interface, moved to our teams, while the stuff that needed to be stable (data and information management) remained with SST.  By breaking things up this way, the process teams could specialize on one thing: quick continuous improvement.

“We looked at our value chain.  You know that all of the process folks moved over here, and we had come from a lot of different traditions: Kaizen, Lean Manufacturing, Six Sigma… We all wanted to use our own process tools to solve our own problems.

“We found that the value of careful change control processes drops dramatically when the team writing the code cannot screw up the database.  We call services, and the services protect the database.  Well kinda.  It isn’t perfect, but it works fairly well.

“So we moved to an “agile development” model.  We take small teams, give them a list of features, and ask them to deliver value at the end of the month.  Each feature has “value points” assigned by the business.  We are measured on the maximum number of value points we deliver each month.” 

Marcie stopped and took a drink from her water bottle.  Then she jumped right back in.

“What we stopped doing: big analysis and planning projects.  In fact, all notions of ‘big planning’ are absent.  We let the Strategy and Planning worry about that stuff.  We just focus on delivering small, well-tuned, changes to the processes.  We measure key indicators before and after each process change. We make the changes using process management tools, not reams of complex code.  The most complex things we write are U/I code.

“These days, the users have started writing their own code using our mashup interface in Sharepoint.  It’s reduced some of the strain for small fixes and helps the customers feel “bought in” to the whole idea of using processes and services to define IT.

“So when Tom talked about his status, he was giving you the measurables that we manage by.  We care about value points, so we are always working to increase the number of value points we can deliver each month.  If we do our job well, customer satisfaction goes up.  So far, we’ve found that we can push the delivery of value points higher, but the quality drops and so does satisfaction.  So we don’t measure by value points alone.  We aim for a target “customer satisfaction rating” of 80% every month.  Tom has been nailing it consistently.

“One other thing we found is that the amount of overtime that the teams are working is a major driver of dissatisfaction.  Team members leave when you make them consistently work overtime.  So we capture the number of “forced overtime” hours per month, weight it towards the developers, and try to keep it low.  We find that does a good job of keeping morale high.  Staff turnover has dropped off dramatically since we started this plan.

“And when a service has to be changed, we involve the Shared Services team.  We collect the information and process needs, and the Shared Services team designs the service changes.  Sometimes it is simple, but often these changes are difficult, and they occasionally slip.  We find ways to deliver value anyway, which keeps our customers happy, even if we can’t deliver things in a predictable order. 

“The backlog is a measure of the amount of work that the business wants to have done.  We try to keep about six weeks in the queue at any time.  That way, if a shared service slips off the current cycle, we have useful and valuable work for our teams to do while they wait.  It is all described in terms of features, dozens of small changes that can be done fairly independently.”  Another drink of water, and she stopped talking. 

Joe took a minute to let it sink in.  What he was witnessing was a totally new way to “do Information Technology.”  These teams were not built the same way that his prior IT teams had been, and they were not fighting the same fights.  But they had solved the biggest problems that he had been facing: how to deliver value to the business customers on a regular basis.

The answer is here: combining Business Process Management with Agile Development in business-facing process teams. 

Finally Joe was getting it.  SOA is not about services.  It is about empowering a new way to deliver value. 

As the meeting proceeded, and Joe listened intently to the numbers that each team was delivering on, his mind kept wandering to his past IT teams.  “I wonder why we didn’t do this earlier?” 

He resolved to place two phone calls.  One to his replacement in his prior group, to invite him over to see this.  The other to Michaela… to thank her.

(authors note: this is the last of a three part series on Tom Freeflier.  See Part one and Part two).

The new life of Joe – Part Two – Managing Complexity

By |2007-10-01T12:27:00+00:00October 1st, 2007|Enterprise Architecture|

Selina Colander is up in Joe’s face.  Joe Freeflier, the new IT manager for the Sales, Marketing, and Public Relations IT group, called an all-day meeting for the project managers to attend a complete budget and timeline review for all in-flight projects.  Selina is not sure it is a good use of people’s time.

 “Don’t get me wrong, Joe.  You have every right to see what is going on.”  Selina was clearly frustrated, but went on explaining.  “I’d really appreciate it if you would talk to me before calling a meeting like this.”  

Selina continued, “I would have pointed out that the team already has a standing monthly meetings to discuss progress and more importantly, demonstrate features.  We call it our “retrospective.”  As for status, we have a portal with current information that is continuously updated directly from the projects.  The retrospective meeting lasts all day for you, but the project managers and developers only spend an hour in the room, and they don’t have to prepare.  They are primarily demonstrating progress, not talking about it.  If you want data, that is in the portal.”

(Author’s note: this is the second part of a series of blog posts about Joe Freeflier.  The first post can be found here.  The third and final post can be found here.)

Joe was pleasantly surprised, as he has been many times in the past few days.  Selina spent the next hour walking him through the portal and the underlying organizational structure.

“We have six teams.  They are:

  • Strategy and planning,
  • Sales Process Development
  • Marketing Process Development
  • Public Relations Process Development
  • Shared Services
  • Business Intelligence

“Everyone runs on a cadence with releases every two months.  The dates don’t move.  Each of the Process Development teams are able to deliver more frequently, but not less frequently.  The Shared Services team may deliver upgrades to many services, but they will all occur on the same day: the tenth of the month of January, March, May, July, September, and November. 

“Funding comes in through the Strategy and Planning team that allocates projects to each of the Process teams.  These teams own the user experience and all long-running business processes and workflow.  However, they do NOT own a single database among them.  Not one.  All operational data is managed, and all services delivered, by the shared services team.  The Business Intelligence team is responsible for managing the reporting infrastructure.

“This allows us to avoid a huge amount of overhead,” Selina pointed out.  “We did a study to show the amount of time that we spend in managing our projects.  We figured that good project management could reduce project risk by about 15%, but we were spending about 30% of our net time on project management, oversight, and control.  The distraction was huge.  Everyone was worried about managing the projects, to the point that we were forgetting to demonstrate progress on a regular basis.  Michaela changed all that.”

At the mention of her former manager’s name, Selina brightened.  She clearly had immense respect for Michaela, and Joe got the impression that if she could, she would have followed her when she left and moved out west.  He was really glad she had stayed. 

Joe thanked Selina and then spent the next hour alone, thinking about how he was going to take over this team.  They had created a system, not just a set of processes.  They had not been busy fixing little things about software development, like his last team.  They had started over, starting with the basic principles. They had rewritten the rules, and now he had to learn them, and quick, before he screwed up this machine by tinkering where it wasn’t broken.

What surprised Joe the most wasn’t that Michaela had created a good system, but that it moved so quickly.  It was all he could do to keep up.  Perhaps he didn’t need to. 

If he could go back in time and take a newspaper press operator from the 17th century, where a newspaper was printed one sheet at a time, and put them into a modern newspaper facility with the paper whizzing by so fast that you couldn’t see it, they would be starting over from scratch.  He wondered if he wasn’t starting over as well.

Better to learn the ropes first.  He headed off to his next meeting, this time with the lead architect for the IT group.  He was to learn about the Business Process Management technology that the process teams used. 

Something told him that this was not going to be boring…

(Author’s note: this is the second part of a series of blog posts about Joe Freeflier.  The first post can be found here.  The third and final post can be found here.)