//April

SOA by Wiki

By |2007-04-29T23:46:00+00:00April 29th, 2007|Enterprise Architecture|

I must be insane. 

I’m trying to organize a new SOA initiative, to cross some of the remaining boundaries between the different IT groups by taking “central” out of the picture.  I’m trying to get architect-to-architect collaboration in an area where we have not openly shared before.  And I’m trying to do it while still working at my regular work… no extra resources for this part of my job.

So, I’m using a wiki.  Direct.  Immediate.  Unedited.  Completely outside my control.

SOA by wiki. 

Nearly every initiative I’ve ever read about was “SOA by central planning” or “SOA by chaos.”  Most failed. 

Let’s see if coordination get us past the failures of those models.

This should be very interesting.  I’ll let you know how it works out.

Testing in a SOA world

By |2007-04-27T17:41:00+00:00April 27th, 2007|Enterprise Architecture|

SOA brings change.  It is a change to the way we do IT business.  No question of it. Anyone who has tried to ‘tack’ SOA onto the side of an organization has seen the resistence that this generates.  “We’ve always done it that way before… why change?” 

One place where SOA has an impact, but few people speak openly about it, is the change that SOA has on the world of software testing.

There are some huge changes here:

1) Regression testing: The only hope you have for insuring that your service can handle itself in a changing environment is to create automated regression tests.  This was optional in the past. Now it is both Required and quite Feasable.  Since services have no User Interface, there is no need to worry about whether the control has moved. Maintaining an automated regression test script is much easier.

2) Boundary testing: If the intended use of a service is a good thing, what about the unintended use of a service?  Can the service survive being hit with FAR more requests than it was designed for?  Does it throttle itself?  Does it protect itself?  Can private data leak out? 

3) Integration testing: A LOT of the capability of a Service Oriented business app will move to the composition layer.  Testing the services helps to establish a baseline to show that the defects should be avoidable below the service line, but many of the bugs will occur because a composed service assumed that an underlying service would have a side effect that it does not have, (or vice versa).  As defects are found in Integration testing, the test cases need to be updated at the service level to insure that assumptions are constrained and tested.

4) Stub testing: you may need to test the composition layer before a service is available or while it is only ‘local’ instead of ‘enterprise’.  For that reason, the test team needs to be able to generate ‘stub services’ that apps can call that behave in a manner that is compatible with the service definition, but has far less effect.  Otherwise, integrated testing is a joke.

5) New service validation and service compatibility validation: if a new service is entering an environment, and the goal is for the users of the existing service to transition over to it, then there has to be a way to test the new service to insure that it is compatible with the existing one.  Automated regression tests that were designed to test the existing service need to be pointed to the new service and failures noted.  Note that the team developing the new service is not likely to be the same team as the one that developed the prior one, so source code for the regression test must be available and shared and documented and stable.  This requires a level of ‘test team integration’ that many organizations will find challenging.

This is, I’m sure, a subset of the changes that SOA brings to the world of testing.  I encourage those who are involved in testing to share other ideas and concerns that they have come across with respect to SOA development.

Improper use and the SOA free market

By |2007-04-26T14:33:00+00:00April 26th, 2007|Enterprise Architecture|

I asked some folks in my talk before the US Partner Senior Architects Summit yesterday if they had created services that no one used.  Not surprisingly, some hands went up.  One architect piped up with this “I’ve seen a service that was used improperly and it brought down the enterprise.”

Um, OK.  Sure.  I can write a service badly.  I’m sure you can too, without much effort.

Some folks would say “that’s why we need SOA Runtime Management Tools!” 

Bunk.

Every app is completely responsible for protecting itself. 

Whether it is from a Denial of Service Attack to the web site, or an attempt at unauthorized access to the data, or an improper use of the service interface, apps must manage their own stability.  We need to realize that a service has the same flaws and foibles as any app.  It can be used.  It can be misused.

Plenty of ways to solve for this.  A subset, off the top of my head:

  • test the heck out of your service interface.  Testing is our friend.
  • configure for DoS attacks against the service interface
  • use intermediaries that support throttling, so that you can manage the inbound traffic
  • secure the interface, so that only members of known groups or even specific system accounts, have actual access.
  • don’t advertise.  Don’t flag your ‘service’ entry in the repository as ‘public’ or ‘feely callable’ if you aren’t. 

That is not a SOA Governance problem.  This is a training problem. 

Redefining SOA Governance

By |2007-04-22T05:32:00+00:00April 22nd, 2007|Enterprise Architecture|

My third post in a row about the notion of SOA Governance.  You’d think I was planning to speak about Governance this week.  (I am).

In general, Governance is a set of processes, responsibilities, and tools that reinforce good behavior and help avoid bad behavior.  With SOA Governance, we want to build a useful SOA environment, prove the ROI, and insure we don’t screw up security.  We do this with policies and processes (mostly process).  Policy can only be applied to a small fraction of the governance problem.

Now, break it down further.  Only a fraction of SOA policies can be verified or reinforced with tools. 

Therefore, while SOA tools are useful, they don’t deliver governance.   In my honest opinion, it is not rational to use the word Governance to refer to a tool at all.  After all, we don’t refer to to Microsoft Operations Manager as a “systems governance tool,” and SQL Server is not a “data governance tool.” 

Here’s a novel idea: Let’s use similar words that have been successfully applied in other areas, so that the meaning is clear.  If MOM is a Network Management tool, then Systinet, IONA, and Amberpoint are Service Management tools. 

Tools manage.  People govern.

SOA Governance – Software is about 20 percent

By |2007-04-21T00:40:00+00:00April 21st, 2007|Enterprise Architecture|

So your CIO says “build SOA.”  You do a search and plop down your hard earned cash on a SOA Governance tool.  Do you now have what you need for SOA Governance?  Nope.

Most of it is outside the scope of software.

Don’t get me wrong.  If I have a SOA environment, I’d like to know some things that software CAN help me with.  I’d like to know what services are running that are not compliant with security policy, or that expose private data, or that allow unauthorized access to otherwise-secure system services.  That is useful.  That is ‘runtime governance’ or ‘service monitoring.’ 

But it is not comprehensive SOA governance.  Not even close. 

The whole point of SOA is to create an agile environment, making it easier to build fully integrated applications from the get-go.  This is the goal.  If your services don’t allow you to build service oriented applications, then you have wasted your money and time.  Governance is about making sure you don’t waste your time and money by building the services you don’t need, or failing to build the services you do need.

Governance helps you to do the following activites.  These activities occur at particular stages of software development (planning, envisioning, design, construction, deployment, support, maintenance) as follows:

Activity What it gives you Stage
Business Service Analysis An understanding of the data entities and process steps that drive the need for the creation of a service. Planning
Service Partitioning An understanding of the different levels of services (data level, orchestration, composition, management) needed to meet the needs of the business, what each service will do.  This drives the definition of business events and documents. Planning, Design
Event and Schema design A plan for the behavior of the services that meets the operational, informational, and business process needs of the organization.  Behavior is often described as a protocol, but it can include service level expectations, exception management and compensation definition Planning, Design
Security Policy Creation / Management A set of standards for how services will be secured, what level of authorization is needed for services of different types, how network boundaries will affect the access to different forms, levels, and types of data. Planning
Operational Policy Creation / Management A set of standards for how services will be constructed so that they can be seen, tracked, managed, audited, and monitored.  Planning
Policy enforcement Automated application of policies to services running in the network Deployment, support 
Service Monitoring Automated monitoring, logging, and tracking of service calls to insure that service levels are maintained and to aid in debugging and exception handling. Deployment, support 
Rogue service discovery Automated discovery of services running in the network to capture services that may offer uncontrolled functionality, backdoor access, and audit gaps. Support
Service Registry / Repository Tools for sharing information about services, both with consuming applications and with the people who create or use them. Planning, Design, Construction, Support
SOA Project Compliance A process for insuring that projects funded in corporate IT departments actually consume or deliver the services needed by the enterprise. Envisioning, Design, Construction

I highlighted only a few rows: Policy enforcement, Service monitoring and Rogue service discovery.  These are the areas largely covered by the leading “SOA Governance Tools.” While these elements are important (honestly), they are about 20% of the story. 

 A little CYA here, so I’m not flamed by the vendor of such-and-such software:

1. There are probably software packages that overlap in some ways with the ‘uncovered’ areas, but there is not a lot of visibility to these areas, and these are not largely the features that these tools compete on.  When they exist at all, they are “extra” features.

2. Many tools, in order to support policy enforcement, will provide a tool for entering and managing a library of policies.  That is not the same as Policy creation.  It is policy encoding.  To say this is policy creation is like saying Outlook’s address book creates customers.  Policy creation is a business process.  You can buy policy templates, but you cannot buy policies.

3. My opinions are my own and do not reflect those of my employer, or the partners of my employer, or anyone else on Earth. 

Unfortunately, the competition between the vendors hoping to capitalize on the SOA ‘movement’ have become louder and more strident as each day goes by.  Because it is normal to draw attention to your product, and proclaim it as loudly as you can, I cannot blame the vendors of “SOA Governance” software for drawing attention away from the rest of the needs in this list.

However, if you are a SOA practitioner, you inevitably run into needs in each of these areas.  You need to do each activity in some way, as part of your governance strategy.  You will have people, process, and tools aligned around each and every one.

So if you are setting out on your SOA journey, don’t for a minute think that you can purchase a software package to give you comprehensive SOA governance.  Most of the governance you need is outside the scope of software at all.  It is in the people and process, decision rights, funding mechanisms, and IT leadership that allow you to build, govern, and manage a SOA-based infrastructure.

One more word for the heap: governance

By |2007-04-20T02:36:00+00:00April 20th, 2007|Enterprise Architecture|

No one in IT wants to talk about Governance.  Why?  Because no one has a consistent clue what it is, and those folks that venture a guess usually come up with something frightening, overbearing, and/or expensive.

Good old FUD (“Fear, Uncertainty, and Doubt”).  A great way to spoil your day.

Some words get overused.  SOA has been overused, although there is now a consensus on what it means, which means we can actually keep using it.  Other words, like alignment, and strategy, have so many different meanings that they can be twisted to mean “something good that I’m doing.” 

“Governance” went the other way.  It is a word that has become synonymous with “something bad that you shouldn’t do if you want to ship your code on time.”  Except in the SOA world, where it means something altogether different.

Governance is basically a system of processes and decision rights that reinforce the good behavior of the organization and help to maintain balance between the passionate creative brain-stormers and the careful, conservative, dependables.  It is a system that fosters creativity while restraining wild thrashing.  Governance keeps good things in the mix while preventing bad things from happening.

It is not software for monitoring uptime.  It is not a process of oversight and financial audits.  Most of all, it is not a system that squeezes every idea until any spark of creativity is winked out. 

Governance is a good thing.  Unfortunately, we need to come up with another word.  This word is no good anymore.

How about “Constrained Empowerment”?  🙂

To boldly crawl where no one has crawled before

By |2007-04-18T05:13:00+00:00April 18th, 2007|Enterprise Architecture|

Yes, I’m a trekkie.  I’m also a student of leadership. 

A good friend once asked me how I can get a particular manager to take up a challenge that he wasn’t taking on.  Both of us thought that he should.  To us, it was obvious. To him, it was not.  He was basically staring at the opportunity, doing nothing.  At one point, I even privately criticized him to a peer.  I regret that.  I hope to learn from that mistake and never repeat it.

What’s going on here, you ask?  Why not get the incompetent people out of the way, so the rest of us can get our jobs done?  Why?  Because there, but for the grace of God, goes I.

Every day this happens.  It has happened to you, at some point.  Possibly many times.  You (or someone you know) has been placed into a role where you have no experience.  Complicate this with a sadly common correlary: you have no support to learn the role well.  Sink or Swim.  That’s what happened to this manager.  He was a good man in a bad spot. 

Whose responsible?  That person’s manager, perhaps.  That person himself?  A little of both, methinks, but no matter how he got in, leadership is the only way out.  You don’t have to be that person’s manager to lead them.  We all challenge each other, every day.   I could have led him out.  I did not.  Another regret, unfortunately.  Another lesson to learn.

If you place a challenge in front of someone, and it is a challenge that they have not done, and you have not done and no one they know has done, you are asking that person to demonstrate a combination of creativity, leadership, and sheer chutzpah.  A very very very rare combination that is even more rare among techies.   Music, we are good at.  Math, sure.  Leadership and guts?  Pass the pizza.

Even if you are experienced, you can’t do another person’s job for them.  Not for long, anyway.  So how do you get a person, in an unfamiliar role, to take up a challenge and do the things that you think need to be done to solve for that challenge?

I have one answer.  I’m sure there are others, but the answer I have is this: give them a specific problem to solve… and convince them of the value of solving it… Let it be a problem that leads to other problems, until the student see the end goal: solve all problems of this sort.  Get their head in the game. 

“To boldly go where no one has gone before” is a good “trekkie” motto, but that isn’t leadership.  That’s blind faith.

Leadership is to ask someone to go one step towards an attainable (short-term) goal they understand, even if it is not a goal they have ever sought before.  As a leader, you give them a reason to go there, criteria for knowing when they are there, and rewards for getting there.

And whatever you do, leave the sharp wit at home.  This stuff is hard enough as it is.

SOA Question: should we carve service independence into the database?

By |2007-04-16T08:24:00+00:00April 16th, 2007|Enterprise Architecture|

One question we will be looking at is a fairly key one: as we build our Service Oriented Business App, and therefore drive the creation of services, we will be taking our nice, app-specific, logical data model and breaking it up to show which parts are in support of which services.

Do we then break those parts up into different actual databases?  Alternative: we could simply create different sets of tables within the same db (a practice that the dev team is already familiar with).

Let’s look at both sides. 

  • If the services are truly independent, they may need to scale seperately.  They may need to move to other locations.  They may need to change structure independently.  All good reasons to seperate.
     
  • On the other hand, if I have two databses, and make backups of them at 3:02am and 3:04 am, and the server hard drive goes down at 10am, prompting a restore, do I risk creating an inconsistent database situation because of the two-minute gap?  Do I add complexity in database operations by creating the hard line between my service databases?

Important:

  • If the services are designed well, there should be no cause for a single transaction that adds data under two different services.
     
  • There should not need to be direct referential integrity between any two independent service databases. 

I’m leaning towards forging a hard line.  I’d like to know what you think.

Is Central Planning impossible?

By |2007-04-14T21:14:00+00:00April 14th, 2007|Enterprise Architecture|

My friend (and sometimes office mate) Harry Pierson blogged a quote from Christoph Schittko and added to it:

[Christoph] “wonder[s] how many more attempts for “enterprise wide” thingies we need for people to figure out that there’s too much complexity involved to coordinate anything enterprise wide.” I couldn’t agree more, though I think it’s more than just complexity at work here. There are significant forces driving decentralization in society in general and IT in particular, and anything enterprise wide is by definition centralized.

I agree and disagree, depending on what you mean by “central.”  (As you may expect, since I work for a ‘central’ architecture team).

When Walmart moved in to my neighborhood two years ago, they went to the city council and zoning board and asked for a permit.  Of course, no one had to issue the permit, but it was in the best interest of the city to do so, so they did.  They didn’t have to change the structure of the city to allow them to come in.  There were already roads and electricity, because the planning board had already planned on the existence of a retail hub. 

Could you consider the zoning board to be central?  I would.

I live in a small-ish suburban area.  What about the zoning board(s) for larger cities?  Are they ‘central’?  Is there “too much complexity to coordinate anything enterprise wide” in the case of a city zoning board?  I don’t think so.  They regularly deal with zones for industrial, single and multifamily housing, retail, distribution, roads, utilities, etc.  Zoning works.  (If you think otherwise, spend some time in Houston, where there is no zoning board.)

So why does central control work for city zoning boards but fail for IT?

  1. Most IT architects don’t understand the boundary between solution architecture and enterprise architecture.  City planning decides that a shopping area needs to sit next to a housing area in order to meet the growth goals and trends of the town.  It does not decide who the builders are, what stores will be built, and what products will be sold.  The board does help decide where water, electricity, sewer, rainwater, and pollution mitigation needs to play into the bigger picture.   Solution architecture should take on the role of ‘architecture within the land’ and let Enterprise Architecture take on the role of ‘architecture within the city.’
     
  2. Most companies have not completely embraced Enterprise Architecture.  As a result, there isn’t an acceptance either of it’s value, or of the need to seek the ‘permit’ process, much less the need to comply with it.  Imagine a zoning board in a wild west town.  Now you get most IT departments.
     
  3. Most companies don’t have an EA permit process, so even if an IT team “wants” to play, they can’t.  Where the company may have a process, it is rarely as well managed as a city permit process.  For some reason, Enterprise Architects think that their work is somehow “different” from other large environment planning positions, when it is not.  It is time for EA to take a page from central planning models that work.
     

Central planning works, but not in a vacuum and not with a heavy hand.  We haven’t figured it all out yet, but that doesn’t mean we throw in the towel and live in the wild west. 

 

 

Simplification happens

By |2007-04-13T21:02:00+00:00April 13th, 2007|Enterprise Architecture|

It’s nice to know that IT Simplification is happening in other shops, not just Microsoft. 

Citigroup announced that they will be cutting a billion dollars from their expenses, and a large part of that will be IT simplification.

 Excerpt:

The company will:

Continue to rationalize operational spending on technology. Simplification and standardization of Citi’s information technology platform will be critical to increase efficiency and drive lower costs as well as decrease time to market. Examples of this are: consolidation of data centers; improved capacity utilization of technical assets and optimizing global voice and data networks; standardizing how the company develops, deploys and runs applications; and maximizing value by limiting the number of software vendors to operate at scale.

In Microsoft, we are pursuing simplification, although I can’t say if we will save anywhere near that much money.  I don’t believe we are quite that screwed up ;-).  On the other hand, one downside to working for a company that writes software… we have software.  LOTS of it.  For a company of our size, we have way too much of it.

And that is where I come in.  In Enterprise Architecture, and in our own Simplification initiative, I am one of the ‘noisemakers’ for simplification.  I honestly believe, and I back my words with works, that if EA is going to truly affect the bottom line of the company, we will do it through two mechanisms: agility and simplification. 

That’s my scorecard.