//March

If you don't trust your experts, how expert are they?

By |2007-03-30T23:11:00+00:00March 30th, 2007|Enterprise Architecture|

I had a great chat with an experienced IT software development leader in IT this afternoon.  He was telling me of a business customer that he had once worked with.  His customer had the habit of coming to IT with not only the problem to be solved, but the entire solution already mapped out.  They didn’t want IT as a partner.  They didn’t want them as a vendor or even a contractor.  They wanted to use IT as a hired lackey.

He began sharing an expression with his business partners to get them to face the consequences of this kind of behavior.

If you come to me with a problem, you’ll get a solution.
If you come to me with a solution, you’ll get a problem.

I love that.  It’s true in so many ways. 

But it speaks volumes about a lack of trust between business and IT.  If our business customers believe that we are experts at what we do, they would want to ask our opinion.  So why do some folks still insist on solving the problem first, without IT in the room?

Perhaps they don’t believe we are experts? 

And if they don’t believe it, just how expert are we?  Doesn’t the core competency of “expertise” include the ability to communicate competence and reassure partners that problems will be solved, value delivered, and integrity maintained? 

So, the next time you find that a business partner comes to you with a solution… thank them.  They have shown you two things: that they have creative ideas, and that they don’t trust you.  Listen to their ideas.  Earn their trust.  (I’m not sure you can successfully do one without the other). 

Then, do what is right for the customer and the enterprise. 

Getting a system to consume a "fictional" service

By |2007-03-24T09:42:00+00:00March 24th, 2007|Enterprise Architecture|

Have I got an architecture for you!

It’s cool.  We have a set of services that underlie the user interface and collaboration infrastructure.  The U/I can be built entirely on services, and the services are independent of one another.  On paper, we are isolated and clean.

One problem… some of those services don’t belong in this app!

You see, we are building an infrastructure that will allow integration as a “root behavior” of the app, not an add-on.  The app will call services in other systems to get key data.  That data won’t be local.  So, part of the services infrastructure includes services in other apps. 

Those services don’t exist. 

Three choices here

  1. 1. We write a service that drills down to the database of the other app, on behalf of that app.  When our project is done, that service becomes part of the other app.  To do this well, we need to study the code of the other app, and make sure that our new services correctly implements the business rules that the other app delivers, as well as the data elements.
     
  2. We write a service inside our app that turns around and calls an existing integration point.  For example, let’s say that the other app routinely exports a flat file.  We write code to bring that flat file into DB tables and write a service that presents the data from our DB tables as though it was from that other app.  At some point in time in the future, we write a new service, in the other app, that mimics the service we wrote for ourselves.  We kill off the data feed at that time.  The data is right, but the business rules are simply absent, and the data is mostly read-only.
     
  3. We write a stub.  This is a service that behaves similar to the way the other app behaves, but the data under it is either entered by hand or manufactured.  Entered by hand may be the developers doing a “one time feed” or it may be business users with an interface where they type in data, or support engineers modifying db tables (or even an XML file) that contains the data.  Now, our stub had to include business rules that we invent, hoping that we get kinda close to those implemented in the other app, and we make someone jump through data entry hoops.

All three are poor options. 

If you want to know why SOA has a hard time taking off, this is it. 

Two ways to counteract this, both must be done simultaneously:

  1. When planning an app, build the services that you “think” others “may” need, even when no one is consuming them.  You will be wrong, of course, but it provides a hook for another dev team to create a service that they really do need, and that service calls yours.  The other team doesn’t have to learn your business rules, just your protocol.  Later, when your project has time, you rewrite your service to meet many or all of their needs.
      
  2. When building a consumer app, add the expense to the project to build a formal service in the producer app (option 1).  This is best done by finding someone already familiar with the data and capabilities of that app.  If you cannot do that, then picking one of the other two options takes on risk.  Be prepared to have a follow-on project to clean up the mess you make.

SOA is intentional.  In order to build services that others “may” need, you have to know what those services are, before you need them, and you have to know what systems should provide them, so that when funding happens against those systems, you are ready to pounce with a requirement to build the service.  That requires you to plan out the interactions between most every system… a complete future state architecture.  It doesn’t get much more difficult, or important, than that.

Web 2.0 vs. the IT department

By |2007-03-23T08:42:00+00:00March 23rd, 2007|Enterprise Architecture|

Michael Platt posted a set of observations recently that offered up some troubling conclusions.  In his post, which you should read, he noted that most of the folks interested in creating Web 2.0 sites were not talking to their IT departments to make them happen.  These creative and interested business and marketing leaders were turning to external firms to create their Web 2.0 sites, while the IT departments at the very same companies were reporting that their business customers were not interested in Web 2.0!

I live in an IT department.  I can certainly vouch for the IT side of that story.  I’ve gone to various business teams and asked them about Web 2.0 capabilities, and they appear interested, but not enough to fund a project that will allow their content to ebb and flow, or empower their segment of the community to interact. 

Personally, I find it both frustrating and a little humiliating, but I don’t think it is an IT problem per se.  I think it is the bruising politics of IT – business relations.  In other words, there is so little trust, that over time a kind of “mean politic” has emerged which is fought out mostly in the budgetary space.  No one gets a dime without BOTH teams spending a fortune on “oversight” to prove that the projects won’t fail or will deliver (know what?  Oversight doesn’t deliver projects that meet business needs.  Agile does.  Hard lesson.  Different post).

In the climate of massive oversight, and detailed inspection, there are rituals that MUST be followed in order to make something creative or interesting happen.  Those rituals are governed by people who are, to be honest, tired of following them.  For some reason, we all accept the basic assumptions that these rituals are actually useful or necessary. 

In this climate, if a business user really wants to connect with their community, they have no way to get their project funded.  It takes huge amounts of time and effort on the business side to create a justification for a project, so that the project costing can be fed to the financial teams who perform the budgeting for IT, where senior folks inspect every detail of the business case and rank projects on the basis of net present value. 

No way a Web 2.0 project would survive that.  Heck, it would be laughed out of the process by the business team responsible for submitting projects to IT, long before it gets to an IT group for cost estimation!  And then, it would be slapped with a huge “new stuff” tax that IT groups place on any project that makes them do something new or different.

In our little corner of the company, we wanted to put in Web 2.0 two years ago.  The problem that we faced was this: the portal software we were using didn’t allow REST or SOAP services because of the security model.  As a result, we built a “psuedo-service” so that our Ajax-inspired-but-not-directly-derived-and-therefore-expensively-hand-coded web controls could pull down the data they needed when they needed it.

It’s an excellent first step.  But did the business want to go the extra step, on the same project, and describe what that psuedo-service was and did, and share that information with our end customers?  Nope.  That bit was never suggested, by either side. 

So here we sit, two years later, with no more of a Web 2.0 story than we had when were first inspired to create one.  We got as far as we did, as an IT group, because we didn’t ask if the business actually wanted Web 2.0.  The business wanted functionality, and we could deliver functionality in this way.  The oversight process let us build it because the “mean politic” didn’t realize that we were changing the landscape.  We didn’t tell the funding machine we were trying to build anything radical or new. 

On paper, we looked ordinary, creating a “consistent user experience for our customers.”  So we got funded.  (I had nothing to do with it.  I shine in their reflected glory, but I give credit to key individuals inside the team and one or two consultants who believed in new ideas and made them happen).  Maybe that is how other IT teams can put Web 2.0 to work… by not using the word.

Do we, as an IT group, have the courage to make the next step, and lead the business to a real Web 2.0 world?  Can we convince the brutal, conservative, mistrusting funding process to let a little more innovation to survive?  We shall see. 

Until then, the Web 2.0 boutique companies will not get a lot of competition from IT. 

Understanding Enterprise Architecture

By |2007-03-22T11:12:00+00:00March 22nd, 2007|Enterprise Architecture|

Just came back from an all-day offsite with the EA team in Microsoft IT.  It occurs to me, in speaking with my collegues and in side conversations, that we have a good idea of what Enterprise Architecture is, and how it benefits the company, but many of our stakeholders do not.

That’s not to say that we are perfect, or that other people should just “guess” our value.  The Value of EA needs to be carefully tailored to fit the actual needs of the organization and we have to do a good job of making that value visible and sharing it with others.

On the other hand, I know that some good folks in both business and the rest of IT have different impressions of what “enterprise architecture” is and what it does for an enterprise.  Some may feel that the value of architecture is to design the solutions that the company needs to consume (build the solutions right!).  The real value, in my mind, is to help make sure that the money spent on IT is building agility and not complexity (build the right solutions).

I cannot emphasize this enough, and as we share the message with our friends and collegues, we need to be clear about this point.

Enterprise Architecture is not about “building solutions right”

Enterprise Architecture is about “building the right solutions”

There is a misimpression in IT: the business knows what they want, let them pick the projects.  That works if you believe that the value of IT is to take orders.  However, if that is the value of IT in your company, you should toss out your entire infrastructure and outsource the IT function to India or China… they can take the same orders for a lot less money.  That is the commodity view, and it is a dead-end for IT development.  It is also a huge lost opportunity.  The folks in IT often understand the business much better than the policy teams in the business itself.  We not only had to learn the business needs to build the systems, but long after the sponsors of those projects have moved on, we’ve been the shephard of those needs through multiple revisions of the code.  That knowledge has built up in IT.  Failing to leverage that knowledge is an invitation to your competition to stomp you.

The real value for IT is in providing strategic information, opportunities, and success to the company as a whole.  That means that IT figures out what projects need to be worked on, in partnership with the business.  IT works not as a servant but as a knowledgable and capable business partner, suggesting business opportunities that the business may not have thought of, and taking it to the business and executive management for funding. 

Ideas need to come from both sides, and both sides need to respect the ideas that come from the other.

The business already has an idea mill.  It is new only in IT.  And where does this “new” IT capability lie?  In an IT Strategy and Planning function that includes Enterprise Architecture.

When a task consumes your life

By |2007-03-16T22:33:00+00:00March 16th, 2007|Enterprise Architecture|

I’m working on a single task, part of a much larger project.  It is on the critical path, and I know it is important, but darn it, it has consumed my life.  I hate it when one single decision, one single thread of effort, becomes so time consuming that I have to move other, important, items out of the way to get it done.  I’m definitely working more than 40 hours in a week.

When this happens, it is good to look up every now and then and realize that I cannot “invent” more time in the day.  I have to be efficient with my time and my effort.  My family has already suffered from all the time-stress I’m under and I need to think about every scrap of time.  When the time bucket is full, saving a minute in the afternoon is a minute I don’t spend in the evening.  Each minute I spend, whether it is at 9am or 9pm, is time that I could be spending with my family. 

That’s the metric: is this bit, this task, this meeting, more important than tickling my 8-year-old daughter or helping my son with his homework, or just holding my wife’s hand and talking about our day? 

What’s your metric? 

 

Hero or Rebel

By |2007-03-13T03:04:00+00:00March 13th, 2007|Enterprise Architecture|

If you do what is needed, but not what you are told, are you a hero, or a rebel?

In software, as in life, there are situations where you have to choose. Literature is filled with stories where the daring young man is told to “stay put” and he rushes into danger to save the damsel, or the brave soul takes on injustice when all counsel her to “stay out of it.”

So when you are writing software, what room do you have to do the right thing?  If you are looking at a set of requirements, and it is clear that a use case is missing, do you write code for it?  Do you invent your own process because the customer clearly needs it?

I think, in a team environment, the answer is muddy.  Perhaps that’s why our heros of literature and the movies all prefer to “work alone.”  In a team environment, you have to trust the skills of those around you.  If a use case is missing, point it out, but if you write code, you may find that the customer Chose not to implement that use case, and you just risked the delivery of the project!  Perhaps the functionality belongs in another app, or perhaps the data is supposed to be stable so no interface is needed. 

Personally, I like team heros.  A “team hero” is a team member who offers up honest answers when asked, who praises team mates in public, and offers constructive feedback in private.  They get their work done efficiently so that they can help other team mates in need.  If a team member is not doing well, for whatever reason, they help rally the strength of the team to assist that team member over their hump.

A team hero doesn’t have to ask if it is OK to do what is right, because they would be asking the team, and the decision is already made.

So cudos to team heros: those few key players who, through sheer passion, are singularly responsible for both the quality and capability of the teams fortunate enough to have them. 

So what's in a roadmap, anyway?

By |2007-03-12T08:14:00+00:00March 12th, 2007|Enterprise Architecture|

Microsoft is an odd bird.  It is a dynamic place, with different business units moving at their own rate, not hindered by much of a ‘central authority.’  It’s an approach designed to foster innovation, and it works pretty well.

Where this doesn’t work is in IT and business process simplicity.  Every different team will approach mostly the same problems over and over, but because no one wants to rely on anyone else to deliver for their business, or be responsible for a mistake, then everyone develops their own solution to the common problems.  Finance and HR are the exceptions.  These well-run teams created simplified processes long ago because the company would never have been able to survive at the size and speed it moves without them.

Different processes and tools cost a lot of money and slow the company down.  They need to be streamlined, simplified, and removed. 

That’s where a roadmap comes in.  This is a single document that includes the information that everyone AGREES upon with respect to tools, processes, and timelines for bringing all interested parties to a single ‘reasonable’ answer.  The roadmap has to be updated every year, and there has to be a way to making sure that different roadmaps don’t overlap one another or leave big gaps.  In other words, every key business capability that has to be automated or addressed with a process should have one and only one roadmap where it is included in scope.  (that’s harder than it sounds.  different blog… different day).

We have a roadmap for every solution platform.  A platform is a business grouping of applications that solve similar needs.  (Don’t think OS or SQL Server… think ERP or CRM).

The end result is one document for each platform.  Here’s what I have in the template for that document:

  • Purpose of this Document  – why write the doc
  • Stakeholder Chart – whose needs are being met with the solution
  • Signoff Chart – timestamp and id of person who signs off.
  • High Level Platform Capabilities – the solution capabilities demanded by the business
  • Business Drivers for the Capabilities – the list of business programs or business teams that will use the system
  • Capability Demand – which business are asking for which capabilities.  An interesting grid.
  • Technical Interdependencies – what other systems rely on this solution.  What other systems does this solution rely on?
  • Enterprise Architecture Concerns – what agreements have been made with EA to get alignment of the solution to EA standards.
  • Architectural Context – one or more diagrams showing how the solution platform will evolve over time.
  • Methodology – how this document and concensus was created.  What meetings were held.  Who was in the room.  (it matters)
  • Roadmap schedule – what timelines everything thinks are reasonable for delivering the needs to different business customers
  • System Quality Attributes – what quality attributes will be stressed in each of the iterations.
  • Alternatives considered – what alternatives to this roadmap were considered and why this one was chosen.
  • Roadmap Risks – what could go wrong and who is assigned to watch for it
  • Platform Historical Narrative – previous decisions and narrative so that we can always answer the question: how did we ever get in a bind like this?

So if you find yourself in the position of having to create a roadmap, and getting your businesses to sign off on developing one platform well, rather than 4 competing applications with competing data streams and incompatible processes, use the list above.  It is simple, doable, and a good starting point to keep your customers from ‘going off the rails’ and stomping on another’s needs.

 

fun activity for kids: count to 1000 on your fingers

By |2007-03-10T13:37:00+00:00March 10th, 2007|Enterprise Architecture|

Not that it’s fun to count to 1000, but kids will not believe you until you show them not only that you CAN count to 1000 on your fingers, but how to do it. 

(I suspect some of the adults reading this aren’t convinced either). 

I taught this to my 11 year old son last week.  When we started, he didn’t believe me, until I showed him how he could count to 31 on one hand.  He counted to 31 at least four times, to get it in his head exactly how it works.  Then, we added one finger of his other hand, and he made it to 63.  We added two fingers of his left hand, and we made it to 127.  After that, he was convinced.  I chatted about the way to represent numbers with fingers and we talked about how high you can go on two hands (the answer is 1023).  He accepted that it was not only possible, but he wanted to be the master of it to show his friends. 

It’s fun and interesting, and helps introduce kids to basic math concepts that will be useful in middle and high school and, hopefully, into careers in the sciences, technology, engineering, or mathematics. 

How can you count to over 1000 on the fingers of your hands?  I’ll come up with a graphic and post it here in a day or so.  Until then, let’s see if my computing friends have figured it out…

Sharepoint Business Data Catalog and system adapter patterns

By |2007-03-10T00:48:00+00:00March 10th, 2007|Enterprise Architecture|

A pattern is a solution that occurs many times.  You recognize it and describe it and others use it.  The Sharepoint Business Data Catalog is an example of a pattern that I’m looking for in other places, the system adapter pattern.

In any enterprise system, if you want to provide a ‘generic’ mechanism to allow data to be drawn from another system, you have to provide a way to connect the source system to your enterprise app in a manner that allows your app to consistently draw that data out of all source systems.  Of course, that consistency can be tough.  Some systems want you to reach into their database, others have web services, others still have more esoteric communication mechanisms.  They are certainly not consistent.

So you create an adapter interface that allows an adapter to be configured or written.  That adapter will draw the data from the external app and make it available using a consistent interface to your app.

 But what does it mean to “make that data available?’  Why would we need to do this?  One reason is that you need to get these data values from another location because you intend to feed these data values back to that location.  In other words, data from FOO is going to BAR, so get data from BAR to validate against.  But do it generically. Do it in a way that the source of the data is configured, not hard coded, and the number and types of fields are discovered at run time, not coded at compile time.

Sharepoint faced this problem, and they added the Business Data Catalog.  It’s a nice feature and useful for what it is good for: pulling data from a remote source and displaying it in a portal.  It is NOT good for local data cache, because it doesn’t store the remote data.  It just stores the metadata that allows it to be retrieved, but that’s OK for sharepoint.  Two sharepoint users are unlikely to share data.

On the other hand, if you do have a query that most of your users need from a remote system, and it always returns the same rows, you need this pattern plus one more: a local data cache.  That’s another blog.

An Agile Business Planning Model

By |2007-03-08T10:34:00+00:00March 8th, 2007|Enterprise Architecture|

In IT, for years now, we’ve been debating and arguing for changes in the way in which we write software.  Clearly, models where all requirements come before all design, and all design comes before all code, is just not realistic.  That model doesn’t reflect the dynamic nature of modern business.  (if you want to argue with me on this point, get your own blog).

So why are things so slow to change?  I think because many corporate leaders PLAN in a waterfall model.  That makes sense.  It takes time for an organization to react, so their form of change control is to create change slowly.  So strategy meetings only produce big public announcements on an annual basis.  That keeps the organization from thrashing, and keeps them focused on the plan in place.  Of course, in executive planning circles, there are many strategies that have been suggested… not all of them are announced.  Why is that?

For the same reason as you would have all your requirements in a backlog, but you only release to the development team the subset of requirements that they can accomplish in a single sprint.  Stability.  The dev team appreciates the control they have by taking on only a subset of requirements for each sprint.  In return, they demonstrate value on every sprint. 

Unfortunately, executive planning is pretty conservative when it comes to this.  Rather than letting the organization see the entire queue, and letting them decide what to take on, planning committees usually just pick the requirements that “should” get done, and then dump them on the organization and say “make it so.”  They have an arbitrary date (one year hence) to make it happen.

Look at that for a moment.  Seem familiar?

Requirements from above.  Arbitrary deadlines.  The team doesn’t get to see the entire list of requirements.  They are told what to do, when to do it, and given a fixed budget to do it with. 

Sounds like waterfall software development.

Looks something like this.

Waterfall Planning Model

 

Unfortunately, this idea is just as ineffective in business as it is in IT.  It is time that we, in IT, take our ideas of Agility, and share them with our friends in the business side.  This is happening in some very agile corporations, including Microsoft, but not in enough places.  We, in IT, can help share this notion. 

The real key to making this work is the Prioritized Queue.  Just as an agile model allows developers to see the most important things that need to be done, and THEY can choose what they will deliver in a cycle or sprint, this model needs to work itself all the way up the chain.  Executive Management doesn’t have to participate.  They probably won’t.  But everyone else should, especially as it comes to planning for changes in the IT side.

The think to keep in mind about this model is this: At each level, work is iterative and continuous.  EACH LEVEL IS A SPRINT TEAM.

At each level, the work is continuous.  Development is continuous.  Requirments Gathering is Continuous.  Planning is Continuous.

Here’s the new model.  Click the image to see it full size.

Thumbnail image of agile process

 

One big advantage for us: if we get to this model as a normal form of business, then a lot of the “waterfall pain” we feel in IT goes away.  Something to think about.