/Tag: Metrics

Moving Towards a Theory of Enterprise Architecture

By |2015-01-16T13:28:41+00:00January 16th, 2015|Enterprise Architecture|

I’ve been asked a number of times over the years if I can explain the theory of Enterprise Architecture.  I decided recently to reopen that idea.  It’s not a new discussion.  I refer to Tom Graves post on the Theory of EA from 2012 where he posits that the theory of EA, if one were to be described, cannot be used to prove the value of an EA design.  The not-so-subtle hint is that there is, therefore, no value in creating a theory at all.  I disagree.  I believe that there is value in developing a theory of enterprise architecture. 

Let me first recap my gentle readers on what I mean by a “theory of EA.” 

Typically, in science, we start with observations.  Observations are objectively real.  They should be mathematical and measurable.  They exist within a natural setting and may vary from one setting to another.  We may observe a relationship between those observations.  The goal is to explain those observations in a manner that is good enough to predict outcomes.  To do this, we suggest a hypothesis, which is simply a guess.  We then see if we can prove or disprove the hypothesis using data.  If we cannot disprove the hypothesis, we have passed our first test.  We use the hypothesis to predict new observations.  We then check to see if the predictions are correct.  If so, we have a useful theory

Underlying Observations

Theories are created to explain observations and help predict new ones.  So what kinds of observations would I include in the Theory of Enterprise Architecture?

  1. The rate at which companies can adapt to change varies widely from company to company. Let’s call this the “rate of potential change” (RPC) because it refers not to the actual rate of change, but the potential rate of change which will never be less than the actual rate, but may in fact be more.
     
  2. This rate is important to the survival and health of a company.  Companies can die very quickly when their marketplace is “shocked” by a big change in customer expectations or competitive offerings.  If the Rate of Potential Change (RPC) is high enough, then any shock to the marketplace can be absorbed by an enterprise by responding competitively.  The cost of response appears to increase exponentially as time from the shock increases.  For example, from the date Amazon announced their cloud platform to the date that Microsoft produced a product that was as good as the Amazon initial offering, the time that elapsed created a steep obstacle for Microsoft to overcome.  The cost of overcoming that obstacle is much higher than if Microsoft had been able to respond sooner. The faster you can respond, the more chance you have of survival.  RPC measures how fast you can respond.
  3. The Rate of Potential Change (RPC) appears to be correlated with observable factors like the amount of alignment between strategy and execution, the quality and testability of company strategies, and the measurable maturity of key capabilities for absorbing and coping with change.

These observations need to be measured, collected, and validated.  And we need more observations to be researched, shared, and enumerated.  We don’t know quite what EA explains just yet, and building out the list of observations gives us a place to start.

The EA Hypothesis

At the highest level, the basic premise of Enterprise Architecture is simple:

The EA Hypothesis: The structure of and both intentional and unintentional relationships among enterprise systems has a direct and measurable influence on the rate of potential change and organizational cost of operating and maintaining those systems.

That simple statement is quite powerful. 

The EA hypothesis demands that we create a definition for “enterprise system” and a method for describing the “structure” of an enterprise with respect to those systems and to describe the “relationships” between them.  Clearly an enterprise system has to include socio cultural systems, information technology systems, workflow systems, and governance systems.    

The EA hypothesis suggests that the relationships between these systems are important.  That the relationships themselves influence the rate of potential change. as well as the cost to own a system.

The EA hypothesis demands that we measure the rate of potential change, and that we describe “organizational cost.” To do the latter, we must develop a clear idea of what is involved in operating and maintaining each of the included systems. 

The hypothesis is also fairly unbounded.  It leaves us with important questions to answer.

  • Can we cleanly and concisely define what we mean by “system” so that two architects independently examining the same enterprise would develop the same list of systems?
  • What are the types of relationships among systems and how do we differentiate relationship?  What attributes do these relationships have?  What attributes make sense?
  • Does it apply to one system?  A subset of systems? or can it only be truly understood to apply to the complete system-of-systems that is, in effect, a complete description of the enterprise?
  • What standard methods can we develop for identifying ALL of the relevant systems of an enterprise quickly and effectively for the sake of understanding the architecture of the enterprise?

I’m intentionally not answering these questions here because it is rational to leave all of these questions open for scientific research.  It is entirely possible that the answers may help us separate useful EA models from useless ones.  It is simply too soon to tell.

Why the EA Hypothesis matters

The rationale for creating an EA hypothesis is the requirement, often expressed through strategy, placed on an enterprise by its senior leaders, to do one of two things:

  1. improve the quality and reduce the organizational cost* of performing existing enterprise capabilities, or
  2. creating or expanding capabilities in an enterprise through targeted, specific and managed changes to the network of systems

This matters because nearly all strategy hits one of these two buckets.  This goes from corporate strategy all the way down to personal improvement: either you are improving your production, or your production capacity.  Either you doing what you know how to do, or you are learning new things.  Either you are getting better at the normal stuff, or innovating to add new stuff. 

Enterprise architects are called upon to help in both ways.  We have to answer questions like: “what does “innovation X” do for us, and what does it do to us?” We also have to contribute to ongoing concerns like “how do I grow my business in “Market Segment Y” in an innovative and compelling way?” and “How do I cut the cost of our IT expenditures?” and “How do I improve the quality of my customer data?”  These questions fall under the category of “organizational cost”.

* Cost and quality come together to include a balance of monetary cost, effectiveness, customer satisfaction, efficiency, speed, security, reliability, and many other system quality attributes.

We need a clear theory of Enterprise Architecture because answering these questions is difficult to do well. We have operated without a theory because we were able to “guess and check.”  We would guess an the scope and value
of an initiative, undertake it, and check on its value later.  But we are not able to say, in advance, that “proposed initiative A” is foolish compared to “proposed initiative B” because we have no science here.  It’s all just “guess and check.”

The term “guess and check” is not new.  My kids learned to use the “guess and check” method in elementary school math class as a way of exploring a problem.  But that’s where the “guess and check” method belongs.  Elementary school.  Grown ups use proven science.

All except EA.  We still use “guess and check.”  It’s time to grow up.

Next steps

  • First off, we need a long list of valid observations that we are trying to explain and understand.  The naturalists of a hundred years ago started with detailed drawings and descriptions of plants and animals and the habitats that they inhabit.  Perhaps we should start with detailed drawings and descriptions of the structure of different enterprises and the niches that they operate in.  We also need a valid way to measure and observe the “Rate of Potential Change” in an organization.
  • Secondly, we need simple reusable methods for conducting research in the area.  A consistent way to count and categorize systems, for example, and an accepted methodology for measuring their cost and quality that can be applied across different types of systems and companies.
  • Lastly, we need evidence of the cause and effect of making changes.  We need a solidly understood and measured system to be captured in a snapshot, and then a series of changes results in another solidly understood and measured system.  That gives us evidence of the value of the changes. 

 

Moving forward from here requires research. More on that connection in another blog entry.

Rumination on the concept of “best practice”

By |2013-01-28T18:15:13+00:00January 28th, 2013|Enterprise Architecture|

I heard some very interesting talks today from Len Fehskens and Jeff Scott at the Open Group conference.  One thing that I picked up in a meeting yesterday was the notion that TOGAF 9.1 is built on “best practices.”  Today, as Jeff spoke about the transformation of a technical architect into a business architect, and as Len spoke about the challenges of communicating complex ideas, the notion of a “best practice” kept bothering me, and I cross-pollinated my concerns with the concepts that they were sharing.

I agree that the intent of the people who shared their practices with the Open Group was to provide practices that can be taught and followed.  I even agree that the people on the TOGAF committees that accepted the content felt that the practices represented the best that the industry had to offer at the time.  But I wonder if any of the work done in framework committees of any stripe (not to pick on the Open Group) can be held to the standard of being a “best practice.”

Are the practices in the TOGAF framework truly “best” practices?  Are these practices the best ones that the EA field has to offer? 

I guess I would have to follow the EA rabbit hole and ask “what criteria do we use to judge if a practice is the best one?”

After all, when Jeff Scott talks about business architecture using capability modeling, he believes that the practice of capability modeling is the best one to use for the results he is trying to achieve.  (I nearly always agree with Jeff, BTW.  We sometimes differ in language, but nearly never in approach).  That said, as much as Jeff and I agree, our agreement does not mean that the practice should be considered a “best” practice.  Who are we to say?  We are practitioners.  While that is good, it is not enough in my mind to qualify the practice as “best.”

To be a best practice, in my opinion, a method or approach has to meet a higher bar.  There has to be evidence that it is, in fact, better than just a “good practice.” 

I think a best practice should have:

  • Some measurement (evidence) that demonstrates that it is an effective practice, and that the measurement shows that it is at least as effective as other practices,
  • A clear understanding of the results of the practice and the context in which it is to be performed (think “Pattern Language” criteria),
  • Some analysis to show that it meets other criteria like broad applicability and simplicity, and
  • We should demonstrate the ability for that practice to be understood and performed by people who are currently in the role (e.g. can we teach it, and if we teach it, can others do it?).

 

I wonder if we went through most of our frameworks and highlighted the text that is able to meet a higher bar, like the one I describe, how much of the text would we cover?  2%?  10%? 

Is 10% coverage enough to say that a framework is based on best practices?

Time-to-Release – the missing System Quality Attribute

By |2012-03-09T01:26:25+00:00March 9th, 2012|Enterprise Architecture|

I’ve been looking at different ways to implement the ATAM method these past few weeks.  Why?  Because I’m looking at different ways to evaluate software architecture and I’m a fan of the ATAM method pioneered at the Software Engineering Institute at Carnegie Mellon University.  Along the way, I’ve realized that there is a flaw that seems difficult to address. 

Different lists of criteria

The ATAM method is not a difficult thing to understand.  At it’s core, it is quite simple: create a list of “quality attributes” and sort them into order, highest to lowest, for the priority that the business wants.  Get the business stakeholders to sign off.  Then evaluate the ability of the architecture to perform according to that priority.  An architecture that places a high priority on Throughput and a low priority on Robustness may look quite different from an architecture that places a high priority on Robustness and a low priority on Throughput.

So where do we get these lists of attributes?

A couple of years ago, my colleague Gabriel Morgan posted a good article on his blog called “Implementing System Quality Attributes.”  I’ve referred to it from time to time myself, just to get remind myself of a good core set of System Quality Attributes that we could use for evaluating system-level architecture as is required by the ATAM method.  Gabriel got his list of attributes from “Software Requirements” by Karl Wiegers

Of course, there are other possible lists of attributes.  The ISO defined a set of system quality attributes in the standard ISO 25010 and ISO 25012.  They use different terms.  Instead of System Quality Attributes, there are three high level “quality models” each of which present “quality characteristics.”  For each quality characteristic, there are different quality metrics.

Both the list of attributes from Wiegers, and the list of “quality characteristics” from the ISO are missing a key point… “Time to release” (or time to market).

The missing criteria

One of the old sayings from the early days of Microsoft is: “Ship date is a feature of the product.”  The intent of this statement is fairly simple: you can only fit a certain number of features into a product in a specific period of time.  If your time is shorter, the number of features is shorter. 

I’d like to suggest that the need to ship your software on a schedule may be more important than some of the quality attributes as well.  In other words, “time-to-release” needs to be on the list of system quality attributes, prioritized with the other attributes.

How is that quality?

I kind of expect to get flamed for making the suggestion that “time to release” should be on the list, prioritized with the likes of reliability, reusability, portability, and security.  After all, shouldn’t we measure the quality of the product independently of the date on which it ships? 

In a perfect world, perhaps.  But look at the method that ATAM proposes.  The method suggests that we should created a stack-ranked list of quality attributes and get the business to sign off.  In other words, the business has to decide whether “Flexibility” is more, or less, important than “Maintainability.”  Try explaining the difference to your business customer!  I can’t. 

However, if we create a list of attributes and put “Time to Release” on the list, we are empowering the development team in a critical way.  We are empowering them to MISS their deadlines of there is a quality attribute that is higher on the list that needs attention. 

For example: let’s say that your business wants you to implement an eCommerce solution.  In eCommerce, security is very important.  Not only can the credit card companies shut you down if you don’t meet strict PCI compliance requirements, but your reputation can be torpedoed if a hacker gets access to your customer’s credit card data and uses that information for identity theft.  Security matters.  In fact, I’d say that security matters more than “going live” does. 

So your priority may be, in this example:

  • Security,
  • Usability,
  • Time-to-Release,
  • Flexibility,
  • Reliability,
  • Scalability,
  • Performance,
  • Maintainability,
  • Testability, and
  • Interoperability.
     

This means that the business is saying something very specific: “if you cannot get security or usability right, we’d rather you delay the release than ship something that is not secure or not usable.  On the other hand, if the code is not particularly maintainable, we will ship anyway.”

Now, that’s something I can sink my teeth into.  Basically, the “Time to Release” attribute is a dividing line.  Everything above the line is critical to quality.  Everything below the line is good practice.

As an architect sitting in the “reviewer’s chair,” I cannot imagine a more important dividing line than this one.  Not only can I tell if an architecture is any good based on the criteria that rises “above” the line, but I can also argue that the business is taking an unacceptable sacrifice for any attribute that actually falls “below” the line.

So, when you are considering the different ways to stack-rank the quality attributes, consider adding the attribute of “time to release” into the list.  It may offer insight into the mind, and expectations, of your customer and improve your odds of success.

Has the concept of “alignment” entered the infamous “trough of disillusionment?”

By |2011-05-31T03:33:42+00:00May 31st, 2011|Enterprise Architecture|

In the May 15th issue of CIO magazine, there is a rather interesting article, especially for Enterprise Architecture practitioners.  The article, written by Stephanie Overby (a freelance writer), is titled “RIP I.T. Value” and opens with the following paragraph:

The end of IT-business alignment is nigh.  And no one is happier about it than the business focused CIO. 

“If you stand in front of an audience of CIOs and start talking about IT-business alignment, at best you get eye-rolls and at worst, you get people walking out of the room” says Shawn Banerji, a New York based CIO recruiter with Russell Reynolds Associates.  “Alignment has been a big trend for quite some time and—as with most trends—it has gotten a lot of lip service.  But the ability to move beyond that into the practical manifestation of IT and the business being one is a progressive reality for a lot of organizations today.”

Reading down through the article, I’d like to include some interesting excerpts:

Excerpt 1:

According to Gartner, by 2015, the primary factor determining incentive compensation for the CIO will be the amount of new revenue generated from IT initiatives.

Excerpt 2:

CIOs used to measure their personal value by budget or headcount and their team’s value by delivering on time and on budget.  “That’s an arcane approach to measure your relevance,” Banerji says.  “Historically, they’ve talked about alignment because that’s all they could possibly hope for.” Today, CIOs are more likely to be rewarded for overall business performance than for some technology project or initiative, Banerji says. 

Excerpt 3:

CIOs further along in their efforts to measure business impact are putting IT-centric measures like uptime and systems availability on the back burner to focus on business goals like customer retention and operating efficiency. 

To be fair to the author, the article cites some fairly well-versed individuals.  In addition to the recruiter (Banerji), it also references current CIOs (Louie Ehrlich, CIO of Chevron, and Brian Hardee, CIO of Oxford Industries), a Gartner VP (Dave Aron), and a well-placed attorney (Edward Hansen, partner, Baker and McKenzie).  I do not fault the author.  However, I wonder if we are throwing the baby out with the bathwater.

In most businesses, IT is not a front-office endeavor.   Clearly, deep information integration is blurring that line, but only partly.  If IT moves to be revenue focused, I am concerned that IT can lose sight of 80% of its’ value: keeping the lights on for the parts of the business that traditionally make money. 

Now, also to be fair, the quotes from the cited CIOs reflect a focus on changing the measurement structure, not ignoring existing business.  To whit, one very interesting example at Chevron where the old “99.9% uptime” metric has been augmented with a “business incident index” that tracks the number of IT service issues that affected business and the impact of each in terms of revenue and reputation.  This is healthy, and I applaud this kind of metric.  I’m simply concerned that we keep focus on many of the existing metrics as well, and not change CIO compensation to focus on new revenue to their exclusion.

Interesting, also, is the fact that the author never mentions Enterprise Architecture.  Clearly, we need to be reaching out much further than we are.

What do you have to say, fellow architects?  Is the trend towards “IT as business value provider” healthy?  Are there risks here?  What should we do to mitigate them?  What is the opinion of the EA community in this development?

Systematic Simplification

By |2009-03-23T11:45:34+00:00March 23rd, 2009|Enterprise Architecture|

When you want to find an answer to a problem, it is critical that you ask the right question.  This bit of advice is playing out in my life, yet again.

Every couple of months, I bump up against one of the core reasons for Enterprise Architecture to exist: to deliver a simpler, more agile, less expensive IT ecosystem.

And there’s the rub.  EA doesn’t deliver that ecosystem by itself.  EA is part of a larger system of business processes, some of which fund projects, some which generate software changes, and some which monitor systems and respond to incidents.  

So how can we effect this change?  That requires a systemic view of the problem we are trying to address.  To describe my approach, I’m sharing “Nick’s opinion” of the mission and vision needed to deliver on this goal.

The Simplified IT Vision: a simple ecosystem, with information that can be used where it lay, without the need for expensive processes to move it, translate it, and understand it.  Information that can be reached simply, on an infrastructure that is reliable and resilient to the peaks and valleys of normal, and even extraordinary, situations.  Support for the business that is more reliable than the business itself.

The Simplified IT mission: to manage the IT ecosystem, and the demands of the business on it, so that needs are quickly met, while the ecosystem itself becomes progressively simpler, more balanced and easier to manage.

To address these issues, appropriate goals must be set to:

  • understand the system as it is,
  • understand the factors that are driving change to it,
  • remove the sources of complexity from business processes,
  • insert the discipline of simplicity into the business of IT,
  • track progress, report results, and systematically improve.

When creating a balanced scorecard for IT, these are some of the things that need to be understood, measured, and delivered through the practice of Enterprise Architecture.

In effect, the question is this:

How can we systematically grow a simple and well balanced information technology infrastructure?

Key words:

  • “we” – As an Enterprise Architect, I take responsibility for my part, and am willing to be accountable for results.
     
  • “systematically” – The effect I seek must be repeatable and scalable, built in to the processes and culture of the organization
     
  • “grow” – I recognize that simplicity takes time and attention.  It does not come over night.  Simplicity must be nurtured and only through a long-term commitment to delivering it, can a simple ecosystem emerge.
     
  • “simple and well balanced” – Simplicity for the sake of itself is not the goal.  The needs of IT to own the infrastructure must be balanced with the needs of business to be able to rapidly change it.  If there is complexity in technology, let that complexity fall to the people best able to make technical decisions. 
     
  • “IT infrastructure” – All the talk of agile development and SOA and patterns and frameworks don’t amount to a hill of beans if, at the end, we cannot reap the benefits of what we have sown in the form of a sustaining system that is able to meet the needs of end users and business stakeholders.  We cannot stop at measuring code quality or time-to-market.  Those are good measures, but we need to go further.  We must  measure the entire system of systems: we must measure the complexity, resiliency, and reliability of IT as a whole.

Let this be the key question for Enterprise Architecture everywhere.  Let’s focus on the thing that matters and let the other stuff die out.

Alignment has effects.  What questions should you ask of yourself to insure you are aligned to a goal like this?  Here’s a list of questions to consider?

  • If you are doing a project in EA, and you cannot see how it aligns to answering that question, ask yourself: are we spending time on the right thing? 
  • Are your processes optimized to root out complexity and drive for simplicity?
  • Are you collecting and managing information that can support the difficult decisions needed to counteract years of “not looking” at complexity as the core issue?
  • Can you describe the system as it stands and describe your part in improving it, using actual measures? 
  • Are your standards worded in such a way that any system that meets or exceeds standards is a system that supports this goal?

That is the power of having a goal and describing it for everyone to see.  You can push towards a vision, measure progress, and perform the necessary work to make it real.