I’ve been asked a number of times over the years if I can explain the theory of Enterprise Architecture. I decided recently to reopen that idea. It’s not a new discussion. I refer to Tom Graves post on the Theory of EA from 2012 where he posits that the theory of EA, if one were to be described, cannot be used to prove the value of an EA design. The not-so-subtle hint is that there is, therefore, no value in creating a theory at all. I disagree. I believe that there is value in developing a theory of enterprise architecture.
Let me first recap my gentle readers on what I mean by a “theory of EA.”
Typically, in science, we start with observations. Observations are objectively real. They should be mathematical and measurable. They exist within a natural setting and may vary from one setting to another. We may observe a relationship between those observations. The goal is to explain those observations in a manner that is good enough to predict outcomes. To do this, we suggest a hypothesis, which is simply a guess. We then see if we can prove or disprove the hypothesis using data. If we cannot disprove the hypothesis, we have passed our first test. We use the hypothesis to predict new observations. We then check to see if the predictions are correct. If so, we have a useful theory.
Underlying Observations
Theories are created to explain observations and help predict new ones. So what kinds of observations would I include in the Theory of Enterprise Architecture?
- The rate at which companies can adapt to change varies widely from company to company. Let’s call this the “rate of potential change” (RPC) because it refers not to the actual rate of change, but the potential rate of change which will never be less than the actual rate, but may in fact be more.
- This rate is important to the survival and health of a company. Companies can die very quickly when their marketplace is “shocked” by a big change in customer expectations or competitive offerings. If the Rate of Potential Change (RPC) is high enough, then any shock to the marketplace can be absorbed by an enterprise by responding competitively. The cost of response appears to increase exponentially as time from the shock increases. For example, from the date Amazon announced their cloud platform to the date that Microsoft produced a product that was as good as the Amazon initial offering, the time that elapsed created a steep obstacle for Microsoft to overcome. The cost of overcoming that obstacle is much higher than if Microsoft had been able to respond sooner. The faster you can respond, the more chance you have of survival. RPC measures how fast you can respond.
- The Rate of Potential Change (RPC) appears to be correlated with observable factors like the amount of alignment between strategy and execution, the quality and testability of company strategies, and the measurable maturity of key capabilities for absorbing and coping with change.
These observations need to be measured, collected, and validated. And we need more observations to be researched, shared, and enumerated. We don’t know quite what EA explains just yet, and building out the list of observations gives us a place to start.
The EA Hypothesis
At the highest level, the basic premise of Enterprise Architecture is simple:
The EA Hypothesis: The structure of and both intentional and unintentional relationships among enterprise systems has a direct and measurable influence on the rate of potential change and organizational cost of operating and maintaining those systems.
That simple statement is quite powerful.
The EA hypothesis demands that we create a definition for “enterprise system” and a method for describing the “structure” of an enterprise with respect to those systems and to describe the “relationships” between them. Clearly an enterprise system has to include socio cultural systems, information technology systems, workflow systems, and governance systems.
The EA hypothesis suggests that the relationships between these systems are important. That the relationships themselves influence the rate of potential change. as well as the cost to own a system.
The EA hypothesis demands that we measure the rate of potential change, and that we describe “organizational cost.” To do the latter, we must develop a clear idea of what is involved in operating and maintaining each of the included systems.
The hypothesis is also fairly unbounded. It leaves us with important questions to answer.
- Can we cleanly and concisely define what we mean by “system” so that two architects independently examining the same enterprise would develop the same list of systems?
- What are the types of relationships among systems and how do we differentiate relationship? What attributes do these relationships have? What attributes make sense?
- Does it apply to one system? A subset of systems? or can it only be truly understood to apply to the complete system-of-systems that is, in effect, a complete description of the enterprise?
- What standard methods can we develop for identifying ALL of the relevant systems of an enterprise quickly and effectively for the sake of understanding the architecture of the enterprise?
I’m intentionally not answering these questions here because it is rational to leave all of these questions open for scientific research. It is entirely possible that the answers may help us separate useful EA models from useless ones. It is simply too soon to tell.
Why the EA Hypothesis matters
The rationale for creating an EA hypothesis is the requirement, often expressed through strategy, placed on an enterprise by its senior leaders, to do one of two things:
- improve the quality and reduce the organizational cost* of performing existing enterprise capabilities, or
- creating or expanding capabilities in an enterprise through targeted, specific and managed changes to the network of systems
This matters because nearly all strategy hits one of these two buckets. This goes from corporate strategy all the way down to personal improvement: either you are improving your production, or your production capacity. Either you doing what you know how to do, or you are learning new things. Either you are getting better at the normal stuff, or innovating to add new stuff.
Enterprise architects are called upon to help in both ways. We have to answer questions like: “what does “innovation X” do for us, and what does it do to us?” We also have to contribute to ongoing concerns like “how do I grow my business in “Market Segment Y” in an innovative and compelling way?” and “How do I cut the cost of our IT expenditures?” and “How do I improve the quality of my customer data?” These questions fall under the category of “organizational cost”.
* Cost and quality come together to include a balance of monetary cost, effectiveness, customer satisfaction, efficiency, speed, security, reliability, and many other system quality attributes.
We need a clear theory of Enterprise Architecture because answering these questions is difficult to do well. We have operated without a theory because we were able to “guess and check.” We would guess an the scope and value
of an initiative, undertake it, and check on its value later. But we are not able to say, in advance, that “proposed initiative A” is foolish compared to “proposed initiative B” because we have no science here. It’s all just “guess and check.”
The term “guess and check” is not new. My kids learned to use the “guess and check” method in elementary school math class as a way of exploring a problem. But that’s where the “guess and check” method belongs. Elementary school. Grown ups use proven science.
All except EA. We still use “guess and check.” It’s time to grow up.
Next steps
- First off, we need a long list of valid observations that we are trying to explain and understand. The naturalists of a hundred years ago started with detailed drawings and descriptions of plants and animals and the habitats that they inhabit. Perhaps we should start with detailed drawings and descriptions of the structure of different enterprises and the niches that they operate in. We also need a valid way to measure and observe the “Rate of Potential Change” in an organization.
- Secondly, we need simple reusable methods for conducting research in the area. A consistent way to count and categorize systems, for example, and an accepted methodology for measuring their cost and quality that can be applied across different types of systems and companies.
- Lastly, we need evidence of the cause and effect of making changes. We need a solidly understood and measured system to be captured in a snapshot, and then a series of changes results in another solidly understood and measured system. That gives us evidence of the value of the changes.
Moving forward from here requires research. More on that connection in another blog entry.
Nick,
…The EA hypothesis demands that we create a definition for “enterprise system” and a method for describing the “structure” of an enterprise with respect to those systems and to describe the “relationships” between them. Clearly an enterprise system has to include socio cultural systems, information technology systems, workflow systems, and governance systems. …
Why do say structure instead of architecture in this sentence? And why is there no mention in your article of the role concepts and principles play in architecture in order to move towards a theory of EA?
Hello Mark,
Good questions. One at a time.
1. Why did I say "structure" instead of "architecture" in the cited sentence? — because the word "architecture" as so many meanings that, in that sentence, my intention is vague. I want to be clear that I'm referring to the composition of systems, which is part of, but not all of, the general discussion of architecture. Composing a system of systems is certainly a critical element. The resulting composition is useful. Both the act of composing and the resulting composition are referred to as "architecture."
2. Why is there no mention of the role that concepts and principles play in order to move towards a theory of EA? — If you follow my logic, those bits are "supporting" elements in EA, and not core to the theory itself. In other words, we can get to a fully functional THEORY of EA without actually using a single principle or modeling a single concept.
I'd separate "theory" from "practice" here. The Theory explains WHY Enterprise Architecture, as a field, can have the effect we think it should. It becomes the rationale for methods. The Practice explains HOW to use those methods, and some of those methods involve principles and concepts.
While we do not need principles to develop the theory, we have found that actually developing an INSTANCE of an enterprise architecture, and then PUTTING IT INTO USE, does require principles. It also requires an organization to develop a conceptual model of their business (for developers: an enterprise domain model).
The act of TRAINING a person to become an Enterprise Architect requires the development of a body of knowledge with field-specific concepts as well.
Thanks for the mention, Nick!
I've now replied to this on my own blog: see post 'Theory and metatheory in enterprise-architecture', weblog.tetradian.com/…/theory-and-metatheory-in-entarch . I hope it makes sense/is-useful?
The key point: yes, I strongly agree that we need a far better understanding and use of theory within enterprise-architecture. We do differ somewhat in how we'd approach that concern, but that it is a valid and urgent concern for enterprise-architecture is a point on which we do most definitely agree.
Discuss further, perhaps? 🙂
Good article, Nick. Your observations are very much in line with my feeling that EA's impact should be measurable in the velocity and quality of the decisions that management makes with regard to executing and/or modifying strategy.
I also agree wholeheartedly that we need some agreed-upon metrics to gauge and assess the EA practices at companies in order to try to assign some sort of value to them. As I have observed in various LinkedIn discussions, attributing value to EA is fraught with complications, not least of which is the need to assess alternatives to what could have or would have happened in a particular case if EA had or hadn't been applied.
Hi Tom,
I read through your rather lengthy response. I really appreciate the time you spent to think it through.
I would like to discuss further, perhaps in a Google Hangout or some such. I think the approach you are taking, of considering multiple theories for different kinds of problems (wickedness) is valid in the theoretical. However, the field if EA needs only know two things: a function that determines if a problem is statistically likely to be wicked, and a theory for how to handle problems that are not.
With a simple tool like this, we can address the vast majority of problems that EA faces. Not from a "surface" standpoint (there are millions of problems, all over the map), but from a quantity standpoint (of those millions, 90+% are probably not sufficiently wicked to invalidate the theory).
Let's remember that Darwin's theory of evolution by natural selection is both right and wrong. It was "directionally correct" and it took sophisticated understanding to both prove it to be inaccurate and to correct those inaccuracies. It is, however, right, in that most problems in evolution CAN be explained rationally using Darwin's theory. In that sense, the "wicked" problem applies to his theory as well.
What I'd like to suggest is that MANY areas of system observational theoretical behavior have this problem: we have a "wickedness" problem, but we don't know where the line is. So we have to create a reasonable useful theory and go from there.
If we proceed with validating the theory proposed above (assuming it is corrected for the oversight of not including "management" as an input) and improving it until we can find that boundary, we may find value in EA that covers a large percentage of those problems. Only then can we find the "wickedness" line.
Thoughts?
"Grown ups use proven science.", but wicked problem solvers/innovators don't!
You make it all sound like a math problem where every problem will have a proven best solution. IMHO EA is by definition wicked because of the multiple viewpoints/mental maps of the people incorporated.
To really change and improve things and innovate stuff you need to question the status quo instead of establishing the status quo like EA is doing. This is well described in the book A More Beautiful Question which every EA should read. After that read The Heretic’s Guide to Best Practices. That's all theory you need to approach wicked problems.
The Heretic’s Guide to Best Practices also predicts with great accuracy what kind of answers I can expect 🙂
BTW:
– there was no EA needed to develop the internet
– there was no EA needed to put a man on the moon
– there was no EA involved when the shipping container reshaped the world of transportation
– there was no EA involved when Steve Jobs changed the shape of a few industries
– Frank Gehry didn't used proven methods to architect the Guggenheim museum in Bilbao
etcetera etcetera
Nick, I like very much the scientific approach and in someway the characteristics of being measurable, repeatable and objectively verifiable.
Along these lines, some times I feel EA is trying to come up with the Great Unified Theory just by watching the sky and the stars, with little true observations.
What I mean by that? You mention observations as the starting point, which I agree.
Observations contain happened facts and related measures. When looking to certain enterprise phenomenon, like Change, normally there are problems of definition (what do you really mean by "that fact"), collection (who can "certify" that a fact is really a fact and it is shared/understood as such) and measurement (what has been the measured performance). All these elements are rarely available and even less shared within an organization. In general the problem of certainty and uncertainty as well as common agreement around a basic scientific method is quite relevant in our job.
I am eager to learn what's your approach towards these type of problems when entering into an enterprise? I think sharing in a post more on this, would greatly help many of us in our every day job.
Thanks Nick for your inputs and considerations
Hi Nick
Many thanks for the reply – much-appreciated! (A request: this blog-engine that you're using here doesn't auto-notify of any responses – I only found your reply above after following up that Peter Bakker had said he'd also posted here. If you want me to know if you've posted a reply, please let me know via email, or via Twitter? – many thanks!)
Yes, would love to chat somewhen soon about this – either G+ Hangout (ID: + my name above) or Skype (you have my ID already). I'm in GMT timezone at present, so 8hrs ahead of you, I think?
I'd warn that if the 'fractals' concept of EA is valid – and I think that it is – then by definition almost-exactly 100% of EA context will include wicked-problem elements, So although it's possible that "90+% [of EA contexts] are probably not sufficiently wicked" to require a primary focus on the wicked-elements, it's quite probable that we can _never_ arbitrarily exclude them. The huge danger with a simple 'draw the line' approach is that it incites folks into doing exactly that kind of arbitrary exclusion – with potentially-nasty consequences, as we've all seen all too often.
I do agree – as you'll have seen in my post – that your 'The EA Hypothesis' is a (very) useful hypothesis that we could apply. It is, however, only _one_ amongst many that we could usefully apply in EA. What metatheory is about is as a consistent, disciplined means to identify develop, test and appropriately apply _any_ hypothesis and concomitant methods that we might need for _any_ aspect of our EA work. I'd love to discuss that with you in more depth.
Do keep in touch, anyways – and thanks again.
Hi Nick. Interesting thought piece, but this whole notion seems a bit convoluted to me. The reason I say that is that I have trying for years to get enterprise and business architects to raise their sights and sweep in important concepts, and yes, theory, from a whole variety of existing, and I think relevant, disciplines. I won't go into a long list here, but two stand out because they have "theory" in their names. By that I mean the theory of the firm, starting with Ronald Coase, and general systems theory, starting almost 60 years ago with von Bertalanffy and the organization ISSS, and leading through cybernetics etc. Oh, and there's information theory a la Shannon. And, well, Goldratt's theory of constraints.
By my lights EA/BA are/is some set of practices. Such as ethnomethodology is to anthropology, and applied as well to the enterprise as workplace ethnography. Which I would recommend to study with some of the time otherwise spent in trying to concoct a "theory" of a practice. It seems like a form of navel-gazing, like, say talking about a "theory of engineering". Which, ooops, as I look that up for this comment, actually does exist! Oh well 🙂
Nick:
Very inspirational! Thanks for laying out the EA hypothesis and use the term structure instead of architecture.
I have been trying to work out the structure part for quite a while, and have just made some inroads recently. But I can't stop thinking what is the use of it. The worth of a software tool based on the theoretical treatment of this structure depends, of course, its usefulness to the enterprises. Your post gives me a very much needed affirmation.
The four questions are like the high level requirements that the theory needs to answer and eventually to fulfill. The key seems to me that whether a theory 1) can describe the structure topologically with its primitives and components 2) can capture the relationship between them, 3) can be used to calculate the component costs of various parts of the enterprise system and 4) can be used for change planning and implementation.
However, I don't know this is a theory. It has some similarities to the corporate accounting system, which has a structure and has to be maintained and monitored.
I want to note that Slade Beard also commented on our discussion at his blog:
sladepb.blogspot.com.au/…/a-theory-of-ea.html
So many excellent responses. I'll try to hit on a few. Short ones first.
Howard Wiener — We agree. Thank you.
Doug McDavid — as always, I appreciate the reminder that we fit in a larger ecosystem of management theory and must both recognize that and be relevant to it. I'm hoping that, by simply making it clear that we should seek to formalize the effects of EA, I will be providing some of the bridge needed to make that connection. Otherwise, we are trying to lasso a cloud.
Pierluigi — you asked me to author another blog post on my approach. I'll start one. That said, I keep things in a "working in progress" state for quite a while. Hopefully, my thoughts will coalesce into a good post sometime in the next few months.
Peter Bakker — A broken watch is right twice a day. You list a short list of achievements and assert that EA was not needed. What you did not list were all the achievements where EA was not involved that FAILED. What you've illustrated through that list is a logical fallacy. I never stated that EA is the ONLY way to success any more than a person walking from Los Angeles to New York would be incapable of successfully reaching the destination with no map. I'm simply stating that I believe there to be a positive measurable effect of using EA that should improve the likelihood of success. To the idea that EA is wicked by definition: nonsense. If I can describe any effective and repeatable methods at all, and I can, then the entire of the practice of EA is clearly not wicked. I reject that assertion as disproven by trivia.
Colin Zhao — You hit on a very important point. The methods of EA deliver actual value and we have to be clear that what we are measuring is not the "function" but the value of one method or another. In that context, we can state that there is value to the method regardless of whether we label that method as "belonging" to Enterprise Architecture. In that context, it's not the "theory" that answers your questions, but the method or composition of methods that you've chosen for your effort that answers those questions. That said, we agree with the core case: If we cannot provide specific value, we are wasting our company's resources by showing up for work.
Tom — I saved you for last. I will drop you a tweet right after I hit "post" on this response. I agree that this platform is limited. I use it for it's visibility, not its features.
I'm not convinced that the fractals concept is "universally" valid. I believe that the term "fractal" is not correct when applied to EA. This is because fractals are infinitely applicable at each level. A fractal is a simple mathematical equation that can be applied over and over in a recursive manner. EA is not. EA is different at every level. The differentiation tends to happen as the ability of the "owner" of an area loses accountability for the result as you work you way "down" an organizational hierarchy. At the level of a single team, you can apply an EA method, but you cannot measure it. It is, at that level, "outside" the frame of reference of an EA theory. So clearly there is a floor. Is there a ceiling? To an extent, yes. There are methods that only work at the "system of systems" level, and then there are methods that only work at the "single owner" level. The ceiling is the level where we run out of methods.
In that context… fractal is a useful analogy but it is flawed similar to the way that the city planning analogy is flawed. It is useful for explaining a few concepts and concerns, but breaks down under inspection.
As for the notion that a "wicked sub problem" falls outside the line of an EA theory, and is therefore excluded –> I never said that something falling outside the EA theory implies that it should be excluded. I'd flip that over and state that something falling outside an EA theory challenges us to come up with a better theory or better methods for coping with it. But I'd also state that by elaborating a theory allows us to do something we've not done before: detect which methods are "inside" it and which are "outside" it, with the goal of either improving the methods or revisiting the theory.
Unfortunately, between the time when I wrote most of this post (Christmas break) and today, I've started a new consulting engagement and may not have the time for that Google Hangout. Let's leave that for a future date.
I'd also like to point out an excellent published article by Drs. Hadi Kandjani and Peter Bernus at the Center for EA Research and Management (CEARM) at Griffith University in Australia:
http://www.academia.edu/…/The_Enterprise_Architecture_Body_of_Knowledge_as_an_Evolving_Discipline
On fractals, it again seems to me that you're blurring theory and meta-theory. For theory itself – i.e. instance-of-theory – then yes, you're absolutely right to say that "EA is different at every level" [of the organisation etc], and that in that type of context the 'fractals' concept breaks down very quickly. Yet when we move 'up' to meta-theory – i.e. theory that guides development of context-specific instance-of-theory – the principles around EA are (and must be) the same at every level, and hence there is 'ceiling' or 'floor', by definition.
I also worry a bit that when you describe fractals as "a simple mathematical equation", you're perhaps somewhat confusing the _description_ of specific characteristics of a fractal (i.e. the math) with the fractal-element _itself_ (e.g. relation between twig, branch and trunk of a tree). Both concepts apply in this context, but we need to be very careful not to mix them up.
You somewhat illustrate both of those blurrings-together in your comment about "fractal is a useful analogy but it is flawed similar to the way that the city planning analogy is flawed". The city-planning analogy is theory, not meta-theory (the meta-theory is the descriptor for the _class_ of analogies of which 'city-planning' is just one); and 'fractal' applies not to the instance but to the relationship between theory and meta-theory, and hence to the _outcomes_ of theory versus the outcomes of meta-theory. I know I'm crazily-pedantic about this, but making meaningful sense of fractals and the like in EA requires a level of precision that far too few EA-folks seem to realise is actually needed, and we both need to work together to get this point across wherever we can.
On your comment that "something falling outside an EA theory challenges us to come up with a better theory or better methods for coping with it", I would _very strongly agree_ with that. The catch with your follow-on comment that "elaborating a theory allows us to do something we've not done before: detect which methods are "inside" it and which are "outside" it, with the goal of either improving the methods or revisiting the theory" is that, again by definition, we can't use theory to directly test or validate itself: that way lie circular-reasoning and suchlike, which are Not A Good Idea. Hence why I've been pushing on this distinction between metatheory (which _can_ be used recursively/reflexively/fractally upon itself) and theory (which can't).
On "a new consulting engagement" and follow-ons: first, congrats on the new consulting-engagement! And second, yeah, well, we'll do the Hangout sometime – it'll happen when it happens, not before and probably not after. 🙂 Will look forward to it, anyway.
Thanks again – and perhaps especially for taking me seriously on this. Good luck, and let's keep in touch!
hi ! i can't understand what are stated above. can you please tell me more specific what is EA all about. Please. Thanks 🙂
Hello Jayz,
This blog post, and in fact, this blog, is not a good overall description of the field of Enterprise Architecture. There are good introductory descriptions available in college level textbooks on Enterprise architecture that I recommend to you.
Here's one that comes well recommended to me:
http://www.quill.com/…/50514470.html
— Nick