//January

What about a Software Development Guild?

By |2007-01-31T09:00:00+00:00January 31st, 2007|Enterprise Architecture|

I work for Microsoft.  However, I wonder if the answer to deciding if a developer is ‘qualified’ wouldn’t be better decided outside these hallowed halls.  Specifically, should software development be self-regulating, like Doctors and Attorneys are?

This discussion came up on a Comp.Object newsgroup thread that started by asking about the Big Ball of Mud (BBoM) antipattern.  The discussion quickly descended into ‘whose fault’ it was that BBoM code had come into existence.  Both Robert Martin and myself took the stand that developers who write this code are delivering code more slowly, in the current sprint, then if they wrote well structured code.  E.G. “Quick and Dirty” is an oxymoron.

From there, the cnversation further evolved into: perhaps we should have a self-policing guild, like crafthalls of old, that would allow us to decide, for ourselves, who is qualified to carry the title “Software Engineer.”

Some folks immediately worried about politics and exclusivity, while others worried about creating a hurdle that the truly gifted among us would make no attempt at (no need to).  Plenty of issues.

My take: I’d like to see a more specific proposal for what a guild would entail.  I’m cautious but not opposed to the idea of Software Developers kicking out one of our own for delivering cr*p on a tight schedule.

What do you think?  Should we consider such a thing?

Gently pushing

By |2007-01-30T12:01:00+00:00January 30th, 2007|Enterprise Architecture|

In think that one of the most valuable traits of an enterprise architect is the ability to push gently.  In other words, if you find that a team is developing a solution that cannot be integrated or creates an entirely new definition for the word, customer, you recognize that this is a negative for the business, but you don’t wave your arms and scream… you push gently, but firmly, on the people who made that decision to ‘unmake’ it within the scope of the planned iterations.

You can go live with a wrong thing if it is on the plan to go live with the right thing in the next release. 

Systems Architecture Interview Questions

By |2007-01-29T04:09:00+00:00January 29th, 2007|Enterprise Architecture|

Next week, I’m interviewing another architect, so I’ve gone over my list of “things to ask an architect candidate” for another time to see if I’d change anything.  Not much popped out, but I thought I’d share some exercises that I ask architecture candidates to run through. 

Note: I care more about experience than book learning, but I care very much that you know the actual names for things.   In other words, book learning does matter.  If you never got around to reading any of those ‘patterns’ books, don’t bother to show up.  You wouldn’t be considered a qualified medical intern if you referred to the Biceps Brachii as the ‘upper arm muscle’ and I won’t consider you an architect if you can’t tell that I’m using a Strategy pattern in my case study.

The things that a systems architect must do to survive an interview with me:

  • Demonstrate the ability to work on a project team with developers productively
  • Describe a time when you created a vision for a systems architecture, communicate it, and others made it come into existence
  • Tell me how you know when it is time to review code for compliance and when you are better off getting out of a developer’s hair
  • Be able to guide and direct the team in the art and science of visual modeling (it’s not enough to know… you must also teach).
  • Detect the gaps in the early design artifacts of a project and determine what risks are going unmanaged.
  • Be able to convince me that you understand, in your bones, the concepts of coupling, abstraction, and encapsulation. 
  • Be able to describe at least five GoF patterns in fine detail.  Be able to describe at least five messaging patterns in fine detail.
  • Pick a single system quality attribute that should be paramount in a particular situation, and then explain why.

Some exercises to run folks through:

A. James is an IT architect working with a team of 5 developers and 5 testers on a new system.  He has a situation with his development team.  He presented the high-level design for their new system at a dev meeting and the team listened politely.  Then, after reviewing his diagrams, some of the members of the dev team concluded that he has created a great deal of complexity in one area that they think is overkill.  James hears about it from the project manager three days later.  He feels pretty strongly that the complexity is called for.  Evaluate the situation.  What dynamics are at play?  James comes to you for advice: what do you tell him?

B. Let’s say that you have 10 systems in an infrastructure.  One of them provides the list of all customers, while another provides the list of all orders.  You want to keep the systems as decoupled as possible but they need to share data in real time.  What mechanisms do you put in place to keep the systems independent, yet fully integrated?  Describe the steps you would follow to replace the system that holds orders.

C. Draw the architecture of one of your most complex systems on the white board.  Now:

  • What made you partition the system in this way?
  • What data entities are mastered by your system?  Which components are responsible for actually writing the key data about each entity?
  • What changes do you predict the business will want in the next 2-5 years in this system?  How will the architecture cope with those changes?
  • What system quality attributes did you consider most paramount when designing this system?  What attributes did you sacrifice?

D. The architectural council has invited you to join.  They perform periodic reviews on major projects.  You attend and a project comes before the council that has not been reviewed before.  It is a medium-sized project for your company (your company will perform between three and six projects of this size each year on average).  It consumes data from other systems in real time, creates transactions in other systems, and produces data for reporting.  The following artifacts are provided.  What additional information do you ask for?  What risks are you concerned about? 

  • Logical Data Model showing every data entity in the system’s database
  • Deployment Diagram
  • Class-level architectural diagram illustrating the use of design patterns

E. You review the code of a junior developer, and you see a call where the developer is passing a structure.  The structure contains 25 different values as parameters to the method being called.  The structure is used only to pass parameters to this one method call.  You ask why the call needs so many parameters, and it turns out that it is used in about eight different places in the code, and each one has a slightly different need, so the routine will look to see what parameters are passed to decide what to do.  The code for the method is complex, and contains many examples of code like this:

   if (paramx != null)
         do_something_specific(paramx);

What patterns do you discuss with your junior developer?  What options do you consider?  Do you advise the developer to change the code?  Why?

F.  Pick one of the following practice areas and describe the ideas and concerns that led to this area of software development practice, what the practice entails, the benefits that can be achieved, and the negative consequences that may be observed by teams engaged in the practice.  Note: I may pick one for you… be prepared to be asked about any of them. (If you were to ask: “why this particular list?” I’d turn that around and ask you why I picked this list… )

  • Service Oriented Architecture
  • Test Driven Development
  • Aspect Oriented Programming
  • Dependency Injection / Inversion of Control
  • Software Factories

G. Mary is a fellow architect with a problem and she has come to you for advice.  She is producing an impact analysis on a system we will call “Charlie”.  The Charlie system gets an hourly feed of data from another system called “Bravo.”  The Bravo system is scheduled to be replaced by an altogether new system called “Romeo” that will organize the data in a completely different way.  Unfortunately, this will radically affect the data feed from Bravo to Charlie.  Not only will it vanish, but any feed from Romeo will be structured and organized in a completely different manner than it was before.  Mary needs to develop a roadmap to allow this change to occur.  What advice do you give her?

What do you want said at your funeral?

By |2007-01-29T03:27:00+00:00January 29th, 2007|Enterprise Architecture|

An old saying goes: on their death bed, no one ever turns to their family and says “I wish I had spent more time at work.” 

I’m not waiting that long.  In my life, two experiences combined, and I’m watching them play out.

First: last May, my father became suddenly ill and, within two weeks, he passed away.  I spoke at his memorial service. 

Second: just before my father became ill, I was rereading the Seven Habits book by Covey (for the third time).

One thing that Covey said in his book: What will your family and friends say about you at your funeral? 

Just after reading that book, I had the opportunity to practice it… I looked back at my father’s life and spoke at his funeral.  I spoke of a loving father, a wonderful teacher, and a man who lived until the day he died.  In the last year of his life, he traveled to London, Paris, New Delhi, and Tokyo.  He climbed the stairs at the Notre Dame cathedral.  He lit fireworks with his brother and nieces and their children at Diwali.  He painted paintings and had art shows and hosted lively parties where lively people would come dance in the great dancing room he had converted from a two-car garage, just as he had done ever since I was a boy.

I told this to his friends.  They already knew it.  I said it anyway.  I needed to.

Then we flew home, and things started to change.  I encouraged my wife to finally jump in, quit the job she wasn’t enjoying, and go back to college for that degree she’d always wanted.  (She made the Dean’s list in her first full quarter back in college in over a decade.  I’m so proud of her).  I stopped spending 60 hours a week at work.  I started asking myself “how much less can I do at work” and “how many more minutes can I spend today with my family.” When someone at work would offer up a ‘highly visible’ assignment that was outside my normal duties, I would think twice before taking it.

A few years ago, I pulled back from the traveling that consultants do.  I didn’t enjoy it.  This was the next step: truly trying to find a balance between work and life.  I give my all to my employer during the day, and I work hard, but when the day is over, I come home.  I spend time with my kids… face time.  We talk.  I hug.  I listen. 

If I compare the last week of January 2006 to the last week of January 2007… just pull out that one week from each year and compare them, I can see a change.  I’ve spent more time with each child this year.  I’ve spent a LOT more time with my wife this year.  I’ve spent as much productive time at work… but the unproductive time is disappearing.  I’m squeezing it out.  The overtime caused by never saying “no” is drying up as well. 

My priorities have clearly shifted. 

What I want others to say at my funeral: good father, good husband, good friend, good human being.  The direction I was going wasn’t going to get me there.  This change, I think, is one for the better.

Last year, in this week, my greatest achievement was to make architectural diagrams.

This year, in this week, it’s a three-way toss up: I supported my wife in her studies, took my daughter horseback riding, and took a fencing lesson with my sons.  Oh, yeah, I also worked on architectural diagrams.

What’s your biggest achievement this week?  When all your weeks are done, and your son or daughter stands in front of your friends and speaks about your life, what words will be spoken?  The memories they share then are the memories you build today.

To my father: thank you.  I have thousands of memories of sunsets, swimming, mountain trails, parks, movies, trips, years of breakfast with three cups of fruit juices, fireworks, loud dancing music, meditation, more fireworks, roses, paintings, sculptures, bowls of fruits and nuts, and so much more.  You gave me more than the world.  You gave me you. 

Now it’s my turn.

Case study: create and use Platform Goals to reduce churn

By |2007-01-27T09:59:00+00:00January 27th, 2007|Enterprise Architecture|

If you find yourself in the unenviable position of having to prove to someone that your project is being managed correctly, look to your ‘platform goals’.  Don’t have ‘platform goals?’  Don’t know what they are?  You are not alone.

Platform goals are the statements of principle that describe How and Why you are building your project the way that you are… they are independent of “what” the project is (the project’s functional and technical requirements) but are not independent of the business requirements.

For example: if you have always believed that it is important to deliver value, even if you cannot be perfect in the first iteration, and you run your projects that way, you have created a platform goal.  You need to communicate it.

Also, if your project is building a system that only one team will use, for now, but you are building it for other teams to use, in the future, then you need to gather requirements in a different way, and take on different costs, than if you are building just for one customer.  You need a platform goal to describe this.

Working with some team mates of mine, we came up with some platform goals for the project I’m currently attached to.  The reason was to make sure that everyone saw not only What we were building, but How we were building it, and Why we were building it in this way.  We need to be public about this. 

For the sake of the project, I changed the names and processes.  In this blog post, we are building a system that allows the procurement department pick truck tires for the Contoso Trucking and Transportation Company.

Platform Goals for the Contoso Procurement Platform (CPP):

  • Enterprise: Designed to enable all procurement decisions for the enterprise, starting with truck tires
  •  Agile: Designed for iterative releases over time, with first release occurring in February of 2007
  • Focused: Scope limited to the generation and acceptance of procurement agreements that can be executed by downstream systems
  •  Inexpensive: Minimize cost to operate on a per-agreement basis (process, operating, and support overhead)
  •  Adaptable: Designed to cope with a radically changing IT landscape with respect to upstream and downstream systems, including the new asset management and financial tracking systems.
  •  Integrated: Intended to provide shared data and functional services to other systems that need similar capabilities
  • Strategic: Intended to allow and enable the decommissioning of one or more legacy systems including the “general procurement desk (GPD)” system.
  •  Empowering: Able to readily deliver the needs of new procurement efforts with minimal refactoring, redevelopment cost, or IT resources

By describing the goals in this way, we can counter criticism that may say: why didn’t you collect 100% of the enterprise requirements up front (which in some corporations is an invitation to analysis paralysis) or why doesn’t this system also handle shipping and receiving (which in most places is an invitation to permanent scope creep).

There are a lot of good ideas about the right way to run a project.  This statement says “These are the ideas we are using.”  It says that if you want to challenge the project, you have to first challege this set of ideas. 

That is harder to do. 

As such, you may just get away with reducing some of the churn that happens when projects are inspected, reorganized, and, in some cases, killed, not because the project is failing to deliver or delivering an unusable system, but because someone disagrees with the processes you use, or has a political bone to pick with your manager. 

Inspection is healthy.  Churning is not. 

IT Parable: It's 10 o'clock… do you know where your requirements are?

By |2007-01-25T08:56:00+00:00January 25th, 2007|Enterprise Architecture|

Joshua and Frank were having a chat over coffee the other day.  Well, Frank was having a chat.  Joshua was mostly listening.

“That guy Alexei is driving me completely nuts!  If he wasn’t so darn smart, I’d just ignore him, but I can’t. The business loves him.  And he walks around taking pot-shots at all the projects… You can’t do it this way!  It won’t work for the business!  He didn’t write the requirements.  Heck… he works for IT, what right does he have second-guessing the requirements on a system that he’s not even part of!”

Joshua just sat quietly.  Frank was a great development lead.  Smart.  Careful.  Methodical.  He’d led many teams to using great techniques for understanding and supporting their code.  His teams delivered working systems on time more often than anyone else… but he had never worked with Alexei before.  That was new.

After a few minutes, Joshua stopped Frank’s tirade and asked one question:

“Frank, what does the word “requirement” mean to you?” 

Frank didn’t really think about it.  He shot off on another march about how the business must be the source of requirements, not a smart wild-card guy running around the IT department.  Joshua stopped him.

“Frank, what does the word “requirement” mean to you?”

The repetition drove home the fact that Frank hadn’t answered the question.

“I guess, it is part of a description of ‘what a system needs to do’ in order to be successful,” Frank finally replied

Joshua looked at Frank for a minute, quiet.  Waiting.  Frank started to think he had said something wrong…

“I mean, it’s what the business wants the resulting system to do,” he added, a little less certain this time.

Joshua stopped him.  “I think you were more right the first time,” he said.  It is not about what one person or a team of people want.  It is about what they need.  Would you agree?”

“Sure!”  Frank had stopped venting.  Now, he was looking curious.

“OK, so what defines “need” in this context.  The business needs a new user interface to their system for managing customer orders.  That’s what you are working on, right?  OK.  So they need a new interface, but that’s not all they need, is it?  Do they need more than a pretty set of screens?  Do they need it to work?”

“Obviously,” Frank replied. 

“So what does it mean, ‘to work.’ Over the years, various IT systems have come up for handling types of orders and types of customer requests.  Some decisions were good, some were not.  Regardless, we are here.  The logic for deciding what orders would be fulfilled by the ERP system and what orders would be fulfilled by one of the side systems are complex.  Would you agree?”

“Yes.  Of course.  And we already know that.  Tom, on my team, has worked on order fulfillment for three years, so he’s been able to provide very valuable insights into how we meet those requirements.”  Frank wasn’t quite sure where Joshua was going with this.

“I want you to consider Alexei to be a kind of ‘additional Tom’ who isn’t officially on your team, but can offer equally valuable insights into the requirements,” Joshua said, and then paused for a second to let Frank absorb it. 

“But Joshua, he’s a crazy man.  He drives me nuts.  He says rude things, and says that the code we are writing is terrible.  I can’t stand him.”  Frank was clearly at the end of his rope.

“Frank,” Joshua started, but stopped… not sure what to say exactly.

“Look, Frank, he’s got some very rough edges.  This I know, but consider this.  He’s worked on nearly every system in this division.  He knows where business rules lie that no one else knows about.  Not even the business.  Don’t look shocked.  He’s been in this division of IT for over fifteen years.  How many members of the business side have been with the company, let alone the division, for that long?” 

Frank thought about it for a moment.  “No one that I can think of.”

Joshua continued. “If you go from a manual system to an automated one, it makes sense to get all of your requirements from the business.  They know them.  And if you go from a simple system that just records things, CRUD style, to a more complex one, the business is still a great place for all of your requirements.  But if you have five mature systems that automate hundreds, or even thousands of finely-tuned business rules, and you want to rewrite one of them, you need to know the rules not only for the current system, but each of the other ones in your space.”

“The problem is, no one in the business remembers them.  No one in the business has any idea of how all the rules work together, or how transactions move from place to place.  Alexei knows.  He’s been here that long.  He’s taken great pains to learn all the rules and he knows them… he really does.”  Joshua stopped, finally.  Frank was sitting quietly thinking.

“I went to a meeting with the business, many meetings in fact, where Alexei was in the room,” Joshua started up again.

“The Business said, ‘We want A, and B, and C’ and Alexei replied “No, you don’t because if you do C, then these ten things will break” and he’d patiently explain why they’d break, and the business would then ask him what the requirements should be, and Alexei would tell them.  That’s where the spec would come from… mostly from Alexei’s head.”

“It’s not enough to say that requirements come from the business, because they come from the business OVER TIME.  Sometimes, it’s a long time.  Most of the time, the person who understood a particularly complex requirement on the business side has left or been promoted or taken on some new challenge.  The new person has no CLUE about the complex rule, but Alexei knows.  He’s our walking, talking, requirements engine.”

Frank stopped him. “But Joshua, if he’s ever hit by a bus…”

“We are toast.  We need to get that knowledge out of his head.  But in the mean time, we have to do what we can to ignore his odd behavior and listen to his knowledge of the systems.  If you want your system to actually succeed, you need to run your requirements past Alexei and have him make changes.  Otherwise, you won’t have a complete set of requirements.  And things could fail, badly.” 

“In a mature environment, requirements have to carry forward.  New requirements have to respect old ones.  And if you don’t have a requirements management system where they are all written down, then you need people… people like Alexei.” 

Frank sat quietly for a few minutes.  “Maybe I can get him to review the specs, look for flaws.”

“I’m sure he’d love to.”

Frank and Joshua finished their coffee.  This break was over.

Should IT report to the CFO?

By |2007-01-23T08:53:00+00:00January 23rd, 2007|Enterprise Architecture|

Christopher Koch at CIO magazine, as part of the research for their annual “state of the CIO” survey, did some number crunching about whether businesses benefit by having their CIO report to the CFO or directly to the CEO.  Excellent Work!

“I’ve been crunching the numbers for weeks now and the results are more disconcerting than I ever imagined. Having the CIO report to the CFO destroys value in nearly every possible way. Just check out this list, from our survey of over 500 IT leaders…”

Read full article

Why should it matter to you?  After all, the CIO is so far removed from some of us that it’s hard to fathom how HIS reporting relationship can affect your day to day work…

Answer: Because the job market for IT is heating back up.  If you are an IT developer and you are working for a business where the CIO reports to finance and not to the rest of the business, you are reporting to a leader who has been intentionally made less effective by the senior management of the business.  Perhaps you will have interesting work to do.  That’s entirely possible, but it’s also possible (quite likely) that the work you are doing is less stragetic than it could be, and contributes less value (or more cost) to the bottom line of the company than it should.

So if you want to learn how to be strategic… which company should you choose to work at?  One where the CIO can make sure that projects occur that raise the level of excellence in IT, or one where the CIO spends most of his time trying to convince middle-level managers that they shouldn’t have their own instance of SAP because they feel like they are special? 

 Hint: the former is better than the latter.

Should a performance cache query run through your EAI hub?

By |2007-01-20T11:39:00+00:00January 20th, 2007|Enterprise Architecture|

When you pass a message from one system to another, you have to decide: do I want the message to pass quickly, or do I want to be certain that the message gets there?  But what does that do to decoupling?  Here’s a specific case.  Take a look at the decisions made and tell me: would you make the same choices?

We have a new system coming in to existence that will create a business document using a new web interface.  The legacy system that sits underneath also creates the same business document in a less-than-appealing manner.  We put in the new system so that we can move people from the old system to the new one.  Once everyone moves over, we can consider decommissioning the old interface.  Eventually, we will replace the back end as well. 

To see an image of the roadmap in another window, click here.

Of course, the devil is in the details.  We are in step 1: adding the new interface to the legacy system.

For now, the legacy datastore has the domain data that the interface needs.  There are two points here: small domains (like list of U.S. States or Document type codes) and large domains (like lists of Customers or Products).  In the latter category, we have one table in the legacy db that is quite large, so when the user needs a record from this table, we create a query message from the new system to the old to get search results.  Note that in the future, this data will probably come from the performance cache or directly from an upstream system.  It does not “belong” to the legacy app.

The user is waiting on this query to return.  This is important.

The question is this: we are trying to keep the two systems as decoupled as possible.  In that vein, all other transactions between the systems happen through a Biztalk interface.  We can handle orchestration, indirection, mapping, and isolation using Biztalk.

However, for this looking up data in the large table, we want to get the search results data quickly.  We don’t need to translate the fields.  We need message speed,  but not message reliability.  If the user cannot search, then the process of creating the business document is stopped anyway.  (The process is similar to creating an order online… if you can’t see the product catalog, it’s hard to create the order).  The data source is highly reliable already, so there is no need to improve that in the messaging system.

So I have to consider: do we make a direct dependency between the new system and the old one by adding a direct call, completely circumventing Biztalk, or do we keep ALL of our connection running through Biztalk so that we can maintain all relationships in one place?

An image that illustrates this choice is here. The choice is whether or not to put in the direct dependency, illustrated as a blue arrow.

The development team chose to go ahead and add this dependency.

For Reliable EAI calls, the transactions run through Biztalk.  For direct queries, the developers went with a seperate direct call to get the search results.  An additional direct call was added to get the rest of the domain data.

So, now the two systems are connected through two pathsways.  One dependency runs directly to get domain data from legacy to new, while the other runs through a messaging interface.

The reasons for this decision are obscured from me at the moment, but I believe that, when I ask, I will hear: we wanted performance and it is slower to run through BTS (somewhat true).

My choices:

1) Approve the design and don’t worry about it.  (always an option)

2) Ask that the performance cache be more formalized in future releases, so that I’m sure that the dependencies are centrally managed and that the cache isn’t treated as a new data master.  This may add complexity, and a constraint or two, but probably won’t affect performance.

3) Kill the additional dependency and require that all data queries run through the Biztalk engine.

(I’m leaning towards #2.)

What do you think? 

Your SOA is JABOWS (Just A Bunch Of Web Services) and I can prove it

By |2007-01-16T22:24:00+00:00January 16th, 2007|Enterprise Architecture|

Are you ready to answer this challenge… can you prove that your Service Oriented Architecture (SOA) is NOT JABOWS?

 

I’m not asking from the standpoint of technology (are services secure, available, composable, etc) but rather from the business.  In other words, if the business wants to achieve flexibility through the SOA, do the available services represent the correct interaction end points to accomplish that goal?

 

SOA saves money in some environments (especially ours), but it’s greatest value happens when the business can add a capability to meet a competitive opportunity quickly and inexpensively.  Speed is important.  Cost is important.  A Good SOA is structured to deliver on that promise.  In my opinion: Evaluating “how good” a SOA is means modeling out the competitive opportunities that are actually in the corporate strategy and seeing if the SOA can react to both expected and potential competitive moves. 

 

If you look at the ‘universe’ of business processes, you get a spectrum from “changes rarely” to “changes frequently”.  If you place that on one axis and on another, you place the spectrum of “happens occasionally” to “happens frequently”, you get a grid like the one below.

  

 Process Chart

 

IT projects provide tools for business processes.  They automate parts of a business process or collect information or reduce errors.  The point is… which processes?  In the past, traditional IT only succeeded with the processes that changed rarely.  Therefore, the greatest successes were in the upper left, with a lot of projects happening in the lower left (a pragmatic, “pick your battles” approach… solve what you can successfully solve).  The problem is that there is a long list of business processes that occur frequently but that are more difficult to automate because they change frequently.  (the upper right). 

 

That is the SOA sweet spot.  To annotate the prior diagram…

 

Annotated Process Chart

 

So, if you want to know what a GOOD SOA is, look to see if the services cluster around the data elements and detailed activities needed to automate frequently changing business processes.  Certainly, an organization can START from their position of strength on the upper left, but if they never move to the upper right, then the business flexibility that is needed, and which SOA can address, will never be achieved.

 

So, to evaluate how good a SOA is, you need to ask:

A.      What business processes occur frequently and change relatively frequently?

B.      Of those processes, which are the most critical opportunities for automated tools to provide value?

C.      Of those opportunities, which are aligned to corporate strategy?  (feedback loop to executives here… opportunities missed)

D.      Of those aligned to strategy, what data elements and activities are needed to compose them in a flexible manner?

E.       Of those atomic elements, and required infrastructure components, how many exist in the enterprise SOA?

 

Step E produces a number.  The closer that number gets to 100%, the better the SOA is.