Martin Fowler wrote a post recently on the value of Design. Quite a good post, actually, in which he states that the productivity gain collected by doing “no design” is illusory for systems of sufficient complexity or size.
I agree with the fundamental precept of the article, but I must disagree with one of the conclusions. Let me explain by way of an anecdote.
The year was 1957. It was late summer and the meeting room has become stifling hot, even with the air conditioning running. IBM CEO Thomas Watson Jr had spent the day listening to an endless debate among engineers about how to cram more information onto drum-style disk drives and keep the data coherent. They had a major show to prepare for in Europe, and the hardware wasn’t quite right. By 3pm, everyone was dying for a break, and the attendees decided to take 15 minutes to get some air.
Mr. Watson was irritated. He expected to spend a late night at the office and he was going to have to call home and explain to his family why the CEO had to work late. As he waited for the elevator to take him to an upper floor, two engineers stood next to him, waiting for an elevator going in the opposite direction. They were arguing (loudly) about the technical details of their work.
Mr. Watson’s elevator arrived first, and he jumped at the chance to get away from the bickering engineers. As he entered his elevator, he stopped, leaned back out, and said:
“If it were 1910, you would be the guys trying to figure out how to build a better horse and plow, instead of thinking about building a tractor!”
Moral: the problem we are solving may be moot if we are not considering the context in which we solve it.
We live in an age of craftsmen. Among the greatest craftsmen, the craft is very important. Skill, quality, accuracy, precision, and beauty are all parts of the mix we call Software Development. Admit it. We are crafting, one at a time, an ornate and beautifully crafted solution to a unique business problem. For some problems, this is justified. For others, it simply is not.
In real life, when my toaster oven broke, I bought another one. I made no attempt to fix it. It was not worth my time.
I’d venture a guess: well over 90% of our problems are not unique. They are common. Every business has nearly all of the same issues, and most are not competitive differentiators. Yes, for all applications, strategic or not, we apply the same craft skills, even though the advantage we gain from craftsmanship only really apply to the unique strategic applications. The rest do not need craft. They need construction from prebuilt components. I need a Home Depot where I can buy the hammer I need, not my own personal forge to make each hammer, one at a time.
We will never kill off complexity. Applications that create competitive advantage may be complex. We need to craft these systems carefully, leveraging prebuilt components and careful design. The rest of the problems do not need craftsmanship. They need to built out of completely disposable components that can be delivered so inexpensively that when it breaks, we discard it and put in another rather than repairing it. Using craftsmanship to address these problems is focusing on the horse and plow when we should be using a tractor.
Martin eloquently argues that a solution that requires no design is, by definition, very simple, because the bar that you need to cross is fairly low before the complexity demands good design, I’d say that is true, but only because of our present immaturity. Sure, we have to live in the here-and-now. But if you really want to change the world, raise that bar. Make it so that the needs of the business can be met with business specific, standard, interchangable components for 90% of all requirements.
Design is, and will always remain, the art of the intelligent and thoughtful few. In the future, the majority of systems will be delivered with very little design at all… simply assembly. The age of craftsmen must someday end. The one who ends it, wins.
"If it were 1910, you would be the guys trying to figure out how to build a better horse and plow, instead of thinking about building a tractor!"
Sure: an internal combustion engine has proven to provide more horsepower than a horse and plow, both theoretically and in hindsight. But, if it’s 1910, you’ve got to convince the CEO and CFO and CTO to risk a business built on top of horses and plows that everyone in management is comfortable working with, cut profits (perhaps to the point of having quarters in the red), put bonuses and raises for employees at risk (and therefore put employee morale at risk), and reduce dividends to shareholders.
And that’s just the risks of making the decision to go ahead. There are no assembly lines to manufacture the machines, no skilled workers to work the assembly lines, and skills for these workers have to be invented, along with new tools for these workers to use, and factories for these workers to work in. This is all new investment, in unproven technologies and techniques that have to be invented from nothing.
But even if we get this far, and we can manufacture the engines, what fuels this new product? Horses only require ubiquitous and relatively-inexpensive food and water. The fuel for internal combustion engines requires oil drills, oil pumps, oil refineries, and … oil! The business must now invest in the pursuit of oil and its refining and distribution and sales and taxation: well outside the domain of a business that simply manufactures plows designed for horses.
And then, there’s the tractor that has to be built around the engine.
And then, customers have to be convinced to commit spending money to use the unfamiliar, unproven tractor instead of the familiar horse and plow that they’ve always used. And, given the investment, customers will be asked for far more money than a competitor’s horse and plow would cost.
Who, in management, would take these risks and losses seriously, especially when they come from someone who is not a manager, and obviously demonstrates no consideration of ROI?
Now, compare this engineer’s solution with another engineer’s solution: there’s a way to make the plow lighter, so the horse can do more work in the same amount of time, or can do more work by working longer before exhaustion, or the horse lives longer due to the reduced stress so that less horses have to be purchased by the customer to use the plow.
If these two engineers argue their solutions, which solution is going to be more attractive to the CEO?
Alternative solutions for managers to choose from are part of the value of engineers arguing solutions. Disagreement should not be anathema. Arguments are simply the bustle in the marketplace of ideas.
@Donald,
Please read about the history of the John Deere company.
They did not invest in oil. Engines were being built by their competition (Ford and International Harvester). They bought the ability to build engines by buying a tractor company (they had been primarily a plow company before then). Note the year: 1915.
http://www.deere.com/en_US/compinfo/student/timeline_1900.html
Companies change. They must. If you are not aware of the competitive pressures for your enterprise, or if you assume you have to invent everything yourself, you will miss opportunities. There are always ways to reuse things. John Deere benefited by being able to reuse the gasoline infrastructure that was already in place from Ford Motor Company’s Model T (which had been selling well for a number of years).
The engineer who builds a better horse drawn plow will have done a great job of solving the wrong problem. That is what Martin wants us to do: build a lighter plow. Perhaps that is the right short term solution, but it is not the long term solution.
Notice that, in the anecdote, Mr. Watson did not stop the debate. He simply asked them to expand their minds to include the actual business problem. It is not to make a better plow. It is to make money.
Surely it has always been the case that what we craft today becomes tomorrow’s commodity component, ready for assembly (with the exception of specialty products). But don’t we then find new and innovative ways to combine components, sometimes venturing into places nobody has gone before? And don’t these new places raise loads of new challenges that nobody has solved before, requiring a craftsman like approach?
New frontiers may be small, i.e. a niche product or industry (GPS navigation, gaming, mobile applications), or large, like the Internet. Funny, I remember a similar conversation before the Internet was "around". Languages beyond 3rd generation promising to make technical people obsolete. I still see it today with tools that are supposed to allow business analysts to generate BPEL or other process workflow type code. I’ve still not seen this work – at all. In some environments it sounds analogous to home brain surgery because some clever guy has invented the power drill.
Surely life doesn’t become so boring that we don’t develop new ideas and have no need for people who can think? Don’t we simply grow to become craftsmen where we are required? Doesn’t assembly require some thought, and dare I say it design. Just because we can now mass produce hammers doesn’t, all of a sudden, do away with the need for architects, designers, and landscapers.
@Andrew,
"Surely it has always been the case that what we craft today becomes tomorrow’s commodity component, ready for assembly "
I have a very difficult time believing that this is how systems are actually developed in your organization. From my observation in a wide variety of companies over the past 27 years (OMG), what we craft today is discarded and replaced tomorrow, usually with something that adds one capability and breaks another. The notion that we are actually building one capability on top of another may be the goal, but it has never been the reality.
As far as "venturing where no one has gone before," nonsense. I have, as a part of Microsoft IT, created a map of the IT infrastructure that I can use to tie all 2,700+ applications in our portfolio to a single hierarchy. That was quite an effort. However, as a result of that, I now have a map of 82 Solution Domains that represent 100% of the functionality of any manufacturing-based enterprise. I’d venture that heirarchy would change less than 10% from our enterprise to ANY other enterprise on earth. Period.
The world has been explored, my friend. Many times. We are too hidebound to see it.
If it is your job to build a house by hand, you may not think of something so routine as standardized windows and doors, but we now use both of those items on a routine basis. Well over 99% of all American homes, built in 2006, used prefabricated doors and windows. My floors are built with standard floor board and standard carpet and standard tile. My kitchen cabinets and appliances have standard sizes and shapes and hardware. My electrical fixtures are not only standardized, but detailed building codes describe what can and cannot be done to provide a safe living environment in which I can live: codes that only vary in small ways across the country. Nearly everything is built out of standardized components. Nothing, and I mean nothing, was crafted onsite. Things were cut, assembled, and finished. That is all. And that is what makes the house affordable.
That does not mean that design does not occur. Certainly the house I live in today was designed. Once. It was built many times by the same homebuilder. I’d venture to say that the design is far from revolutionary. In fact, the design of my home is not substantially different from the designs of homes in 1978 (when I was studying bricks-and-mortar architecture). The look is different, as is the layout, but the elements of quality have changed very little. Did the floorplan for my house require an architect? Yes. But not for each house, and not even an expert architect… just a good one.
As our profession matures, and we begin to understand and then use truly reusable services and interchangable business components, we will rely less and less on "DEEP" design, and more on assembly. Design, in the true sense, in the Martin Fowler sense, will be applied by experts to those key capabilities that provide differentiation between one enterprise and another. That will account for real work, but only for about 5% of total infrastructure of an IT organization. The rest will be assembled from prebuilt components.
I have every intention of making sure this happens. If no one else defines it, I will.
"Surely it has always been the case that what we craft today becomes tomorrow’s commodity component, ready for assembly."
Who said anything about systems. I was speaking more generally. You raised the analogy of the hammer. All I was pointing out was that mankind over some period of time manages to make things into commodities. A small example to start with – I don’t expect to rewrite a String class every time I work on a new project or system. Once upon I time rewriting even something as trivial as a String class did occur. Once there were no toolkits or frameworks for things such as user interfaces, mathematics, distributed communication, etc… A slightly bigger example, I don’t expect to rewrite device drivers for hard disk operation. Again, this was once crafted each time by some clever chaps, until another clever chap thought an operating system would be a good idea.
"The notion that we are actually building one capability on top of another may be the goal, but it has never been the reality."
Hasn’t Microsoft been one of the companies that has help make it a reality, i.e. providing technology on which other technology can sit? Or are you telling me that those things Microsoft sells as reusable components are highly volatile and I should expect that they’ll discard it in future? When is this likely to happen? When might I start to worry about other underlying technologies that make up my solutions, like TCP/IP (or sockets), HTTP, the .Net frameworks, maybe even the Windows operating system. I guess Microsoft has just rewritten their entire operating system so maybe this wasn’t a good example – but by gosh others haven’t.
"The world has been explored, my friend. Many times."
Which world? Does your hierarchy work really well only in manufacturing based companies? What about others, like Telecommunications? Have you solved the complex problems surrounding network alarm correlation for instance? Cause that’s a world that I see a lot of new investigation into all the time. Last time I talked to the CTO responsible for this in IBM there seemed to be a huge amount of DEEP design and ongoing research into solving this problem effectively. No prefabricated window for this nasty little problem yet.
I hear what you’re saying about standardized components. I get the fact that American houses are all the same, right down to the last nail. What I’m trying to point out though is the view that "DEEP" is relative. Maybe it’s just my little corner of the world, but what was deep once becomes trivial and new problems and opportunities come along that reuse in some shape or form (i.e. could be simply a design pattern or way of approaching a problem) what’s gone before. Sometimes these new problems have their own "deep" level of design, i.e. no one has solved it yet.
All I’m trying to say is the world did need more than 640K of memory. There are new inventions. There are new places in which to apply technology in new ways appearing all the time. Along with this comes the need for design. The Internet did happen, and Microsoft didn’t miss it – even though Bill’s view was almost to limited to see a new opportunity. The semantic web is on the horizon. And the earth was proven to not be flat. I don’t believe that there are no new inventions and I don’t believe we’ve fully explored yet. And if we have then why does Microsoft come out with new standards, technologies and versions of its operating system etc…? Or don’t these require any design? Will there be no new disruptive technology inventions? Will there be no new ways of doing business?