Home > Enterprise architecture, Maturity models, Metrics > Babies, Bath Water, And Enterprise Architecture Maturity Models

Babies, Bath Water, And Enterprise Architecture Maturity Models

Recently I took a look at an EA-maturity-model-cum-roadmap from Leo de Sousa, Manager of Business Application Services and Enterprise Architecture at the British Columbia Institute of Technology (click to read Leo’s blog on EA CMM). To my surprise, I liked it. Why was I surprised?

I have never liked EA maturity models. Yes, tracking progress is important. And yes, there should be a consensus about what characterizes a mature EA practice. But I don’t like how they would ostensibly be used to compare one enterprise with another, a la benchmarks. Perhaps I was soured on them by seeing them used as a governance technique in US federal agencies.

The Office of Management and Budget (OMB) required agencies to assess themselves against a standardized maturity model, with progressive hurdles in successive years. Federal agencies, accustomed as they are to all sorts of oversight and compliance mandates, know how to pass compliance audits. Did you ever wonder how (then) Popkin System Architect got so popular  in the federal government? An EA tool was required to demonstrate a certain level of EA maturity and System Architect was the lowest-cost offering at the time (I’m sure there are other reasons as well). Behavior was around letter-of-the-law compliance, and it seldom catalyzed getting with the spirit. Even when Dick Burk at OMB introduced a clever workaround in a second version of the model — you could leapfrog to a level 4 if you showed actual business benefits, regardless of what other checklist items you missed — agencies simply marched through the maturity level checklists getting the requisite items done. The scores were good, but in my opinion they overstated the degree to which EA was ingrained in the culture of the agencies.

And then there’s the notion of a single number representing an enterprise. In most reasonably large organizations, there is some degree of federation of the EA effort. Some business units might have very advanced practices, while others have virtually none, and the enterprise-wide practice may have an entirely different agenda. What does one do, rate all units and average the number? That would be a pretty meaningless metric.

Source: Leo de Sousa, Manager, Business Application Services and Enterprise Architecture, British Columbia Institute of Technology

Source: Leo de Sousa, Manager, Business Application Services and Enterprise Architecture, British Columbia Institute of Technology

So, I’m afraid because  of all this I threw the proverbial baby out with the bath water regarding EA maturity models. But I like Leo’s. He took some fairly standard criteria and modified them to fit his organizational goals and processes. They still resemble other CMMI-like level criteria, so his 3 would be similar to someone else’s 3, but that’s not the main point. The point is to create a roadmap for where he wants his EA practice to go, and then rate and report actual against plan. This approach is about creating a tailored roadmap to consciously mature your EA practice; it is not about scoring maturity of implemented architectures.

No big deal, you say? I tweeted about this and a couple of wise EA folks who usually respond to my tweets essentially said “of course, you have to make it your own, what else would you do?” making me wonder if I’ve been missing the boat all along.  But I don’t think this is a prevalent attitude about EA maturity models. I think the general idea is to rate one’s org against where it ought to be, by some industry-standard scale. I don’t find that worth much. I think an organization should understand where they are and where they want to be, figure out how to get there, and track their progress. If someone else’s model of how EA can improve your org helps you describe your goal or your path, great. But generic doesn’t work — for any generic goal statement, you must edit it into terms specific to your enterprise. And don’t set any goals without including the statement “as measured by ___.” You can use the Goal-Question-Metric technique for filling in the blank, but that’s a topic for another day.

To really manage your EA agenda, I’d recommend taking the maturity model as a roadmap a step further and also populate it with goals that don’t fit the standard CMMI scale language.  For example, if you have an incipient information architecture or business architecture practice, place goals and milestones related to expanding and formalizing those efforts on the roadmap. I’ve been interviewing people about their information architecture initiatives and working with Forrester analyst Jeff Scott on approaches to business architecture that can be broken down into a cookbook-like  sequence, so watch this space for more on phased approaches to these aspects of architecture initiatives.

How do you all measure EA progress in your shops? Do you use a maturity model? And/or have you created concrete custom goals with associated metrics and measure against them? Take the quick poll below and we’ll all get to see!

  1. August 27, 2009 at 3:20 pm

    Great analysis, Gene – have you seen the Micron EA case study where they apply ACCM and J Ross’s model simultaneously? Interesting approach.

    • August 28, 2009 at 2:48 pm

      I haven’t, John, would be very interested. Your comment reminds me of another issue I have with maturity models, which is that they appear to imply a rigid sequence. That’s another misconception that’s not really the fault of the model but rather the user of the model. In the earlier stages of an architecture program you could start working on things that set the stage for the global flexibility that characterizes the most mature stage in Jeannie Ross’ model. Yes, you may need to focus on standardizing the infrastructure to get you from “business silos” to “standardized technology,” but before you complete the journey from one phase to the next there could be tremendous value in taking on some aspect of architecture that is more related to advanced stages, like business architecture and capability modeling.

      Anyway, back to your comment — I tend to like hybrids because they mean someone has thought about what would work in their environment and tailored a couple approaches to their needs, rather than slam in one approach.

      • Doug Mutart
        September 14, 2009 at 4:32 pm

        I created an EA maturity model similar to Leo’s earlier this year tailored to our own wants/needs, and I successfully incorporated several of the characteristics Jeanne Ross describes into the attributes list for each stage. It really works. Reaffirms that it is, indeed, a journey and not a snapshot in time.

  2. Rick Ladd
    August 28, 2009 at 11:00 am

    Gene – I just started following you on Twitter as a result of (I think) Dion Hinchcliffe’s recommendation and this is the first thing from you I’ve read. Glad I did. Although I am not in IT (however, I was recently referred to by our Chief Architect as an honorary member), I have begun using maturity models for several of our overall business efforts. Inasmuch as I believe strongly that one size definitely does NOT fit all, I am heartened by your analysis. I especially like your suggestion to use goals that don’t fit standard CMMI scale language. Thanks for an enlightening and enjoyable post.


    • August 28, 2009 at 2:52 pm

      Thanks for your comments. Quite a compliment for a Chief Architect to name a non-IT person an honorary architect — we’re an elitist bunch, I’m afraid, so that’s high praise!

  3. Doug Mutart
    September 14, 2009 at 4:26 pm

    You really nailed it again, Gene. It’s all about looking through the front windshield at where we want to go, not through the rear view mirror (sorry, but I just can’t help it the automotive analogies) at where we just were. As our goals and objectives change – with time, new CIO (speaking with lots of experience here), new CEO, new skills on the EA team – so must our approach to EA organization, operational model, and outcomes.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: