Programs as Benefit Measurement Frameworks

There’s this myth pervading the portfolio management space that there’s a single benefit measurement model to rule them all.  Whether it be NPV, ROI, IRR, user satisfaction surveys or any other measure of tangible benefits, the reality is that any project may be measured according to any number of benefit measurement models stretching in a spectrum from the tracking of objective data to subjective data.

The trick is to identify the project type, and map that project type to an appropriate benefit measurement model.  For example, an IT project to support a specific application may be designed to reduce down time, to enhance user satisfaction, or to increase the speed of operations.  Each of those benefits are measurable, insofar as they can be assessed through service metrics or satisfaction surveys.

The challenge, however, is that oftentimes we have never implemented the monitoring systems to track these metrics – despite the fact that they’re in the gospel of ITIL as part of Service Design.  Hence, if we want to track against these metrics, we need to incorporate the creation of the baseline and tracking system into the budget of the project.

A PMO can slip that monitoring into the budget of one project, but sooner or later, sheer inertia negates this, as the stakeholders begin to rebel against repeatedly approving extra budget for monitoring project success.  Due to simple budgeting constraints, we can’t incorporate a metrics monitoring system on the front end of every project.  From a logistical standpoint, this won’t work as well, as it typically takes a fair amount of time to build up meaningful performance baseline data, and by the time we put everything on pause to collect this data, the project would have slipped far past the most forgiving stakeholder deadline.

The solution is not to incorporate metrics tracking into every project that passes through the approval process, but as the front end design of every program that passes through our portfolio.  If we treat an application as a program, we can set a threshold, i.e. any application with X number of users or that we will use for longer than Y years will need a metrics tracking system incorporated into the program design.  The metrics tracking system will generate performance data to assess if we are hitting our service performance goals, and our financial goals.

Those metrics then become the performance metrics for every project that is incorporated into the program.  This gives us another answer to the question of how we define programs within our portfolio.  A program is a framework for benefits measurement that may applied to the projects that reside within the program.

This also makes our investment decisions easier as we can budget to the program, and then allow the metrics within the program to dictate the actual composition of projects within the program.  How to assess the value of the program?  By identifying how it ties into the overall operations of the company – which is far easier than doing so at the project level.

Taking this concept one more level, a properly designed program should be tied to ongoing business conditions.  A periodic assessment should be conducted of the business environment to confirm that the program itself remains viable based on certain predefined measures of program success.  When the business conditions change, the program funding spigot can be turned off, effectively killing all projects within that program.  Alternately, additional funding can be added to the program, which in turn is directed to the proposed projects that meet the programmatic success measures.

PMO Symposium San Diego: Where Portfolio Optimization and Guamanian Food Meet

Feel like this blog has gone off the rails and want to vent in person?  Feel like things are just great and want to share your ideas?  Come on out to San Diego in November for the PMO Symposium – where a couple hundred motivated PMO directors will get together to share ideas on how to better provide value to the organization.

I’ll be hanging out in or near the Microsoft booth and looking forward to chatting…although I’m admittedly a bit bummed that my favorite Guamanian restaurant (full disclosure: it’s actually the only one I’ve ever been to), the Islander Grill shut down a couple of years ago.  Luckily, an Internet search reveals Chamorros seems to be carrying the flag.

Strategy Through the Lens of Language

Like many languages, Chinese does not differentiate between the concepts of problem and question.  Those two concepts, which are arguably distinct in English, share the same noun in Mandarin.  To me, this highlights the fundamental semantic similarities between the two concepts:

• A problem represents a discrepancy that exists between the way the world is and the way the world should be.
• A question represents a discrepancy that exists between what I know, and what I believe I should know.

Often we use terms such as “problem” or “question” as synecdoche, to represent a larger collection of concepts through a small subset of those concepts.  In this case, the existence of a problem or question typically also implies the existence of two other unique concepts that often coexist within the portfolio management space:

• The solution (or answer) which resolves the discrepancy between the current state and the desired future state.
• A sense of values that drive the recognition of the discrepancy.  Without a value that tells us this is wrong, we would merely accept what is.  Values are the engine that drive us from the is to the should be.

When we say the words “problem,” or “solution,” we are really referring to the entire solution space that consists of values, problem sensing, problem recognition, and the solution.  It’s just typically we don’t like to be so long-winded, and just talk about the solution while leaving everything else implied or assumed to be understood by the people we are working with.

This concept comes to the fore when we start talking about organizational strategy.  In fact, I would submit that most (if not all) organizations exist to solve a problem….to answer a question.  An organization is an answer writ physical, i.e. a long term solution to a problem that has become a sustained system.  Thus, the study of answers has a lot to do with the study of organizations.  The original question is the raison d’être for the organization. The original question is the urQuestion, the fundamental driving force of the organization.

Once we think we have an answer to the urQuestion, we need to start adding detail to the discussion.

As that detail evolves, it comes to represent the complexity of the modern organization.

Each time a new question is identified, a new subsystem is spawned, and before you know it, the organization is a forest of questions and disparate answers.  Each new solution answers a part of the question, and hopefully, assists in answering the urQuestion.

Portfolio analysis, and in fact almost all consulting, is often involved with identifying the questions on the right side of the diagram, and then gradually working all the way back to the left, so that the solution on the right can be traced to how it contributes to organizational purpose on the left.  This is providing context to value.  Without context, solutions have no value.  Without an understanding of how the solution gets the organization closer to answering the urQuestion, the solution has no real value – only perceived value.

Once we develop this model of an organization, that begs a couple of fundamental questions….

Can the urQuestion Change?

It’s almost inevitable that over a period of time, the urQuestion will shift from the original question to something other.  This is the result of change both within the organization and without.  As the organization becomes a self-sustaining system, or the proverbial solution in search of a problem, it starts to naturally look for other questions to answer.

The trick is periodically to reassess the organization and to recognize that this shift has happened – then to review all of the other answers that comprise the organization, and identify if they still tie back to the new urQuestion.  Each answer represents a system, which becomes self-sustaining and resistant to eradication.  In many older companies, you’ll see these systems proliferate with little or no link back to the organizational purpose.  As the company ages, the risk increases that each of these subsystems becomes baggage weighing down the overall construct.

The urQuestion as an Artificial Teleological Construct

This then leads us to another potential issue.  What if, despite our best efforts, after all of the requirements elicitations, interviews, and reviews, we still have not been able to uncover the urQuestion because it in fact does not exist, or nobody remembers what it fundamentally was?  Perhaps there was never an identifiable managerial incantation that caused the organization to spring into existence.  In such circumstances, it would certainly be natural human behavior to attempt to create structure where no structure exists, and to simply create an urQuestion.

Thus, the urQuestion may actually simply exist as a teleological construct, a mental overlay of what we perceive the organization should be striving for.  That still makes it none the less valid as a guide to making strategic decisions.

…which brings us back to the original concept.  If we treat problems as being synonymous with questions, and if  we treat organizations as physical representations of solutions to problems, then we can apply the study of questioning to the study of organizations.  We can translate the structure of an organization into a series of linguistic constructs.

The Tragicomedic Organization

It’s no secret that I’ve retreated to my underground lair of late to work on upping my presentation game by trying my hand at the odd bit of stand up comedy (and roller derby, but that’s a different story).  That, of course, begs the question, how does one get started in the world of standup?  I could take a class on standup, but as a couple searches quickly determined, Houston appears to be a bit of a wasteland in that regard  – although, as in all cultural comparisons, Austin definitely shows promise.  Hence, I’ve retreated to Plan B, and started my odyssey to become an amateur standup comedian (after all, let’s be realistic, going pro is a non-starter) by going out and purchasing a couple books on comic theory.

Thusly, I found myself reading the Comic Toolbox the other day, as the author, John Vorhaus, deconstructed different comedy types.  One of them particularly resonated.  You see, the ensemble comedy (think Friends, Bad News Bears, Inglorious Basterds,  etc.)  involves what is considered a group protagonist.  The group itself is the protagonist as it strives to reach its goals, whether they be killing Nazis, banding together to win the championship, or raising enough money to buy their old theater from the evil businessman.

The comedy lies in the conflicts between the members of the team whom invariably are written with divergent characteristics and personalities.  The common goal binds them however, because without that common goal, their radically divergent personalities would quickly cause them to lose cohesion and focus on their own specific needs, not that of the team.  The common goal provides the catalyst for the team to work through their differences and unite in the face of overwhelming odds.

Without that driving goal for the group protagonist to work against, the team would quickly split apart and disperse into the four winds.  That would no longer be a comedy.  That would be a tragedy.

Lessons Learned, Stupid Organizations, and the Other Things

Many organizations are stupid.  This always surprises me as stupid organizations almost always have very smart people working within them and are often quite successful.  Stupid organizations are stupid however, because they’re missing actionable models for success.

Last Saturday was the fourth annual TEDxHouston, a great day of slam poetry, obscure conceptual mix-ups, and lessons on how stand up comedy and gender-neutral pronouns can save the world.  All of this was held around the proverbial corner at Rice University.

The basic concept of TEDx, for those of you who’ve never attended, is to pick a topic or concept, and then to invite an eclectic mix of local thinkers in the fields of Technology, Entertainment, and Design to interpret that concept through the lens of their particular passion.  The audience is then invited to translate those concepts through their own perception bias and into the realm of their own professions.  Saturday’s concept was “The Other Things,” a reference to the John F. Kennedy moonshot speech delivered at the Rice University stadium 50 years ago:

We choose to go to the moon. We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.

Last Saturday, a couple dozen speakers took 12 minutes each to identify what they felt those “other things” were.  One such session was presented by a physician from MD Anderson, a local hospital specializing in cancer care.  That’s what got me thinking about lessons learned and the concept of stupid organizations.

The concept is surprisingly simple.  Most people in the world do not have access to the world class cancer treatment available in treatment hubs such as the Houston Medical Center.  MD Anderson is thus teaming up with IBM to develop a rich database of data related to treatments and success models.  If a patient is diagnosed with cancer, the doctor may tap into this wealth of information to prescribe a tested, valid treatment plan. (Link to discussion here)

This is a perfect model for what the lessons learned exercise should be.  In a perfect world, lessons learned should focus on what success looks like, and how the project achieved success.  Specifically, a lessons learned exercise should be designed to refine success and convert it into an actionable model.  This actionable model may be used in at least three distinct ways:

1. To structure the next project to take advantage of this model thereby enhancing the chance of success.
2. To structure the next project to identify when the project is diverging from the approved model, and thereby increasing risk by branching onto a heretofore unknown development path.
3. To structure the organizational processes to ensure that the chance of project success is enhanced (See point #1)

Not only does a smart organization continuously refine the model that brings success, but the smart organization continuously looks at that model as a scientific hypothesis, i.e. a theory that explains how things happen, but that can be jettisoned the moment we come up with a better model.  Models are like any theory: they’re valid until they’re not.

Instead of lackadaisically conducting a couple of interviews after your next project and calling it lessons learned, think about how to convert your current project into a repeatable model for future projects.  In retrospect, what were some of the key indicators and patterns that led to the place you’re currently in?  Break those down into at least two categories:

1. Things that directly impacted the course of the endeavor.
2. Things that enhanced the chance of the endeavor succeeding.

Take those concepts and feed it back into your lessons learned exercises, project structuring, and HR onboarding process.  Don’t start a lessons learned process by focusing on the solution and asking what generic information you need.  Rather, focus on the fundamental organizational problem by really defining success, asking what models lead to that success, and how you can structure your projects to map them to those models – or to detect when the project no longer aligns with a proven model.

Continue to test your models.  Don’t accept a success model as the gospel, instead continue to test it against success patterns to see if you’ve really distilled the essence of what makes your organization successful.

That’s how smart organizations learn.