Macro-Analysis of traffic-light reporting

Whether you are a fan of traffic-light reporting for projects or not, its use is widespread across the industry. There are numerous articles on the internet about the criteria that should used to determine whether or not a project should be RED or GREEN, and whether projects are accurately reporting their status or not

Most PMOs who offer a governance and support service to project teams will have measures that kick in when projects hit key thresholds. A project that is Yellow/Amber for several weeks may be a candidate for an audit, or a project turning Red may trigger the PMO providing additional project support. These measures can help bring projects back on track, or prompt the business to pull the plug on investments that are likely to fail. 

However, traffic-light status' can be used to conduct macro-analysis, revealing trends in your project portfolios that the PMO can help mitigate. 

Status by Period

The data below shows the traffic-light status of a portfolio of projects over a 12-month period. 

Traffic-light status of a portfolio of projects over a 12-month period.

Traffic-light status of a portfolio of projects over a 12-month period.

We can see here that two-thirds of our project status over the year were GREEN. Great job! But closer analysis reveals the distribution of red projects varies by month. Let's tidy the data up a bit:

Traffic-light status of a portfolio of projects over a 12-month period.

Traffic-light status of a portfolio of projects over a 12-month period.

With the data arranged like this, we can see a pattern start to emerge. Around September we saw the number of RED projects increase significantly. But why? This dataset can only tell us so much, but we can assume it is down to one of three possibilities:

  1. The projects have a common factor (eg shared resource, supplier, technology) that was the basis for the status deteriorating
  2. The projects were all cross-dependent, so a problem on one project forced ALL the related projects to go RED (Dependency analysis such as the example I used in this beautiful data post would help reveal these cross-dependent projects).
  3. There is a factor that is external to the projects that has caused the deterioration.

In the case of the portfolio in question, the first two options were quickly ruled out. What was more interesting was that a similar spike was found in data from the previous year! Through interviews with project managers and some targeted retrospective sessions, we were able to identify a 'perfect storm' that occurred on an annual basis. Project resources and PMO teams would go on holiday over the summer. During this period, productivity would decrease, but this was not always recognized by the project teams until late Aug/Sep when resources returned to the office. Furthermore, September was a critical time in the annual business planning process, which resulted in changes in strategic direction and project resources being diverted onto business planning activities. 

The analysis resulted in several PMO initiatives being put in place:

  1. All capacity models reviewed in Q2 to ensure summer leave is accounted for.
  2. Time required for business planning formally blocked out for key resources.
  3. Health checks of all key projects in June – focused specifically on being 'September-ready'.
  4. Project Managers made aware of the macro-trend and the causes to help them shape local mitigations.

Other types of Analysis

There are a number of other ways that traffic-light status can be used by the PMO to create an environment where projects are more likely to succeed. Here are some examples you may want to try:

  1. Risk log and traffic-light status – Are project logs an accurate predictor of future traffic-light status? Whilst some problems can hit out of the blue, it would be reasonable to assume that a project risk scores would serve as an indicator. Depending on the results of this analysis, you may decide to provide more risk management training (if there IS NO correlation) or provide more support to projects where the risk score is high (if there IS correlation). You could also try this with other logs, such as the issue and change logs.
  2. Traffic-light status on successful and failed projects – Compare the aggregate traffic-light status of projects that are considered 'successful' and those that are considered 'failures'. If there is no correlation, then the PMO may be using the wrong KPIs to determine project status.
  3. Traffic-light status by project phase – If your organization uses standardized project phases (initiation, analysis, design, etc.) then you can analyse the macro-trend in traffic-light status by phase. Depending on the results, you may decide to focus training initiatives on specific techniques and consider what early-warning systems you can put in place.
RAG/Traffic-light status by project phase.

RAG/Traffic-light status by project phase.

Does your PMO undertake similar analysis? Share your ideas in the comments box below!