Using the maturity model for benchmarking

Performance monitoring is one of the key elements of the Telenor Revenue Assurance Program (TRAP), focusing on answering which leakages have been discovered, to what extent are all products and value-chain parts covered by controls, and to what extent are the organization and resources allocated to RA adapted to future needs.


Reporting of trouble cases and the monetary effect of RA activities has been part of TRAP from the beginning in 2003. Telenor affiliates have experienced the same development as most other telecom operators - as maturity increases, the number of cases and the size of the leakages diminish.


Measuring a leakage and assessing a value to block it is an ongoing discussion that falls beyond the scope of this article. What can be said here is that relying on historic cases as the only basis for planning future activities is not enough. To get a better basis for long-term planning, Telenor has introduced two strategic benchmarks: The Coverage KPI and The Maturity Model.


Two methods


Introduced in 2005, the Coverage KPI aims to get a measurement of the extent Telenor's affiliates have implemented controls in various parts of the value chain. The value chain is split into five main processes and a number of sub-processes, covering all parts of the value chain from product development through collection.


The assessment is done on a product level, and aggregated results are computed for each of the main revenue streams and processes. As with the Maturity Model, we use a five-step scale ranging from Unreliable to Optimized.


Apart from leakages and controls, we also needed a measure of the RA organization. To get this, we implemented the TM Forum-developed Maturity Model, which provides a comprehensive assessment of the main dimensions of the RA organization, including organization, people, influence, tools and process.


These dimensions align well with what we need to focus on when making long-term roadmaps for developing the RA function. The Maturity Model was run in 2007, and we intend to make another assessment using the same KPI this year. Going forward, we plan for this to be an annual or semi-annual exercise.


Using the Maturity Model



The Maturity Model consists of a questionnaire with more than 50 questions. Each question is given a weight, enabling the computation of a total score for each of the five dimensions.  A total score is then calculated as the average score of the five dimensions.


Instead of making our own version, we used the questionnaire as it was from TM Forum, as we saw this as an opportunity for benchmarking not only between Telenor affiliates, but also with the rest of the industry.


Every year, we run the TRAP Review, where all affiliates are visited to go through their status and discuss the roadmaps going forward. The Maturity Model was the main item on the agenda for the TRAP Review last year, and we went through the questionnaire in workshops with local RA managers.


Representatives from the Telenor Group level were part of the assessment in the affiliates, to secure consistency in the interpretations of the questionnaire. The results were then compiled at Group level, and each affiliate could compare the results on the aggregate level, as well as per dimension and per individual question.


Combining the two


Coverage could be built into the Maturity Model as a sixth dimension. In my view, however, the TM Forum Maturity Model and the Coverage KPI supplement each other.


The expectation is that high levels of maturity will coincide with high coverage.  Although this will be the trend, it is possible to have wide coverage in controls, but a relatively low level of maturity as it is defined in the Maturity Model. It is possible that an organization may have a narrow scope for RA combined with a high score in the Maturity Model, even though I think this situation is not commonly found in practice.


It is important to note that both KPIs attempt to measure the status in the whole organization, not just the RA department.


Planning tool


The total score for an affiliate gives an indication of where this company stands compared to the other affiliates in the Telenor Group. Looking into the individual dimensions of the Maturity Model and comparing on a question-by-question basis makes it possible to identify strong and weak points in the individual affiliate.


The same is true for the Coverage KPI. Because the assessment is made on a product and process level, it is easy to identify which areas need an extension of the control coverage.


To get the full value of these KPIs, it is necessary to analyze the results in detail and spend time on discussing what can be done to improve the score. Used this way, it makes sense to run the assessment more often than on an annual basis on the local level. 


One clear recommendation is to include target scores in these KPIs in the targets for the RA manager. Doing this, the RA manager can assess the effect of a project on the KPIs and use this as part of the justification of the project. This helps direct attention and resources to areas where the effect is greatest. Just as important, this can help maintain management focus and attention to the long-term development of RA - not only short-term focus on leakages but also the "dollar" effect of reactively blocking these.


Einar Nymoen is a  project director for Telenor ASA/Global Coordination