• Subscribe
  • Log in
  • search icon

Performance.

Results Overview

Performance scores were based on up to 47 questions across seven components that were common to all, and two (member services and funded status) that were only applicable for some organisations. Components were re-weighted to accommodate what was not applicable, so that each individual fund was scored out of 100. The overall average score was 64, up from 62 last year and the second highest scoring factor after governance (71). Average country scores ranged from 32 to 89.  

Disclosures were generally comprehensive for the current year and at the total fund or investment option level. In contrast, reporting on longer time periods and asset class results were more often minimal or missing although more funds were observed disclosing intermediate (i.e. three to seven years) performance figures. 

The components with the highest scores continued to include asset mix and portfolio composition and risk policy and measures. Similarly the lowest scores were seen for asset class returns and value added and benchmark disclosures. 

The Canadian funds now lead the way, with an average country score of 89 for the performance factor. The US funds typically had extensive and good quality reporting across all performance components. All country rankings and scores were as follows.

The performance factor was assessed by looking at the following criteria

 

1. Total fund or investment option returns and value added.
Average country score: 76 (last year 75)

Scores in this component increased marginally to 76 from last year’s average score of 75. Current year disclosures continued to be almost universal. Scores dropped off for intermediate-term (any of 3 – 7 years) and long-term (10 years or more) disclosures. This year’s review found that 88 per cent of funds disclosed intermediate returns, an increase of 9 per cent from last year. Six countries had average scores over 90, an increase of two from last year. 

  • 100 per cent of all funds reported one-year total fund or investment option returns
  • 86 per cent of all funds reported one-year value added at the total fund or option level, down slightly from 88 per cent last year.
  • 86 per cent of all funds reported current performance (e.g., quarterly, monthly) on their websites, a slight increase from the 83 per cent observed last year.
  • Only 50 per cent of all funds reported long-term value added
      Questions relating to total fund or investment option returns and value added

      For defined benefit and insurance plans, questions are answered in respect of the total fund. For defined contribution funds, questions are applied to the largest investment options.

      • Are 1-year returns disclosed? 
      • Are intermediate returns disclosed (any of between 3 and 7-year)? 
      • Are long-term returns disclosed (10 years or longer)? 
      • Is 1-year value added and/or policy return/benchmark disclosed? 
      • Is intermediate value added and/or policy return/benchmark disclosed (any of between 3 and 7-year)? 
      • Is long-term value added and/or policy return/benchmark disclosed (10 years or longer) ?
      • Is there disclosure of ‘current’ (e.g. monthly or quarterly) performance?

      2. Asset class returns and value added.
      Average country score: 37 (last year 35)

      Similar to fund level disclosures, we observed a slight improvement in asset class level disclosures, with the average country score increasing from 35 to 37. We continue to be surprised by the contrast between the generally good total fund/investment option scores and the much lower asset class level scores. Asset mix decisions drive about 97 per cent of total fund or investment option results and asset class performance is fundamental to understanding overall results The US continued to lead the way with an average score of 100 but only three countries had scores of 60 or more. In contrast, seven countries had average scores less than 30, one more than last year. Individual fund scores ranged from 0 to 100. 

      • One-year asset class return disclosure continued to be the highlight: 76 per cent of all funds reported this, unchanged from last year.
      • 37 per cent of funds now report intermediate asset class returns, up from only 31 per cent last year.
      • Disclosures of long-term asset class performance and value added continued to be disappointing with only 23 per cent reporting performance and 18 per cent reporting value-added, both slight increases from last year.
        Questions relating to asset class and value added

        For defined benefit and insurance plans, questions are answered in respect of the total fund. For defined contribution funds, questions are applied to the largest investment options.

          • Are 1-year returns disclosed at the asset class level?
          • Are intermediate returns disclosed (any of between 3 and 7-year) at the asset class level?
          • Are long-term returns disclosed (10 years or longer) at the asset class level?
          • Are 1-year value added and/or benchmark returns disclosed at the asset class level?
          • Are intermediate value added and/or benchmark returns disclosed (any of between 3 and 7-year) at the asset class level?
          • Are long-term value added and/or benchmark returns disclosed (10 years or longer) at the asset class level?

        3. Clarity on basis for return and value-added calculations.

        Average country score: 63 (last year 64)

        To understand and compare total fund and asset class returns and value added it is essential that the basis for return calculations is also understood. Are the returns time-weighted or IRR? Are returns gross or net of investment costs (and all costs or only some costs)? This is not hard to do and some funds do provide concise and clear descriptions where they report returns and value added. Like last year, these disclosures were all too often non-existent, minimal, cryptic, or not provided where returns and value added were reported. The average country score declined slightly to 63 for 64 last year. The US led the way this year with an average score of 88. Switzerland, the top scoring country last year, regressed but was still one of four countries with average scores of 80 or more. The lowest average country score was 34. Individual fund scores ranged from 0 to 100.  

        • 89 per cent of funds disclosed the cost basis for returns and 83 per cent of all funds reported total fund or option returns net of investment costs, both relatively unchanged from last year.
        • In contrast, it was only clear to us that 57 per cent of all funds reported asset class returns net of costs, an improvement from 53% last year.
        • Last year, 60 per cent of all funds clearly reported that they were disclosing time-weighted returns (TWRR), which unfortunately dropped to only 53 per cent of funds this year.
          Questions relating to clarity on basis for return and value added calculations

          For defined benefit and insurance plans, questions are answered in respect of the total fund. For defined contribution funds, questions are applied to the largest investment options.

          • Is cost basis of return calculations disclosed?
          • Are total fund returns expressed net of investment costs? 
          • Are asset class returns expressed net of investment costs?
          • Is basis for return calculations disclosed (eg. IRR, TWRR)?
          • Are return calculations time weighted rates of return?
          • Is total fund net value added disclosed? 
          • Is asset class net value added disclosed?

          4. Benchmark disclosures.
          Average country score: 40 (last year 44)

          Interpreting and comparing of reported value added is challenging because of the diversity of benchmarks or targets utilised across funds. Approaches at the total fund or option level included: real return targets (e.g. CPI + 4 per cent); actuarial-based target returns; competitor returns for similar investment options; and aggregations of asset class passive indices. Positive scores were awarded for value added reporting when any type of benchmark or target return was used at the total fund or option level. But positive scores for value added reporting for public market asset classes were only awarded for what was considered ‘appropriate’ benchmarks. The benchmarks used for private market asset classes were not scored.

          US funds again led the way with an average score of 80, but only four countries had average scores of 60 or more.  

          • This year we scored if funds clearly disclosed how they determine their overall fund level benchmark. Unfortunately, only 16 per cent of funds reviewed did so with clarity.
          • More positively, 57 per cent of all funds had clear disclosures of their benchmarks at the asset class or option level and 44 per cent of all funds reported clear and appropriate benchmarks for public market asset classes. That said, both were slight declines from last year’s scores of 60 per cent and 49 per cent respectively.
          Questions relating to benchmark disclosures
          • Is there clear and specific disclosure of the benchmark composition at the total fund level?
          • Is there clear and specific disclosure of the benchmark composition at the asset class or option level?
          • Are most public market benchmarks appropriate (investable, generally match asset class)?

          5. Asset mix and portfolio composition.
          Average country score: 56 (last year 61)

          Asset mix and portfolio composition reporting provides important context that helps stakeholders understand portfolio diversification as well as the related returns and risks. These disclosures were generally quite extensive, but the communication quality was wide ranging. Some were uninspiring ‘data dumps’ (e.g. endless lists of portfolio holdings) whereas others provided helpful and insightful summaries and context.  

          This year’s review included two items that were previously reviewed in the governance factor. These were whether a fund discussed their philosophies around active/passive management and internal/external management. Disclosures of either were rare, with fewer than one in six funds reporting either.

          Scores in this component were consistently mediocre. Canada achieved the highest average country score at 76, with Sweden and the US the only other countries to score 70 or above. All but Mexico achieved average country scores above 40. 

          • All but one fund reported current asset mix, which was one fund less than last year’s review.
          • In contrast, only 56 per cent of all funds provided an asset mix trend (three or more years), a slight decline from the 59 per cent of funds that did so last year.
          • 71 per cent of all funds provided summaries of at least their largest holdings and many provided all holdings and 66 per cent of all funds provided details of their largest external managers and many provided all holdings, both slight improvements from last year’s review.
            Questions relating to asset mix and portfolio composition
            • Is the current asset mix disclosed across major asset classes or options?
            • Is the fund’s actual mix of active/passive strategies disclosed?
            • Is the fund’s rationale/philosophy for active/passive implementation disclosed?
            • Is the fund’s actual mix of internal/external strategies disclosed?
            • Is the fund’s rationale/philosophy for internal/external implementation disclosed?
            • Is there any asset mix trend disclosure?
            • Is there portfolio composition disclosure related to:
              • Geographic concentration overall or by asset class?
              • Holdings: list of largest holdings of individual companies, fixed income securities, unlisted individual investments, or disclosure of all holdings
              • List of largest external managers, for some asset classes, or disclosure of all external managers
              • Disclosure of other portfolio composition factors: e.g. % listed/unlisted, fixed income categories, derivative types, types of real estate and mortgage holdings, types of private equity, infrastructure. Score of 1 for each factor disclosed to a maximum of 3.

            6. Risk management policies and specific risk measures.
            Average country score: 77 (last year 82)

            This component was revised from last year’s review and now looked only at quantified risk statements. Questions of a qualitative nature, i.e. disclosures of risk practices and risk governance, are now reviewed in the Governance factor.  

            Disclosures in this area continued to be observed among most of the funds. Six countries had average scores of 90 or more and the lowest score was 37. 

            • 81 per cent of all funds provided quantified risk measures at the total fund/investment option or asset class level. Examples of such measures are value at risk (VAR), portfolio volatility, expected years with negative returns).
              Questions relating to risk management policies and specific risk measures
              • Is there disclosure of actual risk levels for the following factors:
                • Actual risk level disclosures for total portfolio or options, asset classes and/or risk ‘factors’. Examples of measures include: Value at Risk (VAR), volatility, tracking error, probability of negative return.
                • Disclosure of any additional risk factor measures. Examples include:. Counterparty risk, credit quality (ratings), liquidity, foreign exchange, derivatives exposure, portfolio concentration. Disclosures can be the specific exposure, or the policy adhered to in the portfolio . Score of 1 for each factor disclosed to a maximum of 3.

              7. Explanation of key results and outlook.
              Average country score: 77 (last year 72)

              This is a component that provides an opportunity for funds to improve the depth and quality of their disclosures. Discussions of results and the key factors driving them can provide important context for stakeholders. There were some very interesting and insightful discussions. Others were more cryptic and did not provide material additional information. The scoring system provided for an assessment of this ‘quality’ range.  

              The country average score improved from 72 last year to 77. The Dutch funds continued to lead the way in this area with an average country score of 96. Seven countries had average scores of 80 or more, two more than last year and the lowest score increased to 42, from 22 in last year’s review. 

               

              • 96 per cent of all funds discussed the impact of economic and market conditions on total fund or investment option key results over the past year, an increase of 5 per cent over last year’s review.
              • The percentage of funds that included a ‘looking ahead’ discussion of economic and market expectations and possible implications increased markedly, from only 61% of funds last year, to 88 per cent of funds this year.

                8. Member service goals, plans and service level reporting.
                Average country score: 60 (last year 65)

                76 per cent of individual funds reviewed had member service operations and 24 per cent did not, with member services typically provided by a different organisation that was not reviewed. Considering that pension funds exist for their members, we continue to be disappointed overall by the levels of disclosures by some funds. In contrast some member service disclosures were excellent. With a score of 100, Canada now has the highest average score, with Australia slipping to third overall. There continued to be a wide dispersion of results: three countries had average scores of 85 or more while five countries had average scores of 50 or less.  

                 

                • 69 per cent of funds with member service operations discussed member service goals, progress, and plans 
                • 63 per cent of funds with member service operations provided actual service level performance outcomes for member transactions or experiences (e.g., call answer time)
                • 53 per cent of funds with member service operations reported any of customer satisfaction, customer effort or net promoter scores.
                • 44 per cent of funds with member service operations discussed service levels with reference to external benchmarks
                  Questions relating to member service, if applicable

                  This section is applicable only to organizations who are also charged with administration of member pensions/benefits.

                    • Are any of: Customer satisfaction, customer effort or net promoter scores disclosed?
                    • Are any service performance metrics for member transactions or life events disclosed?
                    • Is member service performance explained with reference to external benchmarks?
                    • Are member service goals, progress and plans discussed?

                  9. Funded status and discussion of assumptions and risks.
                  Average country score: 92 (last year 96)

                  Funded status and related disclosures were relevant for only 56 per cent of funds studied: those with responsibility for both assets and liabilities for defined benefit plans and ‘insured’ benefit schemes. This is a very complex area plus regulatory regimes and rules are not uniform. Highly technical required reporting was the norm. Some funds did provide excellent reporting that made the material easier to digest and more relevant for non-actuaries. Scores were skewed high: seven countries had an average score of 100 and the lowest average country score was 60.  

                   

                  • All but one fund reported their period-ending funded ratio or solvency position 
                  • At 84 per cent, fewer of these funds disclosed trends in funded ratio or solvency position (3 or more years)
                  • 84 per cent of these funds discussed key assumptions underlying their funded status or solvency position (e.g., longevity, mortality, return and interest rate assumptions)
                    Questions relating to funded status and discussion of assumptions and risks

                    This section is applicable only to defined benefit and insurance organizations.

                      • Is the funded ratio or solvency position disclosed?
                      • Is there any disclosure of trends in funded ratio or solvency position?
                      • Are key assumptions used to determine funded ratio or solvency position disclosed (e.g. mortality table, return assumptions, interest rate)
                      • Is there discussion of sensitivity of funded ratio/solvency position to assumptions/scenario changes/returns?

                    To view all questions to each component, visit the Methodology page here.

                    Average score

                    Highest score

                    Performance questions asked

                    Overall Results Performance.

                    Year-on-Year Comparison

                    Performance Ranking

                    1.Canada

                    2.United States

                    3.Australia

                    4.Sweden

                    5.Japan

                    6.The Netherlands

                    7.South Africa

                    8.Finland

                    9.Norway

                    10.Switzerland

                    11.Brazil

                    United Kingdom

                    13.Denmark

                    14.Chile

                    15.Mexico

                    “However beautiful the strategy, you should occasionally look at the results.”

                    Sir Winston Churchill

                    Performance.

                    The assessment of performance disclosures included 47  questions organized across the components outlined below. 

                    Annual reports usually contained much of the information to be analysed. Some funds, and especially DC funds, also had extensive reporting on their websites. Occasionally, some disclosures were found in other documents such as: listings of investment holdings and external managers; investment policy statements; and risk policy statements.

                    The performance factor was assessed by looking at the following criteria:

                    1. Total fund (or investment option) and asset class returns and value added (25% of the performance factor score)
                    Generating adequate investment returns is fundamental to the success of all pension plans. Asset mix is the biggest driver of long-term returns. Most funds also implement active management programs, aiming to add value over passive public market implementation. The review of return and value-added disclosures covered timelines reported for total fund and asset class results: 1-year; intermediate (any of 3 to 7-year); long-term (10 years or more); and current (e.g. monthly, quarterly).

                    2. Clarity and quality of return and benchmark disclosures (17% of the performance factor score)
                    Evaluating returns and value added is difficult unless you understand the basis for the calculations and the composition of the benchmarks used. Important considerations include: Are returns gross or net of investment costs? Are asset class benchmarks appropriate and comparable?

                    3. Asset mix and portfolio composition disclosures (15% of the performance factor score)
                    Investment performance is driven by asset class exposures, implementation, and holdings of specific investments. Disclosures related to these portfolio factors enhance understanding of performance. Disclosures focused on included: current asset mix and trend; exposures and concentration by market, geography, holding, and manager; and implementation details within asset classes.

                    4. Explanation of risk management policies and specific risk disclosures (5% of the performance factor score)
                    Investment rewards and risk go hand in hand. Risk is unavoidable. But sound risk management policies and practices help organisations prepare for and manage adverse events. The disclosures of actual risk levels were evaluated (eg value at risk, liquidity, credit quality, foreign exchange exposure).

                     5. Explanation of key results and outlook (9% of the performance factor score)
                    Commentary on the economic and market conditions experienced adds color and depth to the understanding of results achieved. The analysis also looked for a ‘looking ahead’ discussion of economic and market conditions and possible implications.

                    6. Member service goals, plans and service level reporting (20% of the performance factor score when applicable)
                    The investment function was included within the mandate of all pension funds reviewed, but for some, the member service function was the responsibility of another organisation that was not reviewed. Where member service was part of the mandate, analysis was on disclosures relating to: goals, plans and progress; actual service levels achieved (eg call answer times, website features and service transaction volumes) and reporting of customer satisfaction, customer effort and net promoter scores.

                    7. Funded status disclosure discussion of assumptions and risks (9% of the performance factor score when applicable)
                    Some of the funds reviewed managed defined benefit assets and had responsibility for both sides of the pension balance sheet – assets and liabilities. For these funds we assessed disclosures related to funded status including disclosure and discussion of key assumptions and risks.