KPIs for Australian Public Mental Health Services: PI 01J – Change in mental health consumer's clinical outcomes, 2021

Coverage/Scope:

State/territory specialised mental health service.

The following episodes of care are excluded:

• brief ambulatory episodes of mental health care
• brief inpatient episodes of mental health care
• ambulatory episodes of mental health care that end because of admission to hospital or residential mental health care service (where the National Outcomes and Casemix Collection (NOCC) ‘reason for collection’ equals change of setting).

Methodology:

• Reference period for 2021 performance reporting: 2019–20.
• Only the Health of the Nation Outcome Scales (HoNOS) family of measures (including HoNOS, HoNOS 65+ and HoNOS Children and Adolescents (HoNOSCA)) are considered in the calculation of this indicator.
• Outcomes are calculated for the following consumer groups:
  • Group A: Consumers separated from hospital. All people who were discharged from an acute admitted patient mental health care service within the reference period. Scores should be calculated as the difference between the total score recorded at admission (the ‘baseline’) and discharge (the ‘follow-up’).
  • Group B: Consumers discharged from ambulatory care. All people who were discharged from an ambulatory care episode, provided by an ambulatory mental health care service, within the reference period. Scores should be calculated as the difference between the total score recorded at admission to the episode (the ‘baseline’), and discharge from the episode (the ‘follow-up’). Ambulatory episodes that are completed because the consumer was admitted to hospital or residential mental health care are excluded from the analysis that is, where the NOCC ‘reason for collection’ equals change of setting.
  • Group C: Consumers in ongoing ambulatory care. All people who have an ‘open’ ambulatory episode of care at the end of the reference period. Scores should be calculated as the difference between the total score recorded on the first occasion rated within the reference period which will be either admission or review, (the ‘baseline’) and the last occasion rated which will be a review (the ‘follow-up’) in the same reference period.
• Group change analyses can only be determined for episodes of care where both baseline and follow-up ratings are present. This excludes specific episodes defined by the NOCC data collection protocol as not requiring follow-up as well as episodes where either the baseline or follow-up measure is not available.
• The total score is determined for each individual baseline and follow-up score. This is the sum total of the 12 HoNOS/HoNOS65+ scales or the first 13 items of the 15 HoNOSCA. Where one or more of the HoNOS/HoNOS65+ or HoNOSCA items have not been completed correctly, the collection occasion should only be regarded as valid and complete if:
  • For the HoNOS and HoNOS65+, a minimum of 10 of the 12 items have a valid severity rating (i.e. a rating of either 0, 1, 2, 3 or 4)
  • For the HoNOSCA, a minimum of 11 of the first 13 items have a valid severity rating
  • There are no corrections made for missing items.
• Scores are classified as outcomes measures, i.e. ‘significant improvement’, ‘significant deterioration’ or ‘no significant change’, based on the effect size statistic.
• The reference period for this indicator is typically a single financial year, and the impact of modifying the reference period is unknown.

Definition notes:

• For the purposes of this key performance indicator (KPI), a medium effect size of 0.5 is used to assign outcome scores to the 3 outcome categories. A medium effect size is equivalent to an individual change score of at least one half (0.5) of a standard deviation.
• Individual episodes are classified as either: ‘significant improvement’ if the effect size index is greater than or equal to positive 0.5; ‘significant deterioration’ if the effect size index is less than or equal to negative 0.5; or ‘no change’ if the index is greater than negative 0.5 and less than positive 0.5.
• Analyses of the 2010–11 national pool of NOCC data identified that for:
  • Inpatient and ambulatory settings, a 4 point threshold indicates a medium effect size. In practice this means an individual change score of at least 4 points equates to a medium effect size and significant change. This threshold is the same for all 3 HoNOS measures.

(Numerator ÷ Denominator) x 100

Calculated separately for each consumer group.

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by mental health setting, where significant improvement/significant deterioration/no significant change was identified between baseline and follow-up within the reference period.

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by mental health setting within the reference period.

Service variables: setting (Group A, B or C).

Consumer attributes: age (0-17, 18-64, 65+).

Coverage estimates for this indicator—that is the proportion of mental health care episodes with completed clinical outcome measures at both baseline and follow-up—were previously published in MHS KPI 14: Outcomes readiness. Following a review of the indicator set, Outcomes readiness will no longer be reported as a separate indicator but coverage estimates for both service variables and consumer attributes will be included in the reporting of the Change in mental health consumer's clinical outcomes.

N[NN].N

State/territory level

The NOCC, maintained by the Australian Government Department of Health compiles all state and territory consumer outcomes data on an annual data submission basis. However, the NOCC does not allow linkage of episodes of care across financial years. This limitation does not exist for states and territories' own data sets.

Compared to data constructed from state and territory data sets and using complete episodes, this limitation is likely to result in calculation of the indicator based on a significantly smaller pool of episodes. It may also have a systematic effect on apparent results on this indicator, but the direction and size of this effect has not been systematically tested.

Change in consumers’ clinical outcomes is not presented for states and territories with less than 200 unique consumer outcomes identifiable within the reference period. States and territories excluded from individual publication are used to calculate the change in consumers’ clinical outcomes for Australia.

Greater variability in HoNOS scores can be expected, if effect sizes were calculated using ‘locally’ derived standard deviations. Statistically, fewer observations generally give rise to greater variability.

National Mental Health Performance Subcommittee (NMHPSC) 2013. Key Performance Indicators for Australian Public Mental Health Services, 3rd edn. Canberra: NMHPSC.