Australian Government: Australian Institute of Health and Welfare METeOR Home Page

KPIs for Australian Public Mental Health Services: PI 01J – Change in consumer's clinical outcomes, 2017

Identifying and definitional attributes

Metadata item type:Help on this termIndicator
Indicator type:Help on this termIndicator
Short name:Help on this termMHS PI 01J: Change in consumer's clinical outcomes, 2017
METeOR identifier:Help on this term663800
Registration status:Help on this termHealth, Standard 14/06/2017
Description:Help on this term

The proportion of mental health-related episodes of care where:

  • significant improvement
  • significant deterioration
  • no significant change

was identified between baseline and follow-up of completed outcome measures.

NOTE: This specification has been adapted from the indicator MHS PI 01: Change in consumer's clinical outcomes, 2015– (Service level version) using terminology consistent with the National Health Data Dictionary. There are technical differences in the consumer groups listed in the methodology section between the Service level version and the Jurisdictional level version of this indicator. Caution should be taken to ensure the correct methodology is followed.

Rationale:Help on this term

State/territory specialised mental health services aim to reduce symptoms and improve functioning. Their effectiveness can be compared using routinely collected measures. This will assist in service benchmarking and quality improvement.

The implementation of routine mental health outcome measurement in Australia provides the opportunity to monitor the effectiveness of mental health services across jurisdictions.

Indicator set:Help on this termKey Performance Indicators for Australian Public Mental Health Services (Jurisdictional level version) (2017) Health, Standard 14/06/2017

Collection and usage attributes

Computation description:Help on this term

Coverage/Scope:

State/territory Specialised mental health service.

The following episodes of care are excluded:

Methodology:

  • Reference period for 2017 performance reporting: 2015–16.
  • Only the Health of the Nation Outcome Scale (HoNOS) family of measures (including HoNOS, HoNOS 65+ and HoNOS Children and Adolescents) is considered in the calculation of this indicator.
  • Outcomes are calculated for the following consumer groups:
    • Group A: Consumers discharged from hospital. All people who were discharged from an acute admitted patient mental health care service within the reference period. Scores should be calculated as the difference between the total score recorded at admission (the ‘baseline’) and discharge (the ‘follow-up’).
    • Group B: Consumers discharged from ambulatory care. All people who were discharged from an ambulatory care episode, provided by an ambulatory mental health care service, within the reference period. Scores should be calculated as the difference between the total score recorded at admission to the episode (the ‘baseline’), and discharge from the episode (the ‘follow-up’). Ambulatory episodes that are completed because the consumer was admitted to hospital or residential mental health care are excluded from the analysis that is, where the National Outcomes and Casemix Collection (NOCC) ‘reason for collection’ equals change of setting.
    • Group C: Consumers in ongoing ambulatory care. All people who have an ‘open’ ambulatory episode of care at the end of the reference period. Scores should be calculated as the difference between the total score recorded on the first occasion rated within the reference period which will be either admission or review, (the ‘baseline’) and the last occasion rated which will be a review (the ‘follow-up’) in the same reference period.
  • Group change analyses can only be determined for episodes of care where both baseline and follow-up ratings are present. This excludes specific episodes defined by the NOCC data collection protocol as not requiring follow-up as well as episodes where either the baseline or follow-up measure is not available.
  • The total score is determined for each individual baseline and follow-up score. This is the sum total of the 12 HoNOS/65+ scales or the first 13 items of the 15 HoNOS Children and Adolescents (HoNOSCA). Where one or more of the HoNOS/65+ or HoNOSCA items have not been completed correctly, the collection occasion should only be regarded as valid and complete if:
    • For the HoNOS and HoNOS 65+: A minimum of 10 of the 12 items have a valid severity rating (i.e. a rating of either 0, 1, 2, 3 or 4)
    • For the HoNOSCA a minimum of 11 of the first 13 items have a valid severity rating
    • There are no corrections made for missing items.
  • Scores are classified as either ‘significant improvement’, ‘significant deterioration’ or ‘no significant change’, based on the effect size statistic.
  • The reference period for this indicator is typically a single financial year, and the impact of modifying the reference period is unknown.

Definition notes:

  • For the purposes of this key performance indicator (KPI), a medium effect size of 0.5 is used to assign outcome scores to the 3 outcome categories. A medium effect size is equivalent to an individual change score of at least one half (0.5) of a standard deviation.
  • Individual episodes are classified as either: ‘significant improvement’ if the effect size index is greater than or equal to positive 0.5; ‘significant deterioration’ if the effect size index is less than or equal to negative 0.5; or ‘no change’ if the index is greater than -0.5 and less than 0.5.
  • Analyses of the 2010–11 national pool of NOCC data identified that for:
    • Inpatient and ambulatory settings a four point threshold indicates a medium effect size. In practice this means an individual change score of a least four points equates to a medium effect size and significant change. This threshold is the same for all 3 HoNOS measures.
Computation:Help on this term

(Numerator ÷ Denominator) x 100

Calculated separately for each consumer group.

Numerator:Help on this term

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by setting, where significant improvement/significant deterioration/no significant change was identified between baseline and follow-up within the reference period.

Numerator data elements:Help on this term
Data Element / Data SetData Element

Number of episodes with a completed outcome measure where significant improvement was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetData Element

Number of episodes with a completed outcome measure where significant deterioration was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetData Element

Number of episodes with a completed outcome measure where no significant change was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale score code N
Guide for use

Data Source

National Outcomes and Casemix Collection (NOCC)

Data source type: Clinical outcome measure


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale 65+ score code N
Guide for use

Data source type: Clinical outcome measure

Data Source

National Outcomes and Casemix Collection (NOCC)


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale for Children and Adolescents score code N
Guide for use

Data source type: Clinical outcome measure

Data Source

National Outcomes and Casemix Collection (NOCC)


Denominator:Help on this term

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by setting within the reference period.

Denominator data elements:Help on this term
Data Element / Data SetData Element

Total number of episodes with a completed outcome measure of significant improvement, no significant change or significant deterioration

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Disaggregation:Help on this term

Service variables: Setting (Group A, B or C)

Consumer attributes: Age (0-17, 18-64, 65+)

Disaggregation data elements:Help on this term
Data Element / Data SetPerson—age, total years N[NN]
Guide for use

Age to be calculated as at start of the episode of care.

Data Source

National Outcomes and Casemix Collection (NOCC)


Representational attributes

Representation class:Help on this termPercentage
Data type:Help on this termReal
Unit of measure:Help on this termService event
Format:Help on this term

N[NN].N

Indicator conceptual framework

Framework and dimensions:Help on this termEffective

Accountability attributes

Benchmark:Help on this term

State/territory level

Further data development / collection required:Help on this term

The NOCC, maintained by the Australian Government Department of Health compiles all state and territory consumer outcomes data on an annual data submission basis. However, NOCC does not allow linkage of episodes of care across financial years. This limitation does not exist for states and territories own data sets.

Compared to data constructed from state and territory datasets and using complete episodes, this limitation is likely to result in calculation of the indicator based on a significantly smaller pool of episodes. It may also have a systematic effect on apparent results on this indicator, but the direction and size of this effect has not been systematically tested.

Other issues caveats:Help on this term

Change in consumers’ clinical outcomes is not presented for states and territories with less than 200 unique consumer outcomes identifiable within the reference period. States and territories excluded from individual publication are used to calculate the change in consumers’ clinical outcomes for Australia.

Greater variability in HoNOS scores can be expected, if effect sizes were calculated using ‘locally’ derived standard deviations. Statistically, fewer observations generally give rise to greater variability.

Source and reference attributes

Submitting organisation:Help on this term

Australian Institute of Health and Welfare on behalf of the National Mental Health Performance Subcommittee

Reference documents:Help on this term

National Mental Health Performance Subcommittee 2013. Key Performance Indicators for Australian Public Mental Health Services, 3rd edn. Canberra: NMHPC.

Relational attributes

Related metadata references:Help on this term

Has been superseded by Change in mental health consumer's clinical outcomes, 2018 Health, Candidate 09/08/2018

Supersedes KPIs for Australian Public Mental Health Services: PI 01J – Change in consumer's clinical outcomes, 2016 Health, Superseded 14/06/2017

My items Help on this term
Download Help on this term