Change in mental health consumer's clinical outcomes, 2018

Identifying and definitional attributes

Metadata item type:Indicator
Indicator type:Indicator
Short name:Change in mental health consumer's clinical outcomes, 2018
METeOR identifier:692974
Registration status:Health, Candidate 09/08/2018
Description:

The proportion of mental health-related episodes of care where:

  • significant improvement
  • significant deterioration
  • no significant change

was identified between baseline and follow-up of completed outcome measures.

NOTE: This specification has been adapted from the indicator Change in mental health consumer's clinical outcomes, 2018– (Service level) using terminology consistent with the National Health Data Dictionary. There are technical differences in the consumer groups listed in the methodology section between the Service level version and the Jurisdictional level version of this indicator. Caution should be taken to ensure the correct methodology is followed.

Rationale:

State/territory specialised mental health services aim to reduce symptoms and improve functioning. Their effectiveness can be compared using routinely collected measures. This will assist in service benchmarking and quality improvement.

The implementation of routine mental health outcome measurement in Australia provides the opportunity to monitor the effectiveness of mental health services across jurisdictions.

Indicator set:Key Performance Indicators for Australian Public Mental Health Services (Jurisdictional level version) (2018) Health, Candidate 09/08/2018

Collection and usage attributes

Computation description:

Coverage/Scope:

State/territory specialised mental health service.

The following episodes of care are excluded:

Methodology:

  • Reference period for 2018 performance reporting: 2016–17.
  • Only the Health of the Nation Outcome Scales (HoNOS) family of measures (including HoNOS, HoNOS 65+ and HoNOS Children and Adolescents (HoNOSCA)) are considered in the calculation of this indicator.
  • Outcomes are calculated for the following consumer groups:
    • Group A: Consumers separated from hospital. All people who were discharged from an acute admitted patient mental health care service within the reference period. Scores should be calculated as the difference between the total score recorded at admission (the ‘baseline’) and discharge (the ‘follow-up’).
    • Group B: Consumers discharged from ambulatory care. All people who were discharged from an ambulatory care episode, provided by an ambulatory mental health care service, within the reference period. Scores should be calculated as the difference between the total score recorded at admission to the episode (the ‘baseline’), and discharge from the episode (the ‘follow-up’). Ambulatory episodes that are completed because the consumer was admitted to hospital or residential mental health care are excluded from the analysis that is, where the NOCC ‘reason for collection’ equals change of setting.
    • Group C: Consumers in ongoing ambulatory care. All people who have an ‘open’ ambulatory episode of care at the end of the reference period. Scores should be calculated as the difference between the total score recorded on the first occasion rated within the reference period which will be either admission or review, (the ‘baseline’) and the last occasion rated which will be a review (the ‘follow-up’) in the same reference period.
  • Group change analyses can only be determined for episodes of care where both baseline and follow-up ratings are present. This excludes specific episodes defined by the NOCC data collection protocol as not requiring follow-up as well as episodes where either the baseline or follow-up measure is not available.
  • The total score is determined for each individual baseline and follow-up score. This is the sum total of the 12 HoNOS/HoNOS65+ scales or the first 13 items of the 15 HoNOSCA. Where one or more of the HoNOS/HoNOS65+ or HoNOSCA items have not been completed correctly, the collection occasion should only be regarded as valid and complete if:
    • For the HoNOS and HoNOS65+, a minimum of 10 of the 12 items have a valid severity rating (i.e. a rating of either 0, 1, 2, 3 or 4)
    • For the HoNOSCA, a minimum of 11 of the first 13 items have a valid severity rating
    • There are no corrections made for missing items.
  • Scores are classified as outcomes measures, i.e.‘significant improvement’, ‘significant deterioration’ or ‘no significant change’, based on the effect size statistic.
  • The reference period for this indicator is typically a single financial year, and the impact of modifying the reference period is unknown.

Definition notes:

  • For the purposes of this key performance indicator (KPI), a medium effect size of 0.5 is used to assign outcome scores to the 3 outcome categories. A medium effect size is equivalent to an individual change score of at least one half (0.5) of a standard deviation.
  • Individual episodes are classified as either: ‘significant improvement’ if the effect size index is greater than or equal to positive 0.5; ‘significant deterioration’ if the effect size index is less than or equal to negative 0.5; or ‘no change’ if the index is greater than negative 0.5 and less than positive 0.5.
  • Analyses of the 2010–11 national pool of NOCC data identified that for:
    • Inpatient and ambulatory settings, a 4 point threshold indicates a medium effect size. In practice this means an individual change score of at least 4 points equates to a medium effect size and significant change. This threshold is the same for all 3 HoNOS measures.
Computation:

(Numerator ÷ Denominator) x 100

Calculated separately for each consumer group.

Numerator:

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by mental health setting, where significant improvement/significant deterioration/no significant change was identified between baseline and follow-up within the reference period.

Numerator data elements:
Data Element / Data SetData Element

Number of episodes with a completed outcome measure where significant improvement was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetData Element

Number of episodes with a completed outcome measure where significant deterioration was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetData Element

Number of episodes with a completed outcome measure where no significant change was identified

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale score code N
Guide for use

Data Source

National Outcomes and Casemix Collection (NOCC)

Data source type: Clinical outcome measure


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale 65+ score code N
Guide for use

Data source type: Clinical outcome measure

Data Source

National Outcomes and Casemix Collection (NOCC)


Data Element / Data SetPerson—level of psychiatric symptom severity, Health of the Nation Outcome Scale for Children and Adolescents score code N
Guide for use

Data source type: Clinical outcome measure

Data Source

National Outcomes and Casemix Collection (NOCC)


Denominator:

Number of completed or ongoing episodes of mental health care with completed outcome measures, partitioned by mental health setting within the reference period.

Denominator data elements:
Data Element / Data SetData Element

Total number of episodes with a completed outcome measure of significant improvement, no significant change or significant deterioration

Data Source

National Outcomes and Casemix Collection (NOCC)


Guide for use

Data source type: Clinical outcome measure


Disaggregation:

Service variables: setting (Group A, B or C).

Consumer attributes: age (0-17, 18-64, 65+).

Disaggregation data elements:
Data Element / Data SetPerson—age, total years N[NN]
Guide for use

Age to be calculated as at start of the episode of care.

Data Source

National Outcomes and Casemix Collection (NOCC)


Comments:

Coverage estimates for this indicator—that is the proportion of mental health care episodes with completed clinical outcome measures at both baseline and follow-up—were previously published in MHS KPI 14: Outcomes readiness. Following a review of the indicator set, Outcomes readiness will no longer be reported as a separate indicator but coverage estimates for both service variables and consumer attributes will be included in the reporting of the Change in mental health consumer's clinical outcomes.

Representational attributes

Representation class:Percentage
Data type:Real
Unit of measure:Service event
Format:

N[NN].N

Indicator conceptual framework

Framework and dimensions:Effective

Accountability attributes

Benchmark:

State/territory level

Further data development / collection required:

The NOCC, maintained by the Australian Government Department of Health compiles all state and territory consumer outcomes data on an annual data submission basis. However, the NOCC does not allow linkage of episodes of care across financial years. This limitation does not exist for states and territories own data sets.

Compared to data constructed from state and territory data sets and using complete episodes, this limitation is likely to result in calculation of the indicator based on a significantly smaller pool of episodes. It may also have a systematic effect on apparent results on this indicator, but the direction and size of this effect has not been systematically tested.

Other issues caveats:

Change in consumers’ clinical outcomes is not presented for states and territories with less than 200 unique consumer outcomes identifiable within the reference period. States and territories excluded from individual publication are used to calculate the change in consumers’ clinical outcomes for Australia.

Greater variability in HoNOS scores can be expected, if effect sizes were calculated using ‘locally’ derived standard deviations. Statistically, fewer observations generally give rise to greater variability.

Source and reference attributes

Submitting organisation:

Australian Institute of Health and Welfare on behalf of the National Mental Health Performance Subcommittee

Reference documents:

National Mental Health Performance Subcommittee (NMHPSC) 2013. Key Performance Indicators for Australian Public Mental Health Services, 3rd edn. Canberra: NMHPSC.

Relational attributes

Related metadata references:

See also Change in mental health consumer's clinical outcomes, 2018 Health, Candidate 09/08/2018

Supersedes KPIs for Australian Public Mental Health Services: PI 01J – Change in consumer's clinical outcomes, 2017 Health, Standard 14/06/2017