Australian Government: Australian Institute of Health and Welfare METeOR Home Page

Data quality statement (Specialist Homelessness Services Collection—December quarter 2011)

Identifying and definitional attributes

Metadata item type:Help on this termQuality Statement
METeOR identifier:Help on this term489216
Registration status:Help on this termAIHW Data Quality Statements, Archived 15/12/2014

Data quality

Quality statement summary:Help on this term

Summary of key data quality issues

  • The Specialist Homelessness Services Collection (SHSC) replaces the Supported Accommodation Assistance Program National Data Collection (SAAP NDC). There are significant differences between the two, creating comparability issues.
  • Analysis of the December quarter 2011 SHSC data identified some data quality issues. In particular, the rate of invalid/’don’t know’/missing responses was high for many data items in the December quarter, as it was in the September quarter. Data items with very high rates of invalid/’don’t know’/missing responses are not reported on in this publication.
  • Some new agencies (that did not participate in the September quarter) began submitting data in the December quarter 2011. This explains, to some extent, the lack of improvement in the rate of missing responses between the September and December quarters.
  • All agencies that receive funding under the National Affordable Housing Agreement (NAHA) or the National Partnership Agreement on Homelessness (NPAH) to provide specialist homelessness services are in scope for the SHSC, but only those expected to provide data are covered. Many high volume services and intake services are not covered.
  • 92% of covered agencies returned support period data for the December quarter 2011,
    although some reported for only one or two months. This is a one percentage point
    decrease from the September quarter 2011. The increase in the agencies expected to
    submit data for the December quarter 2011 may account for the decrease in participation.
  • Matching of data from individual clients who presented at different agencies and/or at
    different times requires a valid statistical linkage key (SLK). 93% of support periods had
    a valid SLK in the December quarter 2011.

Description

The SHSC collects information on people who receive services from agencies that receive funding under the NAHA or the NPAH to provide specialist homelessness services. A limited amount of data is also collected about clients who seek, but do not receive, assistance from a specialist homelessness agency.

Data are collected monthly from agencies participating in the collection.

The SHSC replaced the SAAP NDC on July 1 2011. There are significant differences between
the SAAP NDC and the SHSC.

Institutional environment:Help on this term

The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent statutory authority established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health and Ageing portfolio.

The AIHW aims to provide authoritative information and statistics to promote better health and wellbeing. The Institute collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.

The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting. One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national data sets based on data from each jurisdiction, to analyse these data sets and disseminate information and statistics.

The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988, ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality. For further information see the AIHW website www.aihw.gov.au.The SHSC was developed by AIHW in conjunction with the states and territories and is being administered by the AIHW.

Timeliness:Help on this term

The SHSC began on 1 July 2011. Under the collection guidelines, specialist homelessness agencies provide their data to the AIHW each month, although delays in the provision of data from agencies do occur. Once validated, submitted data are regularly loaded to a Master Database. From this Master Database ‘snapshots’ are created at particular points in time for reporting purposes. Snapshots are taken following the end of each quarter, and another one for the whole collection year for annual reporting. From 2012–13, it is planned to publish quarterly results by the end of the following quarter, and annual results in October each year.

The December quarter 2011 snapshot contains data submitted to the AIHW for the October, November and December 2011 collection months, using responses received and validated as at 2 May 2012 when the snapshot was taken.

Accessibility:Help on this term

Published results from the December quarter 2011 are available in this report and elsewhere on the AIHW website.

Data not available online or in reports can be obtained from the Communications, Media and Marketing Unit on (02) 6244 1032 or via email to info@aihw.gov.au. Requests that take longer than half an hour to compile are charged for on a cost-recovery basis.

Interpretability:Help on this term

Information on the development of the SHSC, definitions and concepts, and collection materials and processes can be found on the AIHW website, www.aihw.gov.au. Information on definitions, concepts and classifications can also be found in the SHSC’s collection manual (AIHW 2011b).

Relevance:Help on this term

Scope and coverage

The SHSC is a key source of information for measuring the outcomes and outputs for people who are homeless, or at risk of homelessness, who seek specialist homelessness services.

The SHSC collects, via specialist homelessness agencies, information on people who seek and receive services from those agencies. All agencies that receive funding under the NAHA or NPAH to provide specialist homelessness services are in scope for the SHSC. However, only those identified by state and territory departments responsible for the provision of services are covered, because some ‘high volume’ services that provide short-term interventions (e.g. referral, meals) may not be expected to participate. In this quarter (December quarter 2011), 1,478 agencies were in coverage.

Not everyone in scope for SHSC is homeless, because specialist homelessness agencies provide services to people at risk of homelessness, as well as to people who are currently homeless.

Not all homeless people and people at risk of homelessness are in scope for the SHSC―only those who seek services from specialist homelessness agencies are in scope. In the client collection, data are collected by homelessness agency workers for each client support period. Some basic information is also collected on instances where people seek, but do not receive, assistance from a homelessness agency (‘unassisted people’).

Reference period

SHSC data are reported to the AIHW every month, and, once validated, these data are loaded to a Master Database. Snapshots of this database are created at particular points in time for reporting purposes. Snapshots are taken following the end of each quarter for quarterly reporting.

The ‘December quarter 2011’ data refers to data for October, November and December, 2011. It covers support periods active in at least one of those months: clients who had an active support period in at least one of those months, and unassisted people who sought services in one of those months. Information on unassisted people is not presented in this report due to issues with the quality of these data arising from inconsistent interpretation of the data concepts.

Geographic detail

Data are published at the national and state/territory level.

Statistical standards

A client is defined in the SHSC as a person who receives a specialist homelessness service—assistance provided to a client aimed at responding to or preventing homelessness. A client may be of any age—children are clients if they receive a service from a specialist homelessness service.

A support period is defined in the SHSC as a period of support provided by a specialist homelessness service agency to a client.

Standard Australian Classification of Countries 2008 (ABS 2008) codes were used as the code-frame for questions relating to country of birth.

Australian Classification for Source of Income 2010 (ABS 2010b) codes were used as the codeframe for questions relating to a client’s source of income.

Accuracy:Help on this term

Potential sources of error

As with all data collections, the SHSC estimates are subject to errors. These can arise from data coding and processing errors, inaccurate data or missing data. Reported findings are based on data reported by agency workers.

Data validation

The AIHW receives data from specialist homelessness agencies every month. These data go through two processes of data validation, that is, error checking. Firstly, data validation is incorporated into the client management systems (CMSs) most agencies use to record their data. Secondly, data are submitted through the AIHW online reporting web-portal, Specialist Homelessness Online Reporting (SHOR). SHOR completes a more thorough data validation and reports (to staff of the homelessness agency) any errors that need correcting before data can be submitted.

Agency participation

Ninety-two per cent of agencies returned support period data for at least one month in the December quarter 2011. This is one percentage point lower than in the September quarter 2011. The decrease may be due to an increase in the number of agencies covered for the December quarter 2011—of relevant agencies that were expected to participate in the September quarter, 93% participated in the December quarter.

Statistical Linkage Key (SLK) validity

An individual client may seek or receive support on more than one occasion—either from the same agency or from a different agency. Data from individual clients who presented at different agencies and/or at different times is matched based on a statistical linkage key (SLK) which allows client level data to be created. The SLK is constructed from information about the client’s date of birth, sex and an alphacode based on selected letters of their name. If a support period record does not have a valid SLK, it cannot be linked to a client, and thus it is not included in client-level tables (although it is included in support period-level tables). In the December quarter 2011, 93% of support periods had a valid SLK—the same as in the September quarter.

Incomplete responses

In many support periods, in the December quarter 2011, valid responses were not recorded for all questions—invalid responses were recorded, ‘don’t know’ was selected, or no response was recorded.

For example:

  • ‘facilities/institutions the client has been in in the last 12 months’ and ‘time period the client received assistance for their mental health issue’ have the highest rates of invalid/’don’t know’/missing response―53% and 52%, respectively
  • ‘dwelling type at presentation’ and the outcome variable ‘dwelling type at the last service date in the reporting period’ have invalid/missing/don’t know response rates of 26% and 25%, respectively
  • ‘main reason for seeking assistance’ has an invalid/missing/don’t know response rate of 17%.

The rate of invalid/’don’t know’/missing responses in the December quarter 2011 was
similar to that in the September quarter.

Analysis indicates that support periods in agencies in Victoria and the ACT (which have a
central intake model), in agencies that provide referral services, and in agencies that did not
participate in the collection in the September quarter, are more likely to have many missing
responses. Many intake services in Victoria that did not participate in the September quarter,
began submitting data in the December quarter 2011. This may explains why the rate of
missing responses did not decrease in the December quarter.

Support periods with invalid/’don’t know’/missing responses were retained in the
collection and, due to the difficulty of doing so accurately, no attempt was made to deduce
or impute the true value of invalid/’don’t know’/missing responses.

Data items with very high rates of invalid/’don’t know’/missing responses were not
reported on in this publication.

Non-response bias

Less than 100% agency participation, less than 100% SLK validity and a high rate of incomplete responses do not necessarily mean that estimates are biased. If the nonrespondents are not systematically different in terms of how they would have answered the questions, then there is no bias. Given the results of analyses of agency participation, SLK validity and incomplete responses performed to date, some non-response bias is expected.

Non-response adjustment

As some data are missing because of agency non-participation and SLK invalidity, nonresponse adjustment (or ‘weighting’) has been applied to create a selection of weighted estimates. Only a limited number of estimates were amenable to non-response adjustment in the December quarter 2011.

A more complete method for adjusting for missing information will be applied to annual data; the method used in the December quarter 2011 is a simplified version of that method.

Non-response adjustment in the SHSC is performed for support periods and, separately, for clients. The process used is described in Appendix A.

The process accounts only for agency non-participation and SLK invalidity―due the difficulty of doing so accurately, no adjustments are made for incomplete responses.

In the December quarter 2011, non-response adjustment was performed at the national level only. This may lead to biased estimates, because analysis has shown that agency participation and SLK validity vary with characteristics such as state/territory, age group and sex, and the non-response adjustment process does not account for this.

The non-response adjustment process used in the December quarter 2011, which designates agencies ‘participating’ if they returned support period data at any time during the reporting period, causes weighted estimates to be, on average, underestimates.

Improvements to the weighting methodology will be implemented during 2011–12. In particular, the method used to weight the annual data will be much more extensive than the method used for the December quarter 2011 data. This will lead to comparability issues, as different weighting methodologies give different estimates for the same raw data.

Coherence:Help on this term

The SHSC replaces the SAAP NDC, which began in 1996. The SHSC differs from the SAAP NDC in many respects.

The major definitional differences between SAAP and SHSC relate to the capture of information about children and support. In the SAAP NDC, children who accompanied a parent or guardian were counted as accompanying children; in the SHSC, children are included as clients (in their own right) if they directly receive a service. In SAAP, support was considered to entail generally 1 hour or more of a worker’s time; in SHSC no timerelated condition exists. Further information on the comparability of SHSC and SAAP can be found in the report outlining findings from the September quarter 2011 (see Appendix A, AIHW 2012).

Comparison with other collections

The other major data sources on homelessness are:

  • The ABS Census, which collects data from all persons in Australia on Census night,
    including data allowing respondents’ homelessness status to be derived (see
    Chamberlain & Mackenzie 2008). The cultural definition of homelessness is used, which
    identifies three homelessness categories:
    • Primary homelessness includes all people without conventional accommodation.
    • Secondary homelessness includes people who move frequently from one form of temporary shelter to another, including all people staying in emergency or transitional accommodation provided by specialist homelessness services; people residing temporarily with other households because they have no accommodation of their own; and people staying in boarding houses on a short-term basis.
    • Tertiary homelessness refers to people who live in boarding houses on a medium-to long-term basis (Chamberlain & MacKenzie 2008).

The cultural definition of homelessness does not define ‘at risk of homelessness’.

Data are currently available for 2001 and 2006.

  • The ABS General Social Survey’s homelessness module (ABS 2010a), which collects data from usual residents of private dwellings, including data on whether respondents have ever been homeless. The survey defines homelessness as being without a permanent place to live for a selection of reasons. Data are currently available for 2010 only.
  • The National Census of Homelessness School Students, which collects data on homeless school students via principals of all government and Catholic secondary schools (Mackenzie & Chamberlain 2008). Both the cultural definition of homelessness and a service delivery definition are used. Data are currently available for 2006 and selected prior years. Because these collections differ greatly from SHSC in scope, collection methodology, definitions and reference periods, comparisons between collections should be made with caution.

Data products

Implementation start date:Help on this term04/07/2012

Source and reference attributes

Submitting organisation:Help on this term

Australian Institute of Health and Welfare

Relational attributes

Related metadata references:Help on this term

Has been superseded by Specialist Homelessness Services Collection Data Quality Statement 2013-14 AIHW Data Quality Statements, Archived 14/12/2016

Has been superseded by Specialist Homelessness Services Collection—September quarter 2011; Data Quality Statement AIHW Data Quality Statements, Archived 04/07/2012

Supersedes Specialist Homelessness Services Collection—September quarter 2011; Data Quality Statement AIHW Data Quality Statements, Archived 04/07/2012

My items Help on this term
Download Help on this term