Australian Government: Australian Institute of Health and Welfare METeOR Home Page

Specialist Homelessness Services Collection—September quarter 2011; Data Quality Statement

Identifying and definitional attributes

Metadata item type:Help on this termQuality Statement
METeOR identifier:Help on this term480785
Registration status:Help on this termAIHW Data Quality Statements, Archived 04/07/2012

Data quality

Quality statement summary:Help on this term
  • The Specialist Homelessness Services Collection (SHSC) replaces the Supported Accommodation Assistance Program (SAAP) National Data Collection (NDC). There are significant differences between the two, creating comparability issues.
  • Analysis of the September quarter 2011 SHSC data identified some implementation issues. In particular, the rate of invalid/’don’t know’/missing responses was high for many data items. Data items with very high rates of invalid/’don’t know’/missing responses are not reported on in this publication.
  • All agencies that receive funding under the National Affordable Housing Agreement (NAHA) or the National Partnership Agreement on Homelessness (NPAH) to provide specialist homelessness services are in scope for the SHSC, but only those expected to provide data are covered. Many high volume services and intake services are not covered, nor are Transitional Housing Management (THM) services in Victoria, which had not commenced reporting SHSC data in the September quarter 2011.
  • 93% of covered agencies returned support period data for the September quarter 2011, although some reported only for one or two months.
  • Matching of data from individual clients who presented at different agencies and/or at different times requires a valid statistical linkage key (SLK). 93% of support periods had a valid SLK in the September quarter 2011.

Description

The SHSC collects information on people seeking services from agencies that receive funding under the NAHA or the NPAH to provide specialist homelessness services. Data is collected monthly from agencies participating in the collection.
The SHSC replaced the SAAP NDC on July 1 2011. There are significant differences between the SAAP NDC and the SHSC.

Institutional environment:Help on this term

The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent statutory authority established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health and Ageing portfolio.

The AIHW aims to provide authoritative information and statistics to promote better health and wellbeing. The Institute collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.

The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.

One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.
The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988, ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality.

For further information see the AIHW website www.aihw.gov.au.

The AIHW has been the Data Custodian for the SAAP NDC since 1996. The SHSC was developed by AIHW in conjunction with the states and territories and is being administered by the AIHW.

Timeliness:Help on this term

The SHSC began collecting data from 1 July 2011. The SHSC collects information from all participating agencies every month. Data collected is regularly loaded to a Master Database. From this Master Database ‘snapshots’ are created at particular points in time for reporting purposes. Snapshots will be taken following the end of each quarter, and another one for the whole collection year for annual reporting. It is planned, from 2012–13, to publish quarterly results by the end of the following quarter, and annual results in October each year.

The September quarter 2011 snapshot contains data submitted to the AIHW for the July, August and September 2011 collection months, using responses received and validated as at 28 February 2012 when the snapshot was taken.

Accessibility:Help on this term

Published results from the SHSC first quarter are available elsewhere in this report and on the AIHW website.

Data not available online or in reports can be obtained from the Communications, Media and Marketing Unit on (02) 6244 1032 or via email to info@aihw.gov.au. Requests that take longer than half an hour to compile are charged for on a cost-recovery basis.

Interpretability:Help on this term

Information on the development of the SHSC, definitions and concepts, and collection materials and processes can be found on the AIHW website, www.aihw.gov.au. Information on definitions, concepts and classifications can also be found in the SHSC’s collection manual (AIHW 2011b).

Relevance:Help on this term

Scope and coverage

The SHSC is a key source of information for measuring the outcomes and outputs for people who are homeless, or at risk of homelessness, who seek specialist homelessness services.

The SHSC collects, via specialist homelessness agencies, information on people seeking services from those agencies. All agencies that receive funding under the NAHA or NPAH to provide specialist homelessness services are in scope for the SHSC. However, only those expected to provide data are covered. Victorian Transitional Housing Management (THM) agencies, as well as many high volume services and intake services, are not covered, as it is difficult for such agencies to report. In the first quarter, 1,460 agencies were covered.

Not everyone in scope for SHSC is homeless, as specialist homelessness agencies provide services to people at risk of homelessness, as well as people who are currently homeless.

Not all homeless people and people at risk of homelessness are in scope for the SHSC―only those who seek services from specialist homelessness agencies are in scope.

Data are collected by homelessness agency workers for each client support period. Some basic information is also collected on instances where people seek, but do not receive assistance, from a homelessness agency (‘unassisted people).

Reference period

The SHSC collects information every month, the data is regularly loaded to a Master Database, and snapshots of this database are created a

Accuracy:Help on this term

Potential sources of error

As with all data collections, the SHSC estimates are subject to errors. These can arise from data coding and processing errors, inaccurate data, or missing data. Reported findings are based on data reported by agency workers.

Data validation

The AIHW receives data from specialist homelessness agencies every month. This data goes through two processes of data validation, that is, error checking. Firstly, data validation is incorporated into the client management systems (CMSs) most agencies use to record their data. Secondly, data is submitted through the AIHW online reporting web-portal, Specialist Homelessness Online Reporting (SHOR). SHOR completes a more thorough data validation and reports any errors that need correcting before data can be submitted, to staff of the homelessness agency.

The SHOR data validation system was not fully functional during the September quarter 2011, as more work was needed to ensure edit checks were appropriate to the data being received. However, the data validation systems in CMSs were fully functional; and testing of the data received indicates that erroneous data is minimal, although some invalid values are present.

Agency participation

93% of relevant agencies returned support period data for at least one month in the September quarter 2011. This compares favourably with the SAAP agency participation rate, which was 90% in 2010–11 and 92% on average between 2001–02 and 2010–11.

Statistical Linkage Key (SLK) validity

An individual client may seek or receive support on more than one occasion—either from the same agency or from a different agency. Data from individual clients who presented at different agencies and/or at different times is matched based on a statistical linkage key (SLK) which is constructed from information about the client’s date of birth, sex and an alphacode based on selected letters of their name, allowing client level data to be created.

If a support period record does not have a valid SLK, it cannot be linked to a client, and thus it is not included in client-level tables (although it is included in support period-level tables). In the September quarter 2011, 93% of support periods had a valid SLK.

Incomplete responses

In many support periods, in the September quarter 2011, valid responses were not recorded for all questions, because invalid responses were recorded, ‘don’t know’ was selected, or no response was recorded.

The rate of invalid/’don’t know’/missing responses was higher than expected for many data items. For example:

  • ‘time period the client received assistance for their mental health issue’ and ‘facilities/institutions the client has been in in the last 12 months’ have the highest rates of invalid/’don’t know’/missing response―51% each
  • ‘dwelling type at presentation’ and the outcome variable ‘dwelling type at the last service date in the reporting period’ have invalid/missing/don’t know response rates of 26% and 29% respectively
  • ‘main reason for seeking assistance’ has an invalid/missing/don’t know response rate of 17%.

‘Sex’ and ‘date of birth’ have some of the lowest rates of invalid/missing/ don’t know response―less than 1% and 7% respectively.

Support periods with invalid/’don’t know’/missing responses were retained in the collection, and, due to the difficulty of doing so accurately, no attempt was made to deduce or impute the true value of invalid/’don’t know’/missing responses.

Data items with very high rates of invalid/’don’t know’/missing responses were not reported on in this publication.

Non-response bias

Less than 100% agency participation, less than 100% SLK validity and a high rate of incomplete responses do not necessarily mean that estimates are biased. The non-respondents are not systematically different in terms of how they would have answered the questions, then there is no bias. Given the results of analyses of agency participation, SLK validity and incomplete responses performed to date, some non-response bias is expected.

Non-response adjustment

As some data is missing because of agency non-participation and SLK invalidity, non-response adjustment (or ‘weighting’) has been applied to create a selection of weighted estimates. Only a limited number of estimates were amenable to non-response adjustment in the September quarter 2011.

A more complete method for adjusting for missing information will be applied to annual data; the method used in the September quarter 2011 is a simplified version of that method.

Non-response adjustment in the SHSC is performed for support periods, and, separately, for clients. The process used is described in Appendix C.

The process accounts only for agency non-participation and SLK invalidity―due the difficulty of doing so accurately, no adjustments are made for incomplete responses.

In the September quarter 2011, non-response adjustment was performed at the national level only. This may lead to biased estimates, as analysis has shown that agency participation and SLK validity vary with characteristics such as state/territory, age group and sex, and the non-response adjustment process does not account for this.

The non-response adjustment process used in the September quarter 2011, which designates agencies ‘participating’ if they returned support period data at any time during the reporting period, causes weighted estimates to be, on average, underestimates.

Improvements to the weighting methodology will be implemented during 2011–12. In particular, the method used to weight the annual data will be much more extensive than the method used for the September quarter 2011 data. This will lead to comparability issues, as different weighting methodologies give different estimates for the same raw data.

Coherence:Help on this term

The SHSC replaces the SAAP NDC, which began in 1996. The SHSC differs from the SAAP NDC in many respects.

The major definitional differences between SAAP and SHSC concern children and support. In the SAAP NDC, children who accompanied a parent or guardian were included as accompanying children; in the SHSC, children are included as clients if they directly receive a service. In SAAP, support was considered to entail generally one hour or more of a worker’s time; in SHSC no time-related condition exists. Further information on the comparability of SHSC and SAAP can be found in Appendix A.

The cultural definition of homelessness

In the SHSC, homelessness is defined living in non-conventional accommodation or living in short-term or emergency accommodation due to a lack of options. Some other collections instead define homelessness using the cultural definition, which delineates three homelessness categories:

  • Primary homelessness includes all people without conventional accommodation.
  • Secondary homelessness includes people who move frequently from one form of temporary shelter to another, including all people staying in emergency or transitional accommodation provided under SAAP; people residing temporarily with other households because they have no accommodation of their own; and people staying in boarding houses on a short-term basis.
  • Tertiary homelessness refers to people who live in boarding houses on a medium to long-term basis (Chamberlain & MacKenzie 2008).

The cultural definition of homelessness does not define ‘at risk of homelessness’.

Comparison with other collections

The other major data sources on homelessness are:

The ABS Census, which collects data from all persons in Australia on Census night, including data allowing respondents’ homelessness status to be derived (see ABS 2006). The cultural definition of homelessness is used. Data is currently available for 2001 and 2006.

The ABS General Social Survey’s homelessness module (ABS 2010a), which collects data from usual residents of private dwellings, including data on whether respondents have ever been homeless. The survey defines homelessness as being without a permanent place to live for a selection of reasons. Data is currently available for 2010 only.

Chamberlain and Mackenzie’s National Census of Homelessness School Students, which collects data on homeless school students via principals of all government and Catholic secondary schools (FaHCSIA 2006). Both the cultural definition of homelessness and a service delivery definition are used. Data is currently available for 2006 and selected prior years.

As these collections differ greatly from SHSC in scope, collection methodology, definitions and reference periods, comparisons between collections should be made with caution.

Source and reference attributes

Submitting organisation:Help on this term

Australian Institute of Health and Welfare

Reference documents:Help on this term

Australian Institute of Health and Welfare (2012). Specialist Homelessness Services Collection: first results. Cat. no. HOU 262. Canberra: AIHW.

Relational attributes

Related metadata references:Help on this term

Has been superseded by Data quality statement (Specialist Homelessness Services Collection—December quarter 2011) AIHW Data Quality Statements, Archived 15/12/2014

Supersedes Data quality statement (Specialist Homelessness Services Collection—December quarter 2011) AIHW Data Quality Statements, Archived 15/12/2014

My items Help on this term
Download Help on this term