Specialist Homelessness Services Collection—September quarter 2011; Data Quality Statement
Identifying and definitional attributes
|Metadata item type:||Quality Statement|
|Registration status:||AIHW Data Quality Statements, Archived 04/07/2012|
|Quality statement summary:|
The SHSC collects information on people seeking services from agencies that receive funding under the NAHA or the NPAH to provide specialist homelessness services. Data is collected monthly from agencies participating in the collection.
The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent statutory authority established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health and Ageing portfolio.
The AIHW aims to provide authoritative information and statistics to promote better health and wellbeing. The Institute collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.
The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.
One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.
For further information see the AIHW website www.aihw.gov.au.
The AIHW has been the Data Custodian for the SAAP NDC since 1996. The SHSC was developed by AIHW in conjunction with the states and territories and is being administered by the AIHW.
The SHSC began collecting data from 1 July 2011. The SHSC collects information from all participating agencies every month. Data collected is regularly loaded to a Master Database. From this Master Database ‘snapshots’ are created at particular points in time for reporting purposes. Snapshots will be taken following the end of each quarter, and another one for the whole collection year for annual reporting. It is planned, from 2012–13, to publish quarterly results by the end of the following quarter, and annual results in October each year.
The September quarter 2011 snapshot contains data submitted to the AIHW for the July, August and September 2011 collection months, using responses received and validated as at 28 February 2012 when the snapshot was taken.
Published results from the SHSC first quarter are available elsewhere in this report and on the AIHW website.
Data not available online or in reports can be obtained from the Communications, Media and Marketing Unit on (02) 6244 1032 or via email to email@example.com. Requests that take longer than half an hour to compile are charged for on a cost-recovery basis.
Information on the development of the SHSC, definitions and concepts, and collection materials and processes can be found on the AIHW website, www.aihw.gov.au. Information on definitions, concepts and classifications can also be found in the SHSC’s collection manual (AIHW 2011b).
Scope and coverage
The SHSC is a key source of information for measuring the outcomes and outputs for people who are homeless, or at risk of homelessness, who seek specialist homelessness services.
The SHSC collects, via specialist homelessness agencies, information on people seeking services from those agencies. All agencies that receive funding under the NAHA or NPAH to provide specialist homelessness services are in scope for the SHSC. However, only those expected to provide data are covered. Victorian Transitional Housing Management (THM) agencies, as well as many high volume services and intake services, are not covered, as it is difficult for such agencies to report. In the first quarter, 1,460 agencies were covered.
Not everyone in scope for SHSC is homeless, as specialist homelessness agencies provide services to people at risk of homelessness, as well as people who are currently homeless.
Not all homeless people and people at risk of homelessness are in scope for the SHSC―only those who seek services from specialist homelessness agencies are in scope.
Data are collected by homelessness agency workers for each client support period. Some basic information is also collected on instances where people seek, but do not receive assistance, from a homelessness agency (‘unassisted people).
The SHSC collects information every month, the data is regularly loaded to a Master Database, and snapshots of this database are created a
Potential sources of error
As with all data collections, the SHSC estimates are subject to errors. These can arise from data coding and processing errors, inaccurate data, or missing data. Reported findings are based on data reported by agency workers.
The AIHW receives data from specialist homelessness agencies every month. This data goes through two processes of data validation, that is, error checking. Firstly, data validation is incorporated into the client management systems (CMSs) most agencies use to record their data. Secondly, data is submitted through the AIHW online reporting web-portal, Specialist Homelessness Online Reporting (SHOR). SHOR completes a more thorough data validation and reports any errors that need correcting before data can be submitted, to staff of the homelessness agency.
The SHOR data validation system was not fully functional during the September quarter 2011, as more work was needed to ensure edit checks were appropriate to the data being received. However, the data validation systems in CMSs were fully functional; and testing of the data received indicates that erroneous data is minimal, although some invalid values are present.
93% of relevant agencies returned support period data for at least one month in the September quarter 2011. This compares favourably with the SAAP agency participation rate, which was 90% in 2010–11 and 92% on average between 2001–02 and 2010–11.
Statistical Linkage Key (SLK) validity
An individual client may seek or receive support on more than one occasion—either from the same agency or from a different agency. Data from individual clients who presented at different agencies and/or at different times is matched based on a statistical linkage key (SLK) which is constructed from information about the client’s date of birth, sex and an alphacode based on selected letters of their name, allowing client level data to be created.
If a support period record does not have a valid SLK, it cannot be linked to a client, and thus it is not included in client-level tables (although it is included in support period-level tables). In the September quarter 2011, 93% of support periods had a valid SLK.
In many support periods, in the September quarter 2011, valid responses were not recorded for all questions, because invalid responses were recorded, ‘don’t know’ was selected, or no response was recorded.
The rate of invalid/’don’t know’/missing responses was higher than expected for many data items. For example:
‘Sex’ and ‘date of birth’ have some of the lowest rates of invalid/missing/ don’t know response―less than 1% and 7% respectively.
Support periods with invalid/’don’t know’/missing responses were retained in the collection, and, due to the difficulty of doing so accurately, no attempt was made to deduce or impute the true value of invalid/’don’t know’/missing responses.
Data items with very high rates of invalid/’don’t know’/missing responses were not reported on in this publication.
Less than 100% agency participation, less than 100% SLK validity and a high rate of incomplete responses do not necessarily mean that estimates are biased. The non-respondents are not systematically different in terms of how they would have answered the questions, then there is no bias. Given the results of analyses of agency participation, SLK validity and incomplete responses performed to date, some non-response bias is expected.
As some data is missing because of agency non-participation and SLK invalidity, non-response adjustment (or ‘weighting’) has been applied to create a selection of weighted estimates. Only a limited number of estimates were amenable to non-response adjustment in the September quarter 2011.
A more complete method for adjusting for missing information will be applied to annual data; the method used in the September quarter 2011 is a simplified version of that method.
Non-response adjustment in the SHSC is performed for support periods, and, separately, for clients. The process used is described in Appendix C.
The process accounts only for agency non-participation and SLK invalidity―due the difficulty of doing so accurately, no adjustments are made for incomplete responses.
In the September quarter 2011, non-response adjustment was performed at the national level only. This may lead to biased estimates, as analysis has shown that agency participation and SLK validity vary with characteristics such as state/territory, age group and sex, and the non-response adjustment process does not account for this.
The non-response adjustment process used in the September quarter 2011, which designates agencies ‘participating’ if they returned support period data at any time during the reporting period, causes weighted estimates to be, on average, underestimates.
Improvements to the weighting methodology will be implemented during 2011–12. In particular, the method used to weight the annual data will be much more extensive than the method used for the September quarter 2011 data. This will lead to comparability issues, as different weighting methodologies give different estimates for the same raw data.
The SHSC replaces the SAAP NDC, which began in 1996. The SHSC differs from the SAAP NDC in many respects.
The major definitional differences between SAAP and SHSC concern children and support. In the SAAP NDC, children who accompanied a parent or guardian were included as accompanying children; in the SHSC, children are included as clients if they directly receive a service. In SAAP, support was considered to entail generally one hour or more of a worker’s time; in SHSC no time-related condition exists. Further information on the comparability of SHSC and SAAP can be found in Appendix A.
The cultural definition of homelessness
In the SHSC, homelessness is defined living in non-conventional accommodation or living in short-term or emergency accommodation due to a lack of options. Some other collections instead define homelessness using the cultural definition, which delineates three homelessness categories:
The cultural definition of homelessness does not define ‘at risk of homelessness’.
Comparison with other collections
The other major data sources on homelessness are:
The ABS Census, which collects data from all persons in Australia on Census night, including data allowing respondents’ homelessness status to be derived (see ABS 2006). The cultural definition of homelessness is used. Data is currently available for 2001 and 2006.
The ABS General Social Survey’s homelessness module (ABS 2010a), which collects data from usual residents of private dwellings, including data on whether respondents have ever been homeless. The survey defines homelessness as being without a permanent place to live for a selection of reasons. Data is currently available for 2010 only.
Chamberlain and Mackenzie’s National Census of Homelessness School Students, which collects data on homeless school students via principals of all government and Catholic secondary schools (FaHCSIA 2006). Both the cultural definition of homelessness and a service delivery definition are used. Data is currently available for 2006 and selected prior years.
As these collections differ greatly from SHSC in scope, collection methodology, definitions and reference periods, comparisons between collections should be made with caution.
Source and reference attributes
Australian Institute of Health and Welfare
Australian Institute of Health and Welfare (2012). Specialist Homelessness Services Collection: first results. Cat. no. HOU 262. Canberra: AIHW.
|Related metadata references:|
Has been superseded by Data quality statement (Specialist Homelessness Services Collection—December quarter 2011) AIHW Data Quality Statements, Archived 15/12/2014
Supersedes Data quality statement (Specialist Homelessness Services Collection—December quarter 2011) AIHW Data Quality Statements, Archived 15/12/2014