Australian Government: Australian Institute of Health and Welfare METeOR Home Page

Online Services Report (OSR) Database, 2015–16; Quality Statement

Identifying and definitional attributes

Metadata item type:Help on this termQuality Statement
Synonymous names:Help on this termOATSIH Services Reporting
METeOR identifier:Help on this term661847
Registration status:Help on this termAIHW Data Quality Statements, Archived 10/07/2018

Data quality

Quality statement summary:Help on this term

The Online Services Report (OSR) annually collects information from organisations that receive Australian Government funding to provide health services to Aboriginal and Torres Strait Islander people. The main data quality issues identified in the 2015–16 collection were incomplete reporting and data discrepancies between two or more questions. As part of the data collection, AIHW staff contacted organisations to check any data quality issues and to request additional or corrected data where necessary.

In 2015–16, 277 organisations contributed to the OSR national report. Most organisations (95%) had all of their data included in national analyses; 5% (13 organisations) had 1 or more data items excluded from national analyses because of data quality issues. A total of 32 data items from these organisations were excluded from national analyses. This affected 9 analysis topics (primary health episodes for 2 organisations, primary health contacts for 1 organisation, antenatal care clients and visits for 1 organisation, SEWB clients for 1 organisation, SEWB contacts for 1 organisation, substance-use clients for 5 organisations, substance-use residential clients/episodes for 5 organisations, substance-use sobering up clients/episodes for 2 organisations and substance-use non-residential clients/episodes for 6 organisations).

Institutional environment:Help on this term

The Australian Institute of Health and Welfare (AIHW) is responsible for the data management, analysis and reporting of information collected as part of the OSR. It gets OSR data submitted by health organisations on OCHREStreams, a secure web portal, hosted and managed by the Improvement Foundation on behalf of the Australian Government Department of Health. The Improvement Foundation (Australia) Ltd was established in Adelaide in 2006 as a not-for-profit organisation and a subsidiary of the UK Improvement Foundation. The Improvement Foundation provides expertise in the development and delivery of quality improvement techniques through specialist change management advice and IT systems.

The AIHW is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent corporate Commonwealth entity established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health portfolio.

The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.

The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.

One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.

The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988, (Cth) ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality.

For further information see the AIHW website www.aihw.gov.au.

Timeliness:Help on this term

The OSR collects data for each financial year and has been ongoing since 2008–09. For most years, including 2015–16, the reporting period was for the financial year from 1 July to 30 June. In 2014–15, the Census date for the collection was brought forward by 1 month to 31 May 2015, rather than 30 June 2015. This was done to allow organisations to use the PenCAT data extraction tool, which would not be available for OSR reporting after 30 June 2015. Organisations were still asked to provide data for a 12 month period (1 June 2014 to 31 May 2015) to enable time series analyses. However, this meant data for June 2014 may have been reported in both the 2013–14 and 2014–15 collections for clients, contacts and episodes of care. Nevertheless, comparing 2013–14, 2014–15 and 2015–16 data did not show any obvious underreporting for any data items at the national level.

In 2015–16, 277 organisations were in-scope for the OSR collection. They were required to submit initial data by 31 August which is within two months of the census date (30 June). Around half (51%) of organisations had submitted initial data by this date. Collection of OSR data was delayed in part because many of the organisations also had to submit 2 sets of data for the nKPI collection. As a result in 2015–16 OSR data was finalised 1 month later than in the previous year, i.e. by the end of November rather than the end of October.

Accessibility:Help on this term

National level data are available in AIHW publications. The most current report is the "Aboriginal and Torres Strait Islander health organisations, Online Services Report-key results 2015–16". This publication is available in hard copy or on the AIHW website.

Interpretability:Help on this term

Data were collected using a web-based reporting tool designed to capture information on the health services provided to Aboriginal and Torres Strait Islander clients attending primary health, New Directions mothers and babies services, substance-use, and social and emotional wellbeing services funded by the Australian Government. Data for a small number of non-Indigenous clients attending these services were also included. The statistical analyses used in reports are mainly descriptive. Numbers and percentages are commonly used to show major trends. The annual publication also includes a range of information to assist in the interpretation of data, including a glossary and an appendix on data quality.

A characteristic of the collection is that the number of organisations submitting data changes slightly each year. While for the most part, it is the same organisations contributing to the collection, the number of organisations may change due to changes in funding, auspicing or reporting arrangements at the local level. In 2015–16, 277 organisations contributed to the collection. This was a decrease of 1 from 2014–15 (278 organisations). However, 22 organisations reporting in 2014–15 were not in-scope in 2015–16, while 21 additional organisations were in-scope in 2015–16.

Another characteristic of the collection is that the organisations submitting valid data for a particular data item may also change over time, as data with remaining quality issues after follow-up are excluded from national analyses. This means that each year some organisations' data are partially accepted rather than fully accepted. Both the changing organisations in-scope for the collection and the changing number of organisations with valid data for each data item may impact time series analyses, as this is based on all the organisations that provide valid data in each year, rather than on a subset which have valid data for all years.

Relevance:Help on this term

The database includes information on the activities of organisations from eight reporting periods from 2008–09 to 2015–16. It provides an overview of Australian Government funded health organisations that aim to improve health outcomes for Aboriginal and Torres Strait Islander people. In 2015–16, most OSR organisations (74%) provided primary health-care services, either solely or in combination with the New Directions mothers and babies program, social and emotional wellbeing services and substance-use rehabilitation and treatment services.

Data presented in the national report are reported at an aggregated level. This aggregated analysis provides general information about services offered at primary health-care and other organisations, for example clinical care, health promotion and group activities as well as client numbers, contacts and episodes of care. The data can also be analysed at jurisdictional and remoteness area level. Most of the organisations who report to the OSR also report to the national key performance indicators (nKPI) collection. OSR data therefore provides useful contextual information for organisations participating in the nKPIs.

There were a number of questions that were not relevant to all organisations. There may also be instances where an organisation's data were not included for a question due to quality issues. The denominators used for percentage calculations are the number of services that provided valid data for each question.

Accuracy:Help on this term

Some OSR data were extracted from the Patient Information Recall System (PIRS) used by the organisation and reported through OCHREStreams, a web-based portal built and maintained by the Improvement Foundation for the Department of Health. A few different PIRS were used. Organisations could choose to overwrite any of their data on OCHREStreams, for example if they assessed it to be inaccurate, and all submissions were approved by their CEO, before the data were made available to the AIHW for analysis.

AIHW examined all data submissions to identify missing data and possible data quality issues and in 2015-16 had at least one query for 260 (94%) organisationsAIHW staff contacted these organisations for further clarification and review. Around three-quarters (74%) of these organisations (192) resubmitted some data. Some organisations were unable to provide exact data and provided estimates of client numbers and episodes of care. In 2015-16, 22 organisations funded to provide primary health-care services (11%) estimated their episodes of care and 31 (15%) estimated their client numbers. This data was accepted unless the AIHW assessed their data was likely to overestimate actual numbers.

Each year some organisations' data are partially accepted, rather than fully accepted. Common data quality queries in the 2015-16 collection were around incomplete or inaccurate data (for example, FTE positions were not reported or were reported in terms of the number of people rather than in terms of full-time equivalent positions); data discrepancies between two or more questions (for example, the number of clients exceeded the number of episodes of care); and large increases and decreases in data compared with previous submissions. Where significant data quality issues remained after follow-up then these data items were excluded from national analyses.

By the cut-off date for final data submissions, most (95%) organisations had provided data that could be included in national analyses. The remaining 5% or 13 organisations had a total of 32 data items excluded from national analyses due to remaining data quality issues (some organisations had more than one item excluded). Exclusion rates vary by data item. For example, primary health episodes of care data were excluded for 2 (1%) primary health-care organisations, while substance-use total clients data were excluded for 5 (6%) organisations that provided this service.

Exclusion rates were higher for substance-use clients and episodes of care. In 2015-16, a few organisations asked to report on their substance-use clients and episodes of care for the first time were unable to provide this data.

Coherence:Help on this term

Data were collected and reported by the AIHW for the first time in 2008–09 using a new paper-based OSR questionnaire. This replaced the Service Activity Reporting, Drug and Alcohol Services Reporting, and Bringing Them Home and Link Up counselling data collections. These were previously collected by the Department of Health. It would not be appropriate to make time series comparisons with data collected prior to 2008–09, due to differences in the collection methodology and scope of the collection.

From 2012–13, use of a new online questionnaire commenced. It introduced a reporting module for maternal and child health care programs and activities. Major questions (e.g. episodes of care, clients and contacts) remained the same. Some other questions were revised or restructured. For example, the question on clinical and health related activities/programs from the previous collections was split into several new questions and some answer options were no longer available. Questions about substance-use issues that organisations provided treatment for were now collected for the top 5 issues rather than all applicable issues. Only comparable data were included in time series analyses in the national report. Additional questions on the gaps and challenges faced by organisations in providing health services were included in the questionnaire and analysed for the first time.

For the 2013–14 data collection, New Directions: Mothers and Babies services were in-scope for the first time. It should be noted that the number of organisations receiving New Directions funding that submitted OSR data are different to the number of New Directions funded organisations reported by the Department of Health, due to differences in how an organisation is defined. An organisation defined as a single entity for funding purposes may have multiple organisations reporting OSR data. Most of the organisations receiving New Directions funding were already contributing to the OSR collection, because they also received primary health-care funding.    

Since 2012-13 more information has been collected at the site level, including the address of each service delivery site and the range of services provided from each site. This information can be used to more accurately identify service delivery locations and for data validation. Using this information, in 2015-16 the remoteness category was changed for a few organisations to better reflect where they deliver services, rather than their administrative address.

Data products

Implementation start date:Help on this term01/12/2016

Source and reference attributes

Submitting organisation:Help on this term

AIHW

Relational attributes

Related metadata references:Help on this term

Supersedes Online Services Report (OSR) Database, 2014–15; Quality Statement AIHW Data Quality Statements, Archived 31/05/2017

Has been superseded by Online Services Report (OSR) Database, 2016–17; Quality Statement AIHW Data Quality Statements, Endorsed 10/07/2018

My items Help on this term
Download Help on this term