Australian Government: Australian Institute of Health and Welfare METeOR Home Page

Online Services Report (OSR) Database, 2014–15; Quality Statement

Identifying and definitional attributes

Metadata item type:Help on this termQuality Statement
Synonymous names:Help on this termOATSIH Services Reporting
METeOR identifier:Help on this term623095
Registration status:Help on this termAIHW Data Quality Statements, Archived 31/05/2017

Data quality

Quality statement summary:Help on this term

The Online Services Report (OSR) annually collects information from organisations that receive Australian Government funding to provide health services to Aboriginal and Torres Strait Islander people. The main data quality issues identified in the 2014–15 collection were incomplete reporting and data discrepancies between two or more questions. As part of the data collection, AIHW staff contacted organisations to check any data quality issues and to request additional or corrected data where necessary.

In 2014-15, 278 organisations contributed to the OSR national report. Most organisations (92%) had all of their data included in national analyses; 8% (21 organisations) had 1 or more data items excluded from national analyses because of data quality issues. A total of 40 data items from these organisations were excluded from national analyses. This affected 11 analysis topics.

Institutional environment:Help on this term

The Australian Institute of Health and Welfare (AIHW) is responsible for undertaking the data management, analysis and reporting of information collected as part of the OSR. It gets OSR data from health organisations through OCHREStreams, a secure web portal, hosted and managed by the Improvement Foundation on behalf of the Australian Government Department of Health. The Improvement Foundation (Australia) Ltd was established in Adelaide in 2006 as a not-for-profit organisation and a subsidiary of the UK Improvement Foundation. The Improvement Foundation provides expertise in the development and delivery of quality improvement techniques through specialist change management advice and IT systems.

The AIHW is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent corporate Commonwealth entity established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health portfolio.

The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.

The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.

One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.

The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988, (Cth) ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality.

For further information see the AIHW website www.aihw.gov.au.

Timeliness:Help on this term

The OSR collects data for each financial year and has been ongoing since 2008–09. It should be noted that the reporting period in 2014-15 changed slightly from previous years. The census date for the collection was brought forward by 1 month to 31 May 2015, rather than the 30th June 2015. This was done to allow organisations to use the Pen CAT data extraction tool which would not be available for OSR reporting after 30 June 2015. Organisations were still asked to provide data for a 12 month period (1 June to 31 May 2015) to enable time series analyses. This resulted in 2014-15 data overlapping with 2013-14 data for June 2014 for clients, contacts and episodes of care. Analysis of data comparing to 2013-14 data did not show any obvious effects on the data from bringing the census date forward. There were no unusual decreases in any data item at the national level. The impact of bringing the collection forward by one month will be investigated further in the next collection round when 2015-16 data is available.  

In 2014-15, 279 organisations were in-scope for the OSR collection. They were required to submit initial data within two months of the census date (30th May). Most organisations (98%) had submitted initial data by this date. The data were due to be finalised by mid October. By this date, 99.6% of organisations had submitted data and 1 organisation did not submit data and were the only non-respondent for the collection.

As data were submitted, the AIHW checked it for accuracy and consistency across reporting periods and asked organisations to review and re-submit data where corrections were required. This process continued until 16 October 2015, after which a finalised national dataset was created. In 2014-15, data from 278 organisations were included in the OSR national report. 

Accessibility:Help on this term

National level data are available in AIHW publications. The most current report is the "Aboriginal and Torres Strait Islander health organisations, Online Services Report-key results 2014-15". This  publication is available in hard copy or on the AIHW website.

Interpretability:Help on this term

Data were collected using a web-based reporting tool designed to capture information on the health services provided to Aboriginal and Torres Strait Islander clients attending primary health, New Directions mothers and babies services, substance-use, and social and emotional wellbeing services funded by the Australian Government. Data for a small number of non-Indigenous clients attending these services were also included. The statistical analyses used in reports are mainly descriptive. Numbers and percentages are commonly used to show major trends. The annual publication also includes a range of information to assist interpretation of data, including a glossary and an appendix on data quality.

A characteristic of the collection is that the number of organisations submitting data changes slightly each year. While for the most part, it is the same organisations contributing to the collection, the number of organisations may change due to changes in funding, auspicing or reporting arrangements at the local level. In 2014-15, 278 organisations contributed to the collection. While this was 9 more than in 2013-14 (269 organisations), 3 organisations reporting in 2013-14  did not report in 2014-15 and there were 12 additional organisations in-scope in 2014-15: 2 for primary health care; 1 for New Directions; 1 for social and emotional wellbeing services and 8 for substance-use services. 

Another characteristic of the collection is that the organisations submitting valid data for a particular data item may also change overtime, as data with remaining quality issues after follow-up are excluded from national analyses. This means that each year some organisations have data that is partially accepted rather than fully accepted. Both the changing organisations in-scope for the collection and the changing number of organisations with valid data for each data item may impact time series analyses, as this is based on the organisations that provide valid data in each year, rather than on a subset which have valid data for all years.

Relevance:Help on this term

The database includes information on the activities of organisations from seven reporting periods from 2008-09 to 2014-15. It provides an overview of Australian Government funded health organisations that aim to improve health outcomes for Aboriginal and Torres Strait Islander people. Most OSR organisations (73%) provided primary health-care services, either solely or in combination with the New Direction mothers and babies program, social and emotional wellbeing services and substance-use rehabilitation and treatment services.

Data presented in this report are reported by service providers at an aggregated level. It provides general information about services offered at primary health-care and other organisations (for example clinical care, group activities and health checks) and activities undertaken. The data can also be analysed at jurisdictional and remoteness area level. Most of these organisations also report to the national key performance indicators (nKPI) collection and the OSR data provide useful contextual information for organisations participating in the nKPIs.

There were a number of questions that were not relevant to all organisations. There may also be instances where an organisation's data were not included for a question due to quality issues. The denominator used for percentage calculation is the number of services that provided valid data for each question.

Accuracy:Help on this term

Some OSR data are electronically transferred from the Patient Information Recall System (PIRS) used by the organisation into OCHREStreams, a web-based portal built and maintained by the Improvement Foundation for the Department of Health. Most organisations used a data extraction tool called pen CAT to extract some OSR data (for example client numbers, contacts, and episodes of care) directly from their PIRS and to load it into OCHREStreams. Organisations could choose to overwrite any of their data on OCHREStreams, for example if they assessed it to be inaccurate, before the data were available to the AIHW for analysis. A variety of PIRS were used by organisations and not all PIRS were compatible with the CAT tool.

AIHW examined all data submissions to identify missing data and possible data quality issues and detected at least one query for 262 (94%) organisationsAIHW staff contacted these organisations for further clarification and review. Over three-quarters (79%) of organisations (220) resubmitted data. 

There were two major data quality issues identified in the 2014–15 collection: discrepancies between data provided for two or more questions (for example the number of clients exceeded the number of episodes of care); and inaccuracies in the data provided for some questions (for example, FTE staff were reported in terms of the number of people rather than full-time equivalent positions).

In terms of the reasons for missing data or data with quality issues were the lack of complete records of data and/or insufficient data management resources available at organisations to support the data collection. Some organisations also had problems with the electronic transfer of data from their organisation’s PIRS to the AIHW.

By the cut-off date for final data submissions, most (92% or 257) organisations had provided data that could be included in national analyses. The remaining 8% or 21 organisations had a total of 40 data items excluded from national analyses due to remaining data quality issues (some organisations had more than one item excluded). Exclusion rates vary by data item. For example, primary health episodes of care data were excluded for 2% of primary health-care organisations, while child health check data were excluded for 3% of organisations that provided this service.

Exclusion rates were higher for substance-use clients and episodes of care, with 8 (12%) and 7 (10%) organisations providing these services being excluded respectively. In 2014-15, some new validation checks were added to the collection to assess the quality of substance-use counts. This suggested that some substance-use counts for non-residential clients and episodes of care were being extracted incorrectly in organisations using one particular type of PIRS. This led to inflated counts for these services. Where this data could not be corrected by the organisation it was excluded from analyses. It also led to estimates for substance-use clients and episodes of care being revised down for 2012-13 and 2013-14 time series analyses in the report.

The denominator used for percentage calculation is the number of organisations that provided valid data for each item.

Coherence:Help on this term

Data were collected and reported by the AIHW for the first time in 2008–09 using a new OSR questionnaire. This replaced the Service Activity Reporting, Drug and Alcohol Services Reporting, and Bringing Them Home and Link Up counselling data collections. These were previously collected by the Department of Health. It would not be appropriate to make time series comparisons with data collected prior to 2008–09, due to differences in the collection methodology and scope of the collection.

From 2012–13, use of a new online questionnaire commenced. It introduced a reporting module for maternal and child health care programs and activities. Major questions (e.g. episodes of care, clients and contacts) remained the same. Some other questions were revised or restructured. For example, the question on clinical and health related activities/programs from the previous collections was split into several new questions and some answer options were no longer available. Questions about substance-use issues that organisations provided treatment for were now collected for the top 5 issues rather than all applicable issues. Only comparable data were included in time series analyses in the national report. Additional questions on the gaps and challenges faced by organisations in providing health services were included in the questionnaire and analysed for the first time.

For the 2013–14 data collection, New Directions: Mothers and Babies services were in-scope for the first time. It should be noted that the number of organisations receiving New Directions funding that submitted OSR data are different to the number of New Directions funded organisations reported by the Department of Health, due to differences in how an organisation is defined. An organisation defined as a single entity for funding purposes may have multiple organisations reporting OSR data. Most of the organisations receiving New Directions funding were already contributing to the OSR collection, because they also received primary health-care funding; however a small number of them (17 in 2014-15) did not receive DoH primary health care funding.    

 

Data products

Implementation start date:Help on this term04/02/2015

Source and reference attributes

Submitting organisation:Help on this term

AIHW

Relational attributes

Related metadata references:Help on this term

Supersedes Online Services Report (OSR) Database, 2013–14; Quality Statement AIHW Data Quality Statements, Endorsed 30/04/2015

Has been superseded by Online Services Report (OSR) Database, 2015–16; Quality Statement AIHW Data Quality Statements, Archived 10/07/2018

My items Help on this term
Download Help on this term