Online Services Report (OSR) Database, 2017–18

• AIHW Data Quality Statements, Superseded 02/06/2021

The Online Services Report (OSR) annually collects information from organisations that receive Australian Government funding to provide health services to Aboriginal and Torres Strait Islander people. The main data quality issues identified in the 2017–18 collection were incomplete reporting and data discrepancies between two or more related questions. As part of the collection, Australian Institute of Health and Welfare staff contacted organisations to follow up data queries and to request revised data where necessary.

In 2017–18, 278 (96% of in scope) organisations contributed to the OSR collection. Most organisations (92%) had all of their data included in national analyses; 8% (20 organisations) had 1 or more data items excluded from national analyses due to data quality issues.

The Australian Institute of Health and Welfare (AIHW) is responsible for the data management, analysis and reporting of information collected as part of the OSR. It downloads the OSR data submitted by health organisations on OCHREStreams, a secure web portal, hosted and managed by the Improvement Foundation on behalf of the Australian Government Department of Health (DoH). The Improvement Foundation (Australia) Ltd was established in Adelaide in 2006 as a not-for-profit organisation and a subsidiary of the UK Improvement Foundation. The Improvement Foundation provides expertise in the development and delivery of quality improvement techniques through specialist change management advice and IT systems.

The AIHW is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent corporate Commonwealth entity established in 1987, governed by a management Board, and accountable to the Australian Parliament through the Health portfolio.

The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.

The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.

One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.

The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988, (Cth) ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality.

For further information see the AIHW website www.aihw.gov.au.

The OSR collects data for each financial year and has been ongoing since 2008–09. For most years, including 2017–18, the reporting period was for the financial year from 1 July to 30 June. In 2014–15, the Census date for the collection was brought forward by 1 month to 31 May 2015, rather than 30 June 2015. This was to allow organisations to use the PenCAT data extraction tool, which would not be available for OSR reporting after 30 June 2015. Organisations were still asked to provide data for a 12 month period (1 June 2014 to 31 May 2015) to enable time series analyses. However, this meant data for June 2014 may have been reported in both the 2013–14 and 2014–15 collections for clients, contacts and episodes of care. Nevertheless, comparing 2013–14, 2014–15 and 2015–16 data did not show any obvious underreporting for any data items at the national level.

In 2017–18, 278 organisations were in scope for the OSR collection and were required to submit initial data by 31 August which is within two months of the census date (30 June). Most organisations (93%) had submitted data by this date. By the final cut-off date, 266 organisations (96%) had submitted data and 12 (4%) organisations had not submitted and were non-response.

National level data are available in AIHW publications. Until the 2017–18 collection, national reports were pdfs and these are available on the AIHW website. The most current report is "Aboriginal and Torres Strait Islander health organisations, Online Services Report-key results 2017–18". This web-based report focuses on organisations funded to provide Indigenous primary health services only, and includes interactive tableau data visualisations for the first time. This report is available for viewing on the AIHW website, along with excel supplementary tables with all of the data used in the tableau data visualisations.

Data were collected using a web-based reporting tool designed to capture information about organisations funded by the Australian Government to provide health services to Aboriginal and Torres Strait Islander people. Data is collected about organisations providing primary health, New Directions mothers and babies services, substance-use, and social and emotional wellbeing services to Aboriginal and Torres Strait Islander clients. Data for a small number of non-Indigenous clients attending these services were also included. The statistical analyses used in reports are mainly descriptive. Numbers and percentages are commonly used to show major trends. The annual publication also includes a range of information to assist in the interpretation of data, including a glossary and information on data quality.

A characteristic of the collection is that the number of organisations submitting data changes slightly each year. While for the most part, it is the same organisations contributing to the collection, the number of organisations may change due to changes in funding, auspicing or reporting arrangements at the local level. In 2017–18, 266 organisations contributed to the collection. This was similar to 2016–17. There were 11 organisations reporting in 2016–17 that did not report in 2017–18 (2 were no longer in scope and 9 were non-response), while 11 organisations not reporting in 2016–17 were in scope for reporting in 2017–18. There was 1 organisation in scope for both collections that was non-response in both years.

Another characteristic of the collection is that the organisations submitting valid data for a particular data item may change over time. This is because data with remaining quality issues at the cut-off date for each collection are excluded from national analyses. This means that in each year, some organisations' data are partially accepted rather than fully accepted. Both the changing organisations in scope for the collection, and the changing number of organisations with valid data for each data item may impact time series analyses. In AIHW OSR national reports, time series analyses are based on all the organisations that provide valid data in each year, rather than on a subset which have valid data over all years.

Data are available on funded organisations from 10 reporting periods from 2008–09 to 2017–18. It provides an overview of Australian Government-funded health organisations that aim to improve health outcomes for Aboriginal and Torres Strait Islander people. In 2017–18, most OSR organisations (74%) provided primary health-care services, either solely or in combination with the New Directions mothers and babies program, social and emotional wellbeing services and substance-use rehabilitation and treatment services.

Data presented in national reports are reported at an aggregated level. This aggregated analysis provides general information about services offered at primary health and other organisations, for example clinical care, health promotion and group activities as well as client numbers, contacts, episodes of care and full time equivalent staffing levels. The data can also be analysed at jurisdictional and remoteness area level. Most of the organisations that report to the OSR also report to the national key performance indicators (nKPI) collection. OSR data therefore provides useful contextual information for organisations participating in the nKPIs.

There may be questions that are not relevant to all organisations. There may also be instances where an organisation's data are not included for a question, due to data quality issues.

The denominators used for percentage calculations are the number of organisations that provided valid data for each question.

While the collection and validation processes in 2017–18 were similar to previous collections, there was a change from the 2016–17 collection that is worth noting. In 2016–17, there were changes made by the Department of Health in the collection of episodes of care data that may affect their comparability with previous collections. These were made to ensure consistency in reporting between organisations using different clinical information systems (CIS). There were also some changes in the types of contacts included in the episodes of care count. This meant some contact types (for example, health care delivered over the telephone and hospital-related contacts) were excluded from the episode of care count in some organisations. These changes resulted in an expected decrease in episodes of care counts in 2016–17. There was however also an unexpected decrease in episode counts in a few organisations using Medical Director (MD), where some clinical contacts were not counted in their episodes of care data as they should have been. This meant episodes of care counts were lower in 2016–17 and these were not strictly comparable with previous years. In 2017–18, these contact types were again included in the episodes of care count and the extraction issues around episodes of care counts were resolved.

The AIHW examined all OSR submissions to identify missing data and possible data quality issues and in 2017-18 had at least one query for 201 (76%) organisations. AIHW staff contacted these organisations for further clarification and review and 80% of organisations resubmitted some data. Some organisations were unable to provide exact data and provided estimates of client numbers and episodes of care. In 2017-18, 16 (8%) organisations providing primary health-care services estimated their episodes of care and 22 (11%) estimated their client numbers. This data was accepted unless the AIHW assessed their data was likely to overestimate actual numbers.

Common data quality queries in the 2017-18 collection were around incomplete or inaccurate data (for example, workforce positions were not reported or were reported in terms of the number of people rather than full-time equivalent positions); data discrepancies between two or more questions (for example, the number of clients exceeded the number of episodes of care); and large increases or decreases in data compared with previous submissions. Where significant data quality issues remained after follow-up, then these data were excluded from national analyses.

In 2017-18, by the final cut-off date for submissions, most organisations (92%) had provided data that could be included in national analyses. The remaining 8% (20 organisations) had a total of 40 data items excluded from national analyses due to remaining data quality issues (some organisations had more than one item excluded). Exclusion rates vary by data item. For example, primary health episodes of care data were excluded for 4 (2%) primary health-care organisations, while antenatal clients and visits data were excluded for 2 (1%) organisations that provided this service.

Data were collected by the AIHW for the first time in 2008–09 using a paper-based OSR questionnaire. This replaced the Service Activity Reporting, Drug and Alcohol Services Reporting, and Bringing Them Home and Link Up counselling data collections previously collected by the DoH. It would not be appropriate to make time series comparisons with data collected prior to 2008–09, due to differences in the collection methodology and scope of the collection.

In 2012–13, following a review of the OSR collection, a revised collection instrument was introduced that subdivided the questionnaire into modules for ease of completion. The new form also introduced a reporting module for maternal and child health care programs and activities. Key data items such as episodes of care, client numbers and client contacts remained the same. Some other questions were revised or restructured. For example, the question on clinical and health-related activities/programs from the previous collections was split into several new questions and some answer options were no longer available. Given this, only comparable data were included in time series analyses in the national report. Additional questions on the gaps and challenges faced by organisations in providing health services were included in the questionnaire and analysed for the first time. Another change to the form in 2012–13 meant some information was now available at the site level and not just the organisation level, including the address of each service delivery site and the range of services provided from each site. This information was used to more accurately identify service delivery locations and for data validation. For example, using this information in 2015–16 meant the remoteness category was changed for a few organisations to better reflect service delivery, rather than an administrative address.

From 2012–13, the collection was also assisted by the introduction of the PenCAT audit tool and the OCHREStreams online reporting portal, developed and maintained by the Improvement Foundation for the DoH. The PenCAT tool could be used to extract some data (including counts of clients, client contacts and episodes of care) directly from an organisation’s clincial information system (CIS) onto OCHREStreams. The AIHW could then download this data from the web portal. These changes were designed to improve the accuracy of the data and reporting response times and to reduce respondent burden, however, it may have led to some differences in the data reported for some organisations. It is important to note that organisations could still overwrite this pre-population function and not all organisations used this facility, so many still submitted all of their data manually onto the web portal. The PenCAT tool was not available for use for extracting data onto OCHREStreams from 1 July 2015 and was not used for the 2015–16 collection. For this collection organisations relied on their own data extraction processes and manually entered this data onto OCHREStreams.

The way in which pre-populated data was extracted onto OCHREStreams changed again for the 2016–17 collection. The DoH worked with CIS vendors and introduced direct load to extract this data directly from an organisations CIS onto OCHREStreams. As in previous collections however, some organisations still submitted their OSR data manually and all organisations were able to overwrite and make changes to their data before submitting them to the AIHW.

New Directions: Mothers and Babies services were in scope for the OSR for the first time in the 2013–14 collection. It should be noted that the number of organisations receiving New Directions funding that submitted OSR data is different to the number of New Directions funded organisations reported by the DoH, due to differences in how an organisation is defined. An organisation defined as a single entity for funding purposes may have multiple organisations reporting OSR data. Most of the organisations receiving New Directions funding were already contributing to the OSR collection, because they also received primary health-care funding.

 AIHW