National Health Workforce Data Set: medical practitioners 2015: National Health Workforce Data Set, 2015: Quality Statement
Data Quality Statement Attributes
Identifying and definitional attributes | |
Metadata item type: | Data Quality Statement |
---|---|
METEOR identifier: | 644702 |
Registration status: | AIHW Data Quality Statements, Standard 24/08/2016 |
Data quality | |
Data quality statement summary: | Summary of key issues The National Health Workforce Data Set (NHWDS) 2015: medical practitioners contains information on the demographics, employment characteristics, work location and work activity of all medical practitioners in Australia who renewed their medical registration with the Medical Board of Australia via the National Registration and Accreditation Scheme (NRAS) that was introduced on 1 July 2010. This is the sixth data set for medical practitioners from the new national registration scheme. The data set comprises registration (including demographic) information provided by the Australian Health Practitioner Regulation Agency (AHPRA) and workforce details obtained by the Medical Workforce Survey. The major issues with data quality for the NHWDS 2015: medical practitioners include:
Description The NHWDS 2015: medical practitioners is a combination of registration and survey data collected through the medical practitioner registration renewal process. Registration data All Medical practitioners must be registered with AHPRA to practise in Australia. Medical practitioners are required by law to renew their registration with the Medical Board of Australia through the NRAS. For initial registration, medical practitioners must use a paper form provided by the AHPRA and provide supplementary supporting documentation. Apart from limited and provisional registrations, medical practitioners can renew their registration either online via the AHPRA website or by using a paper form provided by the AHPRA. Limited and provisional registration renewals are done using paper forms. This information is referred to as ‘registration data’. The majority of medical practitioners are due to renew their registrations on 30 September each year. Limited and provisional registration renewals occur on the anniversary of when the individual practitioner last registered/renewed. Whether for renewal or initial registration, this information is referred to as ‘registration data’. Data collected includes demographic information such as age, sex and country of birth; and details of health qualification(s) and registration status. This is the compulsory component of the registration process. (see registration forms at http://www.medicalboard.gov.au/Registration/Types/General-Registration.aspx). Survey data When medical practitioners renew their registration online they are also asked to complete an online version of the Medical Workforce Survey questionnaire. When medical practitioners renew their registration on a paper form, they are asked to complete a paper version of the Medical Workforce Survey questionnaire. Database creation The AHPRA stores both the online registration data and the online survey information in separate databases. They send these two de-identified data sets to the AIHW, where they are merged into a national data set. The paper registration data and paper survey forms were also received by AHPRA. AHPRA then sent these paper forms to the Commonwealth Department of Health (Health) to be scanned into a data set. Health sent this data set to AIHW for merging with online registration data and data from the online survey forms, and for cleansing and adjustment for non-response to form a nationally consistent data set. The final data set is then known as the National Health Workforce Data Set 2015: medical practitioners. |
---|---|
Institutional environment: | The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia’s health and welfare. It is an independent corporate Commonwealth entity established in 1987, governed by a management board, and accountable to the Australian Parliament through the Health portfolio. The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection. The AIHW also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The AIHW works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting. One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these data sets and to disseminate information and statistics. The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988 (Cwlth), ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality. For further information, see the AIHW website at http://www.aihw.gov.au. The AHPRA is the organisation responsible for the implementation of the NRAS across Australia. The AHPRA works with the National Health Practitioner Boards to regulate health practitioners in the public interest and to ensure a competent and flexible health workforce that meets the current and future needs of the Australian community. Until its closure in August 2014, Health Workforce Australia was responsible for the development of the workforce surveys. Its functions were transferred then to the Department of Health. The Department of Health is a Federal Government agency that has a diverse set of responsibilities reflected in their Vision statement: Better health and wellbeing for all Australians, now and for future generations. The AIHW is the data custodian of the NHWDS 2015: medical practitioners. |
Timeliness: | The NHWDS 2015: medical practitioners is produced from the national registration renewal process, conducted from early August to 30 September. Although the reference time is notionally the renewal date, 30 September, legislation allows for a one month period of grace. Thus, the final registration closure date is one month after the renewal date. The AHPRA allows a further two weeks to allow for mail and data entry delays for completeness. Consequently the extraction of data occurs a month and a half after the renewal date. The Medical Workforce Survey was collected between 1 July and 30 September, as it is administered as part of the registration renewal process. The exceptions to this timetable were in relation to limited and provisional registrations, where registrants are renewed on the anniversary of their commencement. These responses were included with the regular survey respondents. The online data was provided to the AIHW from AHPRA in February 2016 while the paper forms were sent to the AIHW from the Commonwealth Department of Health in January 2016. |
Accessibility: | Results from the NHWDS 2015: medical practitioners are published on the AIHW website at http://www.aihw.gov.au/workforce/medical/. Users can request data not available online or in reports through the AIHW data request management system http://www.aihw.gov.au/custom-data-request-service/, via the Communications, Media and Marketing Unit on (02) 6244 1032 or via email to [email protected]. Requests that take longer than half an hour to compile are charged for on a cost-recovery basis. Access to the master unit record files may be requested through the AIHW Ethics Committee. |
Interpretability: | Descriptions of data items in the National Health Workforce Data Set 2015: medical practitioners are available on request from the Expenditure and Workforce Unit at the AIHW. The survey used by medical practitioners is available from the AIHW website http://www.aihw.gov.au/workforce/medical/. |
Relevance: | Scope and coverage The NHWDS 2015: medical practitioners contains registration details of all registered medical practitioners in Australia at 30 September 2015. Medical practitioners are required by law to be registered with the Medical Board of Australia and must complete the formal registration renewal form(s) to practise in Australia. This is the compulsory component of the renewal process. The Medical Workforce Survey is voluntary and only practitioners who are on the register at the time of the survey and required to renew their registration receive a questionnaire for completion. New registrants registering outside the registration renewal period will not receive a survey form. These practitioners will receive a survey form when they renew their registration the following year, during the registration renewal period. |
Accuracy: | Response rates and mode The NHWDS 2015: medical practitioners contains registration details of all registered medical practitioners in Australia at 30 September 2015. The data set also contains workforce information for registered medical practitioners who completed the Medical Workforce Survey. The overall response rate to the 2015 survey was 92.2%. That is, the number of responses to the survey represented 92.2% of registered medical practitioners. Of these responses, 97.6% completed the 2015 version of the survey online, 1.5% completed the 2015 version of the survey on paper and 0.9% completed the 2014 version of the survey on paper. The group with the lowest response rate was limited registrants (2,988 registrations with a response rate of 25.7%) who only complete paper survey forms on the anniversary of their first registration. As a result not only do they have a low response rate but they are more likely to complete the 2014 version of the survey on paper. Of particular note in 2015, the response rate for the 252 limited registrants in South Australia was only 3.9% and the response rate for the 465 limited registrants in Queensland was only 9.9%. This significantly affects the reliability of estimates for these groups. Registration data from the NRAS The NRAS allows a medical practitioner to record more than one specialty, with up to five specialties recorded in 2015. However, the Health Practitioner Regulation National Law 2009 does not require or enable practitioners to identify their primary speciality. The survey now includes the reporting of hours worked in each specialty. The hours reported (where available) were used to determine which specialty was the primary specialty. Medical Workforce Survey 2015 design In 2013, the online survey questionnaire included for the first time electronic sequencing of questions to automatically guide the respondent to the next appropriate question based on previous responses. This approach to improving the data quality has continued with the 2015 questionnaire. In previous surveys and in the paper version of the survey respondents may have made inconsistent responses. Respondents not correctly following the sequencing instructions for the employment questions may be assigned to an incorrect workforce status or not assigned a status, due to incomplete data. In the electronic version of the 2015 survey, a small number of survey items were filled by an automated process that draws on the registration data of respondents. Inconsistencies between workforce survey and registration data There were a number of inconsistencies between the data sourced from the NRAS and the workforce survey data. In the survey, a number of medical practitioners self-reported the principal area in their main job as ‘specialist’ but had no accredited specialty in their registration details or were accredited as general practitioners. A number of these practitioners had overseas specialist qualifications with limited registration status and also answered ‘specialist-in-training’ questions. Under the Health Practitioner Regulation National Law 2009, specialist registration is available only to medical practitioners who have been assessed by an Australian Medical Council accredited specialist college as being eligible for fellowship. Fellowship is not a pre-requisite for specialist registration. Another small number were found to have surrendered their specialist registration between the time of the survey and the extraction of the registration data. The ‘location of principal practice’ recorded in the registration data was often different from the corresponding details of a practitioner’s main job as self-reported in the survey. This may reflect temporary movement. For example, 10.7% more medical practitioners have the Northern Territory as their derived state/territory (largely based on state/territory of main job in week before survey) than have it as their principal practice location on the AHPRA database. The location was therefore derived based firstly on ‘main job’ information, then on ‘principal practice location’ if the main job location was missing, and subsequently on residential address if the principal practice location was also missing. This derived state and territory of main job is used in all published tables except where otherwise stated. As a consequence of this methodology, medical practitioners who were working overseas but maintained a contact address in Australia have been allocated in state and territory tables to the state or territory where that contact address was, though the majority of them remained classified as ‘overseas’. Quality of location information Some post code fields contained values other than valid post codes, and overseas postal identifiers. Suburb fields sometimes contained invalid suburb names. Where state and postcode information did not agree, the suburb was used to look up a postcode and this was used to decide which of the two were more likely to be correct. Estimation procedures The AIHW uses registration data together with survey data to derive estimates of the total medical practitioner workforce. Not all medical practitioners who receive a survey respond, because it is not mandatory to do so. In deriving the estimates, two sources of non-response to the survey are accounted for:
Imputation methods are used to account for item non-response and survey non-response. Imputation: estimation for item non-response The imputation process involves an initial examination of all information provided by a respondent. If possible, a reasonable assumption is made about any missing information based on responses to other survey questions. For example, if a respondent provides information on hours worked and the area in which they work, but leaves the workforce question blank, it is reasonable to assume that they were employed. Missing values remaining after this process are considered for their suitability for further imputation. Suitability is based on the level of non-response to that item. In imputation, the known probabilities of particular responses occurring are used to assign a response to each record. Imputed values are based on the distribution of responses occurring in the responding sample. Therefore, fundamental to imputing missing values for survey respondents who returned partially completed questionnaires is the assumption that respondents who answer various questions are similar to those who do not. Age values within each state and territory of principal practice are first imputed to account for missing values. Other variables deemed suitable for this process were then imputed. These include hours worked in the week before the survey, principal role of main job, principal area of main job in medicine and work setting of main job. Imputation: estimation for survey non-response In 2013, the methodology for survey non-response was changed from a weighting-based methodology to a randomised sequential hot deck-based imputation similar to that used for imputing unreported hours in previous years. The data were sorted into strata so that imputations were made using survey data from records that have similar registration details. The strata used for imputation were registration type (with limited registrants grouped together and specialist registrants grouped with those who also had general registration), a derived primary specialty categorisation, sex, age group, remoteness area and state, in that order. Donor records were spaced evenly within strata to ensure records were used within the strata an equal number of times plus or minus 1, and that most strata within the hot deck were restricted to within strata imputations. For example, if there were 5 respondents and 12 non-respondents in a cell, the expected number of uses would be 2.4, resulting in each donor being used either 2 or 3 times. This is almost equivalent to a weighting strategy, except that instead of all the data being weighted only the non-registration data are weighted. Because the data were imputed and not weighted, some data may be affected in different ways from that previously published. For example, because a practitioner’s location of main job is most likely to be the same as their registration address, this has been used for the location estimation of non-respondents. Using this estimate rather than weighting will improve the accuracy of estimates for small geographic areas, as previously weighted data would scale up data for individuals across the state/territory and the registration information for records would not be taken into account. For variables not used in the imputation (that is, all variables other than the registration type, derived specialty, remoteness area, state and territory of principal practice, age and sex), it is assumed, for estimation purposes, that respondents and non-respondents have the same characteristics. If the assumption is incorrect, and non-respondents are different from respondents, then the estimates will have some bias. The extent of this cannot be measured without obtaining more detailed information about non-respondents. |
Coherence: | Workforce Survey 2015—coherence with previous data Previously published data for 2010 and 2011 include provisional registrants. As a result, growth between the published 2011 and 2012 data is understated by the order of 3.6%. There were a number of additional questions and additions to questions in the survey between 2012 and 2013 and a few other minor changes in the Medical Workforce Survey data between 2012, 2013, 2014 and 2015; most data from 2011 are considered comparable, though later data do provide additional detail in some cases. Queensland and Western Australia did not provide data for 2010. Only minimal comparisons between the 2010 and other data can be made, and 2010 is generally excluded from analysis. Medical labour force data published by the AIHW before the establishment of the NRAS in 2010 was the result of collated jurisdiction-level occupation-specific surveys (referred to as the AIHW Medical Labour Force Survey). The Medical Workforce Survey from 2010 to 2015 collect similar data items to the AIHW Medical Labour Force Survey; however, the survey methodology has changed, as has the method of obtaining benchmark data on which the numbers of total registrations are based. With the establishment of the AHPRA, there is one source of benchmark data instead of eight, and there is less chance of inconsistency between states and territories and years in the scope of benchmark data. The scope and coverage of the Medical Workforce Survey from 2010 to 2015 are also different to that of the previous surveys because in some states and territories not all types of registered medical practitioners were sent a survey form in the period until 2009. Date of birth, country of initial qualification, specialty of practice and sex are some data items previously collected by the AIHW Medical Labour Force Survey, but now collected by the NRAS. However, data for some of these items are either incomplete or inaccurate (see ‘Accuracy’). Supplementary questions were added in 2012 to collect country of first qualification and country of first specialist qualification in the survey. In 2013, the list of countries was expanded and year of graduation from medical school was added to the survey. Speciality of practice Before 2010, main specialty of practice information was self-reported by registered medical practitioners from a set of 50 statistical categories in the AIHW Medical Labour Force Survey. Speciality of practice, from 2010 to 2015, was extracted at the time of registration renewal by the AHPRA from the NRAS data of legally recognised specialties. There are 85 valid legally-defined specialties and subspecialties in the NHWDS: medical practitioners (for example: ‘cardiologist (physician)’ and ‘general practice’). However, the NRAS does not identify main specialty and the primary specialty is derived by the AIHW. Primary specialties in 2010 and 2011 were derived using their recorded specialties and information from the AIHW Medical Labour Force Survey 2009. A new question was included in the 2012 survey to allow a primary specialty to be derived at the detailed level. Thus, comparison of specialty data for 2012 and later years with results from earlier surveys should be treated with caution. General practitioners A change in the response options for the question about ‘principal area of main job in medicine’, from ‘GP/primary care practitioner’ until 2009 to ‘general practitioner’ in 2011 and 2012 has impacts on the comparability of these responses over time, and time-series data should be used with caution. This may have led to the observed increase in responses in the ‘other clinician’ category. Further refinement to the question has led to extra explanatory text being included in the survey, with the survey response for 2013 and 2015 reading ‘General Practitioner (GP) (excluding AGPT program trainees)’. Similarly the Specialist-in-training category now reads ‘Specialist-in-training (including AGPT program trainees)’. The AIHW estimates that of the order of a thousand medical practitioners have probably answered ‘Specialist-in-training (including AGPT program trainees)’ rather than ‘General Practitioner (GP) (excluding AGPT program trainees)’ when they may have answered differently without the explanatory text. The ‘hospital non-specialist’ category changed to ‘hospital non-specialist (including pre-vocational doctors)’ in 2013, but the AIHW did not find any evidence in the data for a significant effect. Work settings Work setting response categories in the current survey are similar to those until 2009. The current categories are more detailed and directed towards service provision; for example, there are three categories of private practice (‘solo’, ‘group’ and ‘locum’) compared with only one available until 2009. While in 2010 and 2011 the survey form provided a distinction between ‘outpatient’ and ‘other hospital’ settings, the 2012 question included only ‘hospital’ as a response category. From 2013 surveys provided a distinction between ‘outpatient’ and ‘other hospital’ settings. In 2012 and later years, further information on hospital work settings was collected as part of the sector question where a more detailed split was included. Response options for the ‘hours worked by sector’ question were restricted to clinical hours only, whereas the equivalent question in 2011 was a split by total hours. From 2012, the question was also expanded to include categories for clinical hours worked in ‘private rooms’, ‘private hospital’, ‘private other’, ‘public hospital (inpatients)’, ‘public hospital (outpatients)’ and ‘public other’. In 2013, a number of additions to the survey that have carried forward include questions on:
Other changes include:
Due to the differences in data collection methods, including survey design and questionnaire, it is recommended that comparisons between workforce data in the NHWDS: medical practitioners from 2010 onwards and AIHW Labour Force Survey data until 2009 be made with caution. |
Source and reference attributes | |
Submitting organisation: | Australian Institute of Health and Welfare |