National Health Workforce Data Set: medical practitioners 2014: National Health Workforce Data Set, 2014: Data Quality Statement
Data Quality Statement Attributes
Identifying and definitional attributes | |
Metadata item type: | Data Quality Statement |
---|---|
METEOR identifier: | 626795 |
Registration status: | AIHW Data Quality Statements, Standard 26/11/2015 |
Data quality | |
Data quality statement summary: | Summary of key issues The National Health Workforce Data Set (NHWDS) 2014: medical practitioners contains information on the demographics, employment characteristics, work location and work activity of all medical practitioners in Australia who renewed their medical registration with the Medical Board of Australia via the National Registration and Accreditation Scheme (NRAS) that was introduced on 1 July 2010. This is the fifth data set for medical practitioners from the new national registration scheme. The data set comprises registration (including demographic) information provided by the Australian Health Practitioner Regulation Agency (AHPRA) and workforce details obtained by the Medical Workforce Survey. The major issues with data quality for the NHWDS 2014: medical practitioners include: · The data are not directly comparable to those collected in the previous (2009 and earlier) AIHW Medical Labour Force Surveys due to changes in methods and scope, including the change in the method of determining the state or territory of practitioners’ main job in medicine. The data are now nationally consistent and more complete that when based on state and territory-based data collection. · Methodological changes, and in particular the inclusion of registration type and an updated specialty classification, mean that some estimates may be affected by changes between the NHWDS 2011: medical practitioners derivation and the derivation of NHWDS 2012-2013: medical practitioners. · Before 2010, the AIHW Medical Labour Force Survey collected state and territory of registration. The survey was administered by individual state and territory health departments or authorities and treatment of provisional registrants was not consistent across the states and territories. Published data for 2011 included provisional registrants in the employed medical practitioner numbers. From 2012, the data published excludes provisional registrants. Description The NHWDS 2014: medical practitioners is a combination of registration and survey data collected through the medical practitioner registration renewal process. Registration data Medical practitioners are required to renew their registration with the Medical Board of Australia through the NRAS. For initial registration, medical practitioners must use a paper form provided by the AHPRA and provide supplementary supporting documentation. Apart from limited and provisional registrations, medical practitioners can renew their registration either online via the AHPRA website or by using a paper form provided by the AHPRA. Limited and provisional registration renewals are done using paper forms. This information is referred to as ‘registration data’. The majority of medical practitioners are due to renew their registrations on 30 September each year. Limited and provisional registration renewals occur on the anniversary of when the individual practitioner last registered/renewed. Data collected at renewal include demographic information such as age, sex, country of birth, details of health qualification(s), and registration status (see registration forms at http://www.medicalboard.gov.au/Registration/Types/General-Registration.aspx). Survey data When medical practitioners renew their registration online they are also asked to complete an online version of the Medical Workforce Survey questionnaire. When medical practitioners renew their registration on a paper form, they are asked to complete a paper version of the Medical Workforce Survey questionnaire. Database creation The AHPRA stores both the online registration data and the online survey information in separate databases. They send these two de-identified data sets to the AIHW, where they are merged into a national data set. The paper survey forms are collected by AHPRA and sent to the Commonwealth Department of Health to be scanned into a data set. The Commonwealth Department of Health then sends the scanned data to AIHW for merging with the online survey forms and registration data. The merged data is cleansed and adjusted for non-response to form a nationally consistent data set. The final data set is then known as the National Health Workforce Data Set 2014: medical practitioners. |
---|---|
Institutional environment: | The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987 to provide reliable, regular and relevant information and statistics on Australia’s health and welfare. It is an independent statutory authority established in 1987, governed by a management board, and accountable to the Australian Parliament through the Health portfolio. The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection. The AIHW also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The AIHW works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting. One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these data sets and to disseminate information and statistics. The Australian Institute of Health and Welfare Act 1987, in conjunction with compliance to the Privacy Act 1988 (Cwlth), ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality. For further information, see the AIHW website at <http://www.aihw.gov.au>. The AHPRA is the organisation responsible for the implementation of the NRAS across Australia. The AHPRA works with the National Health Practitioner Boards to regulate health practitioners in the public interest and to ensure a competent and flexible health workforce that meets the current and future needs of the Australian community. Until its closure in August 2014, Health Workforce Australia was responsible for the development of the workforce surveys. Its functions were transferred then to the Department of Health. The AIHW receives registration (including demographic) information on medical practitioners via the mandatory national registration process administered by the AHPRA and the voluntary Medical Workforce Survey data collected at the time of registration renewal. The registration and workforce survey data are combined, cleansed and adjusted for non-response to form a national data set known as NHWDS 2014: medical practitioners. The AIHW is the data custodian of the NHWDS 2014: medical practitioners. |
Timeliness: | The NHWDS 2014: medical practitioners is produced from the national registration renewal process, conducted from early August to 30 September. Although the reference time is notionally the renewal date, 30 September, legislation allows for a one month period of grace. Thus, the final registration closure date is one month after the renewal date. The AHPRA allows a further two weeks to allow for mail and data entry delays for completeness. Consequently the extraction of data occurs a month and a half after the renewal date. The Medical Workforce Survey was collected between 1 July and 30 September, as it is administered as part of the registration renewal process. The exceptions to this timetable were in relation to limited and provisional registrations, where registrants are renewed on the anniversary of their commencement. These responses were included with the regular survey respondents. The online data was provided to the AIHW from AHPRA in December 2014 while the paper forms were sent to the AIHW from the Commonwealth Department of Health on 28 April 2015. The medical practitioner 2014 data is to be published by AIHW in December 2015. |
Accessibility: | Results from the NHWDS 2014: medical practitioners are published on the AIHW website at http://www.aihw.gov.au/workforce/medical/. Users can request data not available online or in reports through the AIHW data request management system http://www.aihw.gov.au/custom-data-request-service/, via the Communications, Media and Marketing Uniton (02) 6244 1032 or via email to [email protected]. Requests that take longer than half an hour to compile are charged for on a cost-recovery basis. Access to the master unit record files may be requested through the AIHW Ethics Committee. |
Interpretability: | Descriptions of data items in the National Health Workforce Data Set 2014: medical practitioners are available on request from the Expenditure and Workforce Unit at the AIHW. The survey used by medical practitioners is available from the AIHW website http://www.aihw.gov.au/workforce/medical/. |
Relevance: | Scope and coverage The NHWDS 2014: medical practitioners contains registration details of all registered medical practitioners in Australia at 30 September 2014. Medical practitioners are required by law to be registered with the Medical Board of Australia and must complete the formal registration renewal form(s) to practise in Australia. This is the compulsory component of the renewal process. The Medical Workforce Survey is voluntary and only practitioners who are on the register at the time of the survey and required to renew their registration receive a questionnaire for completion. New registrants registering outside the registration renewal period will not receive a survey form. These practitioners will receive a survey form when they renew their registration the following year, during the registration renewal period. |
Accuracy: | Response rates and mode The NHWDS 2014: medical practitioners contains registration details of all registered medical practitioners in Australia at 30 September 2014. The data set also contains workforce information for registered medical practitioners who completed the Medical Workforce Survey. The overall response rate to the 2014 survey was 91.8%. That is, the number of responses to the survey represented 91.8% of registered medical practitioners. Of these responses, 96.3% completed the 2014 version of the survey online, 3.1% completed the 2014 version of the survey on paper and 0.6% completed the 2013 version of the survey on paper. The group with the lowest response rate was limited registrants (4,125 registrations with a response rate of 25.0%) who only complete paper survey forms on the anniversary of their first registration. As a result not only do they have a low response rate but they are more likely to complete the 2013 version of the survey on paper. This significantly affects the reliability of estimates for this group. Registration data from the NRAS Some data items collected as part of the previous AIHW Medical Labour Force Survey, such as date of birth, sex and specialty of practice, are now data items included as part of the registration and renewal process. The NRAS allows a medical practitioner to record more than one specialty, with up to five specialties recorded in 2014. However, the Health Practitioner Regulation National Law 2009 does not require or enable practitioners to identify their primary speciality. The survey now includes the reporting of hours worked in each specialty. The hours reported (where available) were used to determine which specialty was the primary specialty. Medical Workforce Survey 2014 design In 2013, the online survey questionnaire included for the first time electronic sequencing of questions to automatically guide the respondent to the next appropriate question based on previous responses. This approach to improving the data quality has continued with the 2014 questionnaire. In previous surveys and in the paper version of the survey respondents may have made inconsistent responses. Respondents not correctly following the sequencing instructions for the employment questions may be assigned to an incorrect workforce status or not assigned a status, due to incomplete data. In the electronic version of the 2014 survey, a small number of survey items were filled by an automated process that draws on the registration data of respondents. For two survey items, the automated process may not have functioned as intended. The survey items are meant to determine whether the respondent: · was working in General Practice with or without specialist registration in general practice (Question 13); or · has a specialist registration in medicine (including specialist registration in general practice) (Question 22). The response distributions for these two survey items do not reflect what is known about specialist registrations directly from the actual registration data, nor are they consistent with the corresponding distributions from 2013. Inconsistencies between workforce survey and registration data There were a number of inconsistencies between the data sourced from the NRAS and the workforce survey data. In the survey, a number of medical practitioners self-reported the principal area in their main job as ‘specialist’ but had no accredited specialty in their registration details or were accredited as general practitioners. A number of these practitioners had overseas specialist qualifications with limited registration status and also answered ‘specialist-in-training’ questions. Under the Health Practitioner Regulation National Law 2009, specialist registration is available only to medical practitioners who have been assessed by an Australian Medical Council accredited specialist college as being eligible for fellowship. Fellowship is not a pre-requisite for specialist registration. Another small number were found to have surrendered their specialist registration between the time of the survey and the extraction of the registration data. The ‘location of principal practice’ recorded in the registration data was often different from the corresponding details of a practitioner’s main job as self-reported in the survey. This may reflect temporary movement. For example, 14.5% more medical practitioners have the Northern Territory as their derived state/territory (largely based on state/territory of main job in week before survey) than have it as their principal practice location on the AHPRA database. The decision was therefore taken to use a derived location based firstly on ‘main job’ information, then on ‘principal practice location’ if the main job location was missing, and subsequently on residential address if the principal practice location was also missing. This derived state and territory of main job is used in all published tables except where otherwise stated. As a consequence of this methodology, medical practitioners who were working overseas but maintained a contact address in Australia have been allocated in state and territory tables to the state or territory where that contact address was, though the majority of them remained classified as ‘overseas’. Quality of location informationSome post code fields contained values other than valid post codes, and overseas postal identifiers. Suburb fields sometimes contained invalid suburb names. Where state and postcode information did not agree, the suburb was used to look up a postcode and this was used to decide which of the two were more likely to be correct. Estimation procedures The AIHW uses registration data together with survey data to derive estimates of the total medical practitioner workforce. Not all medical practitioners who receive a survey respond, because it is not mandatory to do so. In deriving the estimates, two sources of non-response to the survey are accounted for: · item non-response—occurs as some respondents return partially completed surveys. Some survey records were so incomplete that it was decided to omit them from the reported survey data. · survey non-response—occurs because not all registered medical practitioners who receive a questionnaire respond. Imputation methods are used to account for item non-response and survey non-response. Imputation: estimation for item non-responseThe imputation process involves an initial examination of all information provided by a respondent. If possible, a reasonable assumption is made about any missing information based on responses to other survey questions. For example, if a respondent provides information on hours worked and the area in which they work, but leaves the workforce question blank, it is reasonable to assume that they were employed. Missing values remaining after this process are considered for their suitability for further imputation. Suitability is based on the level of non-response to that item. In imputation, the known probabilities of particular responses occurring are used to assign a response to each record. Imputed values are based on the distribution of responses occurring in the responding sample. Therefore, fundamental to imputing missing values for survey respondents who returned partially completed questionnaires is the assumption that respondents who answer various questions are similar to those who do not. Age values within each state and territory of principal practice are first imputed to account for missing values. Other variables deemed suitable for this process were then imputed. These include hours worked in the week before the survey, principal role of main job, principal area of main job in medicine and work setting of main job. Imputation: estimation for survey non-responseIn 2013, the methodology for survey non-response was changed from a weighting-based methodology to a randomised sequential hot deck-based imputation similar to that used for imputing unreported hours in previous years. The data were sorted into strata so that imputations were made using survey data from records that have similar registration details. The strata used for imputation were registration type (with limited registrants grouped together and specialist registrants grouped with those who also had general registration), a derived primary specialty categorisation, sex, age group, remoteness area and state, in that order. Donor records were spaced evenly within strata to ensure records were used within the strata an equal number of times plus or minus 1, and that most strata within the hot deck were restricted to within strata imputations. For example, if there were 5 respondents and 12 non-respondents in a cell, the expected number of uses would be 2.4, resulting in each donor being used either 2 or 3 times. This is almost equivalent to a weighting strategy, except that instead of all the data being weighted only the non-registration data are weighted. Because the data were imputed and not weighted, some data may be affected in different ways from that previously published. For example, because a practitioner’s location of main job is most likely to be the same as their registration address, this has been used for the location estimation of non-respondents. Using this estimate rather than weighting will improve the accuracy of estimates for small geographic areas, as previously weighted data would scale up data for individuals across the state/territory and the registration information for records would not be taken into account. For variables not used in the imputation (that is, all variables other than the registration type, derived specialty, remoteness area, state and territory of principal practice, age and sex), it is assumed, for estimation purposes, that respondents and non-respondents have the same characteristics. If the assumption is incorrect, and non-respondents are different from respondents, then the estimates will have some bias. The extent of this cannot be measured without obtaining more detailed information about non-respondents. |
Coherence: | Workforce Survey 2014—coherence with previous data Previously published data for 2010 and 2011 include provisional registrants. As a result, growth between the published 2011 and 2012 data is understated by the order of 3.6%. There were a number of additional questions and additions to questions in the survey between 2012 and 2013 and a few other minor changes in the Medical Workforce Survey data between 2012, 2013 and 2014, Most data from 2011 are considered comparable, though later data do provide additional detail in some cases. Queensland and Western Australia did not provide data for 2010.Only minimal comparisons between the 2010 and other data can be made, and 2010 is generally excluded from analysis. Medical labour force data published by the AIHW before the establishment of the NRAS in 2010 was the result of collated jurisdiction-level occupation-specific surveys (referred to as the AIHW Medical Labour Force Survey). The Medical Workforce Survey from 2010 to 2014 collect similar data items to the AIHW Medical Labour Force Survey; however, the survey methodology has changed, as has the method of obtaining benchmark data on which the numbers of total registrations are based. With the establishment of the AHPRA, there is one source of benchmark data instead of eight, and there is less chance of inconsistency between states and territories and years in the scope of benchmark data. The scope and coverage of the Medical Workforce Survey from 2010 to 2014 are also different to that of the previous surveys because in some states and territories not all types of registered medical practitioners were sent a survey form. Date of birth, country of initial qualification, specialty of practice and sex are some data items previously collected by the AIHW Medical Labour Force Survey, but now collected by the NRAS. However, data for some of these items are either incomplete or inaccurate (see ‘Accuracy’). Supplementary questions were added in 2012 to collect country of first qualification and country of first specialist qualification in the survey. In 2013, the list of countries was expanded and year of graduation from medical school was added to the survey. Speciality of practiceBefore 2010, main specialty of practice information was self-reported by registered medical practitioners from a set of 50 statistical categories in the AIHW Medical Labour Force Survey. Speciality of practice, from 2010 to 2014, was extracted at the time of registration renewal by the AHPRA from the NRAS data of legally recognised specialties. There are 85 valid legally-defined specialties and subspecialties in the NHWDS: medical practitioners, (for example: ‘cardiologist (physician)’ and ‘general practice’). However, the NRAS does not identify main specialty and the primary specialty is derived by the AIHW. Primary specialties in 2010 and 2011 were derived using their recorded specialties and information from the AIHW Medical Labour Force Survey 2009. A new question was included in the 2012 survey to allow a primary specialty to be derived at the detailed level. Thus, comparison of specialty data for 2012 and later years with results from earlier surveys should be treated with caution. General practitionersA change in the response options for the question about ‘principal area of main job in medicine’, from ‘GP/primary care practitioner’ before 2010 to ‘general practitioner’ in 2011 and 2012 has impacts on the comparability of these responses over time, and time-series data should be used with caution. This may have led to the observed increase in responses in the ‘other clinician’ category. Further refinement to the question has led to extra explanatory text being included in the survey, with the survey response for 2013 and 2014 reading ‘General Practitioner (GP) (excluding AGPT program trainees)’. Similarly the Specialist-in-training category now reads ‘Specialist-in-training (including AGPT program trainees)’. The AIHW estimates that of the order of a thousand medical practitioners have probably answered ‘Specialist-in-training (including AGPT program trainees)’ rather than ‘General Practitioner (GP) (excluding AGPT program trainees)’ when they may have answered differently without the explanatory text. The ‘hospital non-specialist‘ category changed to ‘hospital non-specialist (including pre-vocational doctors)‘ in 2013, but the AIHW did not find any evidence in the data for a significant effect. Work settingsWork setting response categories in the current survey are similar to those before 2010. The current categories are more detailed and directed towards service provision; for example, there are three categories of private practice (‘solo’, ‘group’ and ‘locum’) compared with only one available before 2010. While in 2010 and 2011 the survey form provided a distinction between 'outpatient’ and ‘other hospital’ settings, the 2012 question included only ‘hospital’ as a response category. The 2013 and 2014 surveys provided a distinction between 'outpatient’ and ‘other hospital’ settings. In 2012 and later years, further information on hospital work settings was collected as part of the sector question where a more detailed split was included. Response options for the ‘hours worked by sector’ question were restricted to clinical hours only, whereas the equivalent question in 2011 was a split by total hours. From 2012, the question was also expanded to include categories for clinical hours worked in ‘private rooms’, ‘private hospital’, ‘private other’, ‘public hospital (inpatients)’, ‘public hospital (outpatients)’ and ‘public other’. In 2013, a number of additions to the survey that have carried forward into 2014 include new questions on: · Overseas field of medicine for registrants employed in medicine overseas. · Occupation for registrants employed in a non-medical occupation. General practitioners (GP) (excluding AGPT program trainees) were asked if they were working in general practice with a specialist registration, and if they were working without a specialist registration they were asked if they were a RACGP/ACRRM/RVTS trainee. · Hospital non-specialists were asked what their position in the hospital was. · Hospital non-specialists were also asked if it was their intent to study to become a specialist and what specialty they were intending to study. Other changes include: · Changing the data collected for respondents who were also working in another rural or remote location. The reported time variable was changed from hours per week in 2012 to a selection of days per week, fortnight, month, quarter or year. AIHW converted all of these responses to a common ‘days per month’ variable. · Questions on specialty of training have been expanded from those included in the 2012 survey to allow two possible fields of study in the survey. There is no ordered structure to the answers, i.e. neither field 1 nor field 2 could be called a primary specialty of training. Additional paired questions on commencement year, year of study, and intended year of completion were also added in 2013. · The 2012 online version of the survey potentially allowed up to 7 specialties of training but this was not apparently intended. · Specialists were asked to identify how many hours they worked in each of their specialties and the primary specialty was chosen on the basis of the maximum number of hours worked. · Supplementary explanation was added to the categories used in 2012 for principal area of main job in medicine. A separate text box after ‘other’ allowed respondents to describe their main job if they did not self-identify with one of the presented categories. In 2013, the AIHW used that information to recode data for some people where it was apparent that the text could be recoded to one of the existing categories. For example, ‘pathology’ was recoded to ‘clinical’. Due to the differences in data collection methods, including survey design and questionnaire, it is recommended that comparisons between workforce data in the NHWDS: medical practitioners from 2010 onwards and AIHW Labour Force Survey data before 2010 be made with caution. |
Source and reference attributes | |
Submitting organisation: | Australian Institute of Health and Welfare |