National Drug Strategy Household Survey 2019– Data Quality Statement
Data Quality Statement Attributes
Identifying and definitional attributes
|Metadata item type:||Data Quality Statement|
|Synonymous names:||Data Quality Statement: 2019 National Drug Strategy Household Survey|
AIHW Data Quality Statements, Standard 16/07/2020
|Data quality statement summary:|
The National Drug Strategy Household Survey (NDSHS) provides estimates every three years of the proportion of the population aged 14 years and older using tobacco, alcohol and illicit drugs. The survey also captures information about drug-related attitudes, perceptions and support for government policy.
The 2019 NDSHS survey was the 13th conducted under the National Drug Strategy. The survey was first undertaken in 1985 and has been undertaken every 3 years since 1995. The data collected through these surveys have contributed to the development of policies for Australia’s response to drug-related issues.
The Australian Government Department of Health commissioned the AIHW to manage the 2019 survey, and the AIHW commissioned Roy Morgan Research to collect the data. A Technical Advisory Group comprising experts in tobacco, alcohol and other drug research supported the AIHW in the management of the survey.
The sample is drawn from private dwellings using stratified, multistage random sampling. In each household, the person aged 14 and over who had the most recent birthday was invited to complete the survey. The respondent could elect to complete the survey via a paper form, online or over the telephone.
The Australian Institute of Health and Welfare (AIHW) is a major national agency set up by the Australian Government under the Australian Institute of Health and Welfare Act 1987(Cwlth) to provide reliable, regular and relevant information and statistics on Australia's health and welfare. It is an independent statutory authority established in 1987, governed by a management board, and accountable to the Australian Parliament through the Health and Ageing portfolio.
The AIHW aims to improve the health and wellbeing of Australians through better health and welfare information and statistics. It collects and reports information on a wide range of topics and issues, ranging from health and welfare expenditure, hospitals, disease and injury, and mental health, to ageing, homelessness, disability and child protection.
The Institute also plays a role in developing and maintaining national metadata standards. This work contributes to improving the quality and consistency of national health and welfare statistics. The Institute works closely with governments and non-government organisations to achieve greater adherence to these standards in administrative data collections to promote national consistency and comparability of data and reporting.
One of the main functions of the AIHW is to work with the states and territories to improve the quality of administrative data and, where possible, to compile national datasets based on data from each jurisdiction, to analyse these datasets and disseminate information and statistics.
The Australian Institute of Health and Welfare Act 1987 (Cwlth), in conjunction with compliance to the Privacy Act 1988 (Cwlth), ensures that the data collections managed by the AIHW are kept securely and under the strictest conditions with respect to privacy and confidentiality.
For further information see the AIHW website www.aihw.gov.au
The AIHW has analysed and reported data from the National Drug Strategy Household Survey (NDSHS) since 1998. In addition, the AIHW has managed the survey process since 2001.
Roy Morgan Research
Roy Morgan Research (RMR) is an Australian market research company founded in 1941. RMR has experience in conducting all forms of research, particularly public opinion polling, attitude studies, social surveys, and large consumer and industrial market surveys.
RMR take pride in maintaining comprehensive in-house production facilities and maintaining the highest quality assurance standards (to AS/NZS/ISO 9001 and ISO 20252 standard for all business processes) in the industry. RMR adheres to the standards set out in the Code of Professional Behaviour of the Australia Market and Social Research Society of Australia.
All RMR staff are familiar with, and adhere to the Information Privacy Principles under the Privacy Act 1988 (Cwlth). As in previous waves, all personnel involved in the 2019 NDSHS project, including interviewers, signed an AIHW Confidentiality Undertaking.
Further details about RMR are available at:
The AIHW has commissioned RMR to undertake at least part of the data collection since 1998.
The NDSHS is conducted approximately every three years over a five to six month period. The 2019 data were collected from 8 April to 22 September 2019.
A preliminary data set was received by the AIHW in late November 2019 and initial data checks were completed in December 2019. The final cleaned dataset was received on 22 January 2020.
The 2019 NDSHS report and related material were released on 16 July 2020.
Results from the 2019 NDSHS are available on the AIHW website. Full published results can be found at National Drug Strategy Household Survey 2019, and include the 2019 NDSHS report, in-brief, supplementary data tables, State and Territory factsheets and interactive data visualisations. In addition, results from the remote Indigenous communities were released on 6 October 2020 and can be found at Developing the evidence base for the National Drug Strategy Household Survey: Remote Indigenous Communities—a case study.
Users can request data not available online by submitting a data request through the AIHW custom data request service. Requests are charged for on a cost-recovery basis.
A confidentialised unit record file will be available for 3rd party analysis through the Australian Data Archive. Access to the master unit record file may be requested through the AIHW Ethics Committee.
Information to aid in interpretation of 2019 NDSHS results may be found in the online Technical Information, available here.
In addition, the 2019 Technical Report, code book and other supporting documentation will be available through the Australian Data Archive website or may be requested from AIHW.
Scope and coverage
The NDSHS collects self-reported information on tobacco, alcohol and illicit drug use and attitudes from persons aged 14 years and over. This is a change in scope in 2019 as surveys from 2004 to 2016 captured persons aged 12 and over. The 2019 NDSHS technical advisory group recommended removing this age group from the NDSHS for the following reasons:
Excluded from sampling were:
The exclusion of people from non-private dwellings and institutional settings, and the difficulty in reaching marginalised people are likely to have affected estimates.
The survey is generally not translated into any other languages and requires high levels of English literacy and the ability to follow skip patterns. However, in 2019 several remote Indigenous communities were included in the NDSHS sampling frame, and a translator was available where required.
The 2019 NDSHS was designed to provide reliable estimates at the national level. The survey was not specifically designed to obtain reliable national estimates for Aboriginal and Torres Strait Islander people. In 2019, the sample proportion of Indigenous Australians aged 14 years and older was similar to population estimates (2.6% compared with 2.4%), however most estimates for Aboriginal and Torres Strait Islander people are based on a sample size of 533 people so results should be interpreted with caution.
The fieldwork was conducted from 8 April to 22 September 2019. Respondents to the survey were asked questions relating to their behaviours, beliefs and experiences covering differing time periods, predominantly over the previous 12 months.
In 2019, data were coded to the statistical area level 1 (SA1). Data are generally published at the national level with a selection of data published at the State/Territory, Remoteness Area, Primary Health Network and statistical area level 4 (SA4) levels.
Data on tobacco and alcohol consumption were collected in accordance with World Health Organization standards and alcohol risk data were reported in accordance with the current 2009 National Health and Medical Research Council ‘Australian Guidelines to Reduce Health Risks from Drinking Alcohol’.
Australian and New Zealand Standard Classification of Occupations (ANZSCO) and Australian and New Zealand Standard Industrial Classification (ANZSIC) codes were used as the code-frame for questions relating to occupation and industry.
The Standard Australian Classification of Countries (SACC) codes were used as the code-frame for the question relating to country of birth.
Types of estimates available
Weighted estimates of drug use prevalence, attitudes and beliefs are most commonly reported. In addition, some population numbers and age-standardised data are available for some aspects of the collection. Time series data are presented for most estimates in the 2019 NDSHS supplementary tables.
The sample was stratified by region (15 strata in total ─ capital city and rest of state for each state and territory, with the exception of the Australian Capital Territory which operated as one stratum). To produce reliable estimates for the smaller states and territories, sample sizes were boosted in Tasmania, the Australian Capital Territory and the Northern Territory. An additional booster sample of 1,000 completed responses was allocated to South Australia.
For capital city strata, statistical areas level 1 (SA1s) were selected with probability proportional to the number of private households calculated from the ABS release 3236.0 - Household and Family Projections, Australia, 2016 to 2041. For the first time in 2019, the major regional centres of Illawarra, Newcastle and Lake Macquarie, Geelong, Cairns, Gold Coast, and Sunshine Coast also used this SA1 selection process, to reduce geographical clustering within the sample.
In all other areas in the ’rest of state’ strata, statistical areas level 2 (SA2s) were selected for the first stage instead, as this had considerable efficiency benefits. SA2s for each stratum were selected with probability proportional to the number of households calculated from 3236.0 - Household and Family Projections, Australia, 2016 to 2041. From within each selected SA2, SA1s were selected with probability proportional to the number of private households calculated in the same way. This random selection process returned 8 SA1s that were identified as remote Indigenous communities in the Northern Territory in 2019.
A starting address within each selected SA1 was randomly selected, and interviewing started at the dwelling next door to this. Interviewers followed a comprehensive set of procedures to select a dwelling, including skip intervals, identifying eligible and ineligible addresses, and dealing with blocks of flats and units.
The method for selecting each household/respondent was different in Indigenous remote communities, as the usual household and respondent selection procedures were not possible to implement in these areas. In some cases, the community provided a purposive selection of respondents and in other cases, the interviewer/chaperone/interpreter purposively selected respondents. The interviewer/member of the community attempted to select a mix of respondents, in terms of age and sex. The impacts of this sampling difference are detailed in the ‘Indigenous data’ section of the data quality statement.
The over-sampling of lesser populated states and territories produced a sample that was not proportional to the state/territory distribution of the Australian population aged 14 years or older. Weighting was applied to adjust for imbalances arising from execution of the sampling and differential response rates, and to ensure that the results relate to the Australian population.
The measure used to indicate reliability of individual estimates reported in 2019 was the Relative Standard Error (RSE). Only estimates with RSEs of less than 25% are considered sufficiently reliable for most purposes. Results subject to RSEs of between 25% and 50% should be considered with caution and those with relative standard errors greater than 50% should be considered as unreliable for most practical purposes.
Programming errors in the online survey
The online survey was tested before and shortly after going live. During this process, some errors in the filtering and sequencing of the online survey were discovered. Each of these errors was fixed immediately upon discovery, and a majority were fixed before the survey was distributed.
However, there was an issue with the Statistical Linkage Key (SLK) section when the survey was launched. This section was not initially presented to people completing the survey online, and respondents could not provide their SLK if they wished to do so. This affected 23 respondents.
Selected individuals could choose to complete the survey via one of three tools (also known as the ‘mode’)—paper form, an online form or via a telephone interview. Certain types of respondents are more likely to choose to complete via the online mode but are likely to provide the same responses if they completed via paper form. However, it is possible that the mode that is used by a respondent could have an impact on the actual information provided, introducing a bias in the data and affecting comparability of data obtained via the different methods.
74% of respondents used the paper form to complete the 2019 NDSHS, 25% completed it online and 0.3% completed it via a telephone interview.
Respondents who elected to use the online form had different demographic characteristics (such as age and level of education) to respondents who used the paper form. Logistic regression analysis was used to test for mode effects, controlling for the known demographics of respondents.
Modelling suggested small to no differences between paper and online completion for drinking status, single occasion risk status and lifetime risk status for alcohol consumption; recent use of inhalants; and recent injecting drug use.
For a majority of drug types, people were slightly more likely to report use in the paper form than the online form. This was true for daily smoking status; recent non-medical use of pain-killers/pain-relievers and opioids, and tranquillisers; recent non-medical use of meth/amphetamines; and recent use of cannabis, cocaine, hallucinogens, ecstasy, and ketamine. The difference in the mode effect of paper and online forms may have had an impact on these estimates. However, other respondent characteristics that were not controlled for in the modelling could also have contributed to these effects.
These differences between paper and online responses should be taken into account when comparing 2019 estimates to 2016 results (when 22% of people completed the survey online) and older years (when the online survey form was not available).
In an attempt to enhance the reliability of estimates in the survey and maximise data quality, a small number of missing and contradictory responses were imputed through a rigorous menu of cross-validation edit and logic checks. For example, if a respondent failed to indicate a lifetime usage response (missing) or answered ‘no—never used’, but then provided detailed responses to subsequent questions (e.g. used in the last 12 months, how used, where used, source of supply) the missing or contradictory response was recoded as ‘yes’. These logic checks have been applied since 1998.
Statistical Linkage Key (SLK) validity
The NDSHS includes a self-complete Statistical Linkage Key (SLK). Approximately 68% of respondents attempted to complete the SLK and about 58% of respondents appear to have fully completed it, equating to 12,848 people. At the time the report was written, no 'cleaning' of the SLK has been undertaken and it is possible that cleaning some of the incomplete SLKs (10.8%) will result in additional completions.
The quality of the SLK will impact on any future linkage of these data and does not otherwise affect the quality of other data collected in this survey.
Non-response bias can potentially occur when selected respondents cannot or will not participate in the survey, or cannot be contacted during the fieldwork period. The magnitude of any non-response bias depends on the level of non-response and the extent of the difference between the characteristics of those people who responded to the survey and those who did not, as well as the extent to which non-response adjustments can be made during estimation.
Response rates and contact rates
A total of 69,741 addresses were approached across Australia to complete the NDSHS questionnaire, of which 45,481 were in-scope for the survey. This represents a contact rate of 65.2%, similar to that achieved in the 2016 survey (65.5%), but higher than the contact rate achieved for the 2013 and 2010 surveys (63.9% and 64.5% respectively).
Of the 45,481 in-scope households contacted, 22,274 questionnaires were received and categorised as being complete and useable, representing a response rate for the 2019 survey of 49.0%. This was similar to the 2013 response rate (49.1%) but lower than the response rates for the 2016 and 2010 surveys (51.1% and 50.6%, respectively).
A low response rate does not necessarily mean that the results are biased. As long as the non-respondents are not systematically different in terms of how they would have answered the questions, there is no bias. Given the nature of the topics in this survey, some non-response bias is expected.
Some survey respondents did not answer all questions, either because they were unable or unwilling to provide a response. The survey responses for these people were retained in the sample, and the missing values were recorded as not answered. No attempt was made to deduce or impute these missing values.
Survey estimates are subject to non-sampling errors that can arise from errors in reporting of responses (for example, failure of respondents’ memories, incorrect completion of the survey form), the unwillingness of respondents to reveal their true responses and higher levels of non-response from certain subgroups of the population.
A limitation of the survey is that the data are self-reported and people may not accurately report information relating to illicit drug use and related behaviours because these activities may be illegal. This means that results relating to illicit drugs may be under-reported. However, any biases are likely to be relatively consistent at the population level over time so wouldn’t be expected to have much effect on trend analysis. Legislation protecting people’s privacy and the use of consistent methodology over time means that the impact of this issue on prevalence is limited.
However, some behaviours may become less—or more—socially acceptable over time which may lead to an increase in socially desirable responses rather than accurate responses. Any potential changes in self-reported behaviours need to be considered when interpreting survey results over time.
The estimation method used takes into account non-response and adjusts for the under representation of some population subgroups in an effort to reduce non-response bias.
The sample was designed to provide a random sample of households within each geographic stratum. Respondents within each stratum were assigned weights to overcome imbalances arising in the design and execution of the sampling. The main weighting took into account geographical stratification, household size, age and sex.
In line with the Australian Bureau of Statistics Standard for Sex and Gender Variables, the ‘Other (please specify)’ response was added to the sex question in the 2016 NDSHS. In 2019, 115 respondents reported their sex as 'other'. These people are included in any ‘persons’ totals presented but are excluded from analysis disaggregated by sex as the data for ‘other’ sex were too unreliable to publish.
Alternative presentation of alcohol questions
Some questions in the NDSHS are presented in ways that save space in the paper form, but may also be more difficult for respondents to understand and answer. In the 2019 survey, the decision was made to trial a different presentation of two questions in the online form, where space is not an issue, to assess whether it would have an impact on how people respond to the questions. Half of online respondents (selected at random) were given questions E15 and E16 in a traditional format, and half in an alternative format.
The change appears to have had an impact on how people responded to the questions. Further analysis is required to assess which of these presentations seems to provide more accurate data, however any change in how people respond has an impact on the comparability of data with previous survey waves.
Question E16 is used in the calculation of alcohol risk variables. To prevent this from affecting comparability with previous data collections, only people who answered the traditional format of E16 were included in the calculation of alcohol risk variables. Additional weights have been included that adjust individual weights within strata to prevent changes in alcohol risk variables from introducing artificial non-response bias. Refer to ‘Technical information’ available here for more information.
The survey was not specifically designed to obtain reliable national estimates for Aboriginal and Torres Strait Islander people. In 2019, the sample proportion of Indigenous Australians aged 14 years and older was similar to population estimates (2.6% compared with 2.4%), however most estimates for Aboriginal and Torres Strait Islander people are based on a sample size of 533 people so results should be interpreted with caution.
In 2019, the stratified, multistage random sampling of SA1s across Australia returned 8 remote Indigenous communities. By chance, all 8 Indigenous communities selected were located in the Northern Territory. Residents within those communities primarily spoke Aboriginal and/or Torres Strait Islander languages. In previous survey waves these SA1s would have been replaced by areas where English is predominantly spoken at home, as the NDSHS uses a self-completion questionnaire (completed via paper form or online survey) and was not available in languages other than English.
Instead of replacing these areas in the 2019 NDSHS, the decision was made to invest in surveying these communities as they are part of the Australian population and the collected data is likely to be of benefit to the people living there. While the respondents from these communities do not form a representative sample of Indigenous Australians, or of people living in remote Indigenous communities, the inclusion of their data does improve the overall representativeness of the results in the 2019 NDSHS. However, the methodology and data collection in these communities was different to the remaining sample, meaning that the results are not directly comparable. New field procedures were implemented and variations to the methodology were required to survey remote Indigenous communities. In addition, 3 out of the 8 remote Indigenous communities completed the survey on a tablet device provided by the fieldwork company, and as a result, effected the responses to some questions due to the following differences:
Due to the differences in the sample selection process, and the unique circumstances of each of the communities, the data collected from the 8 remote Indigenous SA1s is not considered comparable to the data collected from Indigenous people surveyed in non-remote Indigenous communities or the non-Indigenous sample.
As this is the first year that these communities have been surveyed, including these data is likely to cause some differences between 2019 results and results from previous years. However, they also make the national results more representative of the Australian population. The number of respondents from remote Indigenous communities is small compared to the rest of the national sample size (144 compared with 22,130), so observed changes in national results since 2016 are likely to be caused by factors other than their inclusion.
However, smaller disaggregations, such as results pertaining to the Northern Territory, or to Indigenous Australians, are likely to be affected by the inclusion of data from remote Indigenous communities. To preserve trend data and comparisons between 2016 and 2019, data from remote Indigenous communities are excluded from results disaggregated by jurisdiction or Indigenous status in the report.
Surveys in this series commenced in 1985. Over time, modifications have been made to the survey’s methodology and questionnaire design. The 2019 survey differs from previous versions of the survey in some of the questions asked and some minor methodological changes, detailed in the technical information, available at https://www.aihw.gov.au/reports/illicit-use-of-drugs/national-drug-strategy-household-survey-2019/contents/technical-information.
Comparability with 1998 and early surveys
The survey sample design, questionnaire design and sample size were considerably different in 1998 and earlier years (further details can be found in the 2016 NDSHS Data Quality Statement). Comparisons between pre-2001 and 2001 and later data should be avoided where possible.
The 2019 questionnaire was modelled on the 2016 version, to maintain maximum comparability. However, some refinements were made to ensure the questions remained relevant and useful. Only substantial changes that had a notable impact on data quality are included here. For a full list of questionnaire changes in 2019 see the 2019 NDSHS technical information, available at https://www.aihw.gov.au/reports/illicit-use-of-drugs/national-drug-strategy-household-survey-2019/contents/technical-information.
Questions relating to activities while under the influence of alcohol or illicit drugs
The NDSHS asks respondents to indicate whether they had undertaken any activities under the influence of alcohol, or while under the influence of illicit drugs. These questions were unchanged between 2010 and 2016.
In 2019, these questions were moved in an attempt to reduce respondent burden by only asking them to respondents who had drank alcohol or used illicit drugs in the previous 12 months. In the 2010, 2013 and 2016 versions of the survey, the placement of these questions meant the question was generally asked of all respondents, even if they had not consumed alcohol or used an illicit drug, and relied on the respondent to select the response option “I have not drank alcohol/used illicit drugs in the previous 12 months”.
These changes resulted in people responding to the questions differently. The 2019 survey estimates were systematically and significantly different to results seen in the previous three survey iterations. After discussion with the 2019 NDSHS Technical Advisory Group, it was agreed that the changes to these questions meant that the data were no longer comparable and resulted in a break to the time series.
Re-analysis of 2001–2007 data
In 2019, all data tables involving trend data from 2001–2007 were reanalysed. A number of discrepancies were discovered during the re-analysis of past data sets, and some other estimates have been updated or revised. Results in 2019 should be considered correct in cases where they conflict with previously published results.
Revisions to alcohol estimates
Following a review of the code created to calculate a person’s drinking status and discussion with the Technical Advisory Group, it was agreed that a more conservative approach to calculating a person’s drinking status should be taken. Data for people with conflicting responses are reported as missing, rather than imputed based on answers to questions that do not involve drinking frequency.
An additional frequency category was also included, disaggregating the ‘less than weekly’ drinkers into at ‘At least monthly but not weekly’ and ‘Less often than monthly’ to provide further information on frequency of alcohol consumption among occasional drinkers.
The calculation of a person’s drinking status also affects alcohol risk variables—lifetime and single occasion risk. The impact on the majority of the estimates is minor (changed by less than 1 percentage point), but the historical proportions published in 2019 will not match previously published data. However, the trends in alcohol consumption from previous years have remained unchanged.
Revisions to physical abuse estimates
Estimates of physical abuse by someone under the influence of alcohol or illicit drugs have been revised following a review of the methodology for their calculation in 2019. The revised estimates have resulted in slightly fewer people reporting that someone under the influence of alcohol or illicit drugs had physically abused them.
These revisions ensure that the application of the edit methodology is consistent across all modes of completion of the survey and gives primacy to responses to the first question rather than the secondary dependant question.
The change in methodology for calculating these estimates is applicable to data from the 2010, 2013, and 2016 surveys as well as 2019 data. Revised estimates relating to physical abuse will not match previously published data for 2013 and 2016. Revised estimates are not available for 2010 and the previously published 2010 estimates should not be used.
Comparison with other collections
There are a number of nationally representative data sources available to analyse tobacco, alcohol and illicit drug data. Comparisons of data from previous waves of the NDSHS, the National Health Survey and the Australian School Student’s Alcohol and other Drug Survey show variations in estimates. Differences in scope, collection methodology and design may account for this variation and comparisons between collections should be made with caution. Daily smoking rates were substantially higher in remote areas in the NATSIHS than in non-remote areas. The remote Indigenous communities surveyed in the NDSHS—not included in the calculation of the above proportions—also reported a higher daily smoking rate (although this used a different methodology, and was limited to remote Indigenous communities in the Northern Territory).
The most common data sources used for reporting the use of tobacco, alcohol and other drugs by Indigenous Australians are the National Aboriginal and Torres Strait Islander Social Survey (NATSISS), the National Aboriginal and Torres Strait Islander Health Survey (NATSIHS), and the Australian Aboriginal and Torres Strait Islander Health Survey (AATSIHS) which forms part of the Australian Health Survey programme. All of these data sources are collected by the Australian Bureau of Statistics, and while they are not directly comparable due to differences in the methodology, sampling frame, data collection mode, age groups surveyed and the sample size, some analysis has been completed to examine the overall trends in smoking rates provided by each survey (ABS 2017).
The 2018–19 NATSIHS provides the latest data on rates of smoking, alcohol risk and other drug use among Aboriginal and Torres Strait Islander people. The results of the NATSIHS are different to the results obtained for Aboriginal and Torres Strait Islander people from the NDSHS. For example, in the 2018–19 NATSIHS, 37% of people aged 15 and over smoked daily, a decrease from 41% in 2012–13. NDSHS results were generally lower, but followed a similar trend, with the proportion of Indigenous Australians aged 14 and over who smoked daily declining from 32% in 2013 to 27% in 2019.
These differences are likely due to limited data collected from remote Indigenous communities in the NDSHS, while in the NATSIHS, they are deliberately oversampled.
Additionally, the NATSIHS results are weighted by age, gender and geography to give results that are representative of Indigenous Australians at the National, state/territory and non-remote and remote areas. NDSHS results, in contrast, are weighted to give reliable estimates at the national and state/territory level for all people living in Australian households. These two weighting approaches are likely to give different results for Aboriginal and Torres Strait Islander people.
ABS 2017. Aboriginal and Torres Strait Islander Peoples: smoking trends, Australia, 1994 to 2014–15. ABS cat. no. 4737.0. Canberra: ABS. Viewed 18 June 2020.
Source and reference attributes
Tobacco, Alcohol and Other Drugs Unit, Australian Institute of Health and Welfare.
|Steward:||Australian Institute of Health and Welfare|
|Related metadata references:|
Supersedes National Drug Strategy Household Survey 2013 – Data Quality Statement
AIHW Data Quality Statements, Superseded 28/09/2017