Identifying thresholds for clinically meaningful change among clients of drug and alcohol services using the Australian Treatment Outcomes Profile

Abstract

The Austraian Treatment Outcomes Profile (ATOP) is a brief clinical outcomes tool used widely in the Australian alcohol and other drugs treatment sector to monitor clients' substance use, health, wellbeing and clinical risk factors. It has demonstrated reliability and validity, and has recommended clinical cut-offs for assessing single-occasion client-rated health scores. This study determined clinically meaningful change thresholds for ATOP substance use and health and wellbeing variables for use by clinicians in monitoring client progress, and for quality improvement and service evaluation. A framework for assessing clinically meaningful changes scores was developed by (1) calculating statistically reliable change thresholds using data-driven techniques with a reference sample of clinical ATOP data and (2) conducting a multi-disciplinary subject matter expert group to review the utility and validity of data-derived clinically meaningful change. The study was conducted within Outpatient Alcohol and Other Drug treatment services in New South Wales, Australia. The reference sample comprised 6100 ATOPs from clients at entry to public outpatient Alcohol and Other Drug treatment services; the subject matter expert group comprised 29 key stakeholders from the specialist alcohol and other drug treatment sector. We used the Reliable Change Index method to calculate clinically meaningful change thresholds for ATOP variables. For substance use variables, a change of 30% in days of use in the last 28 (minimum 4 days) was the threshold for clinically meaningful change for substance use; for health and wellbeing variables, a change of 2 or more points in psychological health, physical health or quality of life scores (measured on 0-10 scales) was the minimum clinically meaningful change. Clinically meaningful change thresholds have been proposed for Australian Treatment Outcomes Profile substance use and health and wellbeing items, based on statistical reliability and subject matter expert assessment. These will be used in the development of an outcomes metric for assessing change and assigning meaning in aggregated data for evaluation of services.