Validation of the Christo Inventory for Substance-misuse Services (CISS): a simple outcome evaluation tool

https://doi.org/10.1016/S0376-8716(99)00117-9Get rights and content

Abstract

The Christo Inventory for Substance-misuse Services (CISS) was developed as a single page outcome evaluation tool completed by drug/alcohol service workers either from direct client interviews or from personal experience of their client supplemented by existing assessment notes. Its 0–20 unidimensional scale consists of 10 items reflecting clients’ problems with social functioning, general health, sexual/injecting risk behaviour, psychological functioning, occupation, criminal involvement, drug/alcohol use, ongoing support, compliance, and working relationships. Comparison scores indicating low, average or high problem severity were produced by 243 drug users attending a harm minimisation out patient service and 102 alcohol users at an outpatient alcohol service. Means and cut-off scores for abstinence oriented treatments were derived from a 6-month follow-up of 90 treated drug users. Sub sets of the harm minimisation sample were used to derive item alpha, test-retest and inter-rater reliability coefficients of 0.74, 0.82 and 0.82, respectively. The inter-rater coefficient increased to 0.91 when retests were conducted the same day. Among the abstinence oriented treatment sample the CISS produced correlations ranging from 0.43 to 0.99 with the Opiate Treatment Index and measures of trait anxiety, unpleasant life events, poor quality of life and low self-esteem. The simplicity, flexibility and brevity of the CISS make it a useful tool allowing comparison of clients within and between many different service settings.

Introduction

Providers of substance misuse services are frequently required to empirically demonstrate the efficacy of their treatments. However, outcome evaluations pose a challenge for smaller services which may have neither the time nor the expertise to conduct detailed outcome research with multidimensional outcome questionnaires.

The areas considered relevant among substance misuse outcome questionnaires are social functioning, general health, sexual/injecting risk behaviour, psychological functioning, occupation, criminal involvement and drug/alcohol use. Examples of assessment tools in existence include the Addiction Severity Index (ASI; McLellan et al., 1980, McLellan et al., 1992), the Opiate Treatment Index (OTI; Darke et al., 1992) and the Maudsley Addiction Profile (MAP; Marsden et al., 1999). Other factors linked to good outcomes include continued use of ongoing support or aftercare (Christo and Franey, 1995, Ouimette et al., 1998), and treatment compliance as illustrated by length of stay or treatment level attained (Shwartz et al., 1997, Toumbourou et al., 1998).

Current multidimensional outcome questionnaires take a while to complete, require the presence of the client in question, and cannot combine sub sections to produce a single total score for easy analysis by non-researchers. There is still a need for a clinical evaluation instrument that can produce a single score reflective of client problems across all relevant outcome domains. To be acceptable to busy workers, the instrument would have to be short, simple and not dependent on the unreliable attendance of substance misuse service clients. Since workers generally have a wealth of qualitative data from detailed client notes and assessment interviews, it may be assumed that workers are able to furnish required information on their clients’ behalf. Although the information would largely comprise of subjective impressions which may be difficult to validate, a competent worker should be familiar with their client’s status within the relevant outcome areas. Thus the Christo inventory for substance-misuse services (CISS) was developed from elements within the aforementioned references to elicit workers’ assessments of their clients in a quick, standardised and reliable way.

The CISS prototype was used in the evaluation of outcomes of treatment placements purchased by Social Services (Christo, 1998). This pilot study illustrated the scale’s usefulness, sensitivity to change and ease of completion. Feedback from workers was used to refine the nine original items and a tenth item was added reflecting the quality of the working relationship with the client. Therapeutic relationships, pretreatment motivation and program engagement have been shown to be central attributes of effective treatment (Simpson et al., 1997).

The aims of this validation study were to produce reliability data for the refined scale and to standardise it against cross-sectional ‘snapshots’ of criterion groups attending different types of substance misuse services. Concurrent validity data were also produced by comparison to OTI sub scales and some measures of psychological functioning.

Section snippets

Abstinence oriented treatment subjects

These subjects were recruited from eight abstinence oriented treatment facilities in London representing different treatment models: therapeutic community, psychodynamic, homeopathic and Minnesota model (12 step). The homeopathic, and one 12 step treatment were outpatient units, the rest were residential. At recruitment, subjects were required to be six weeks abstinent from their drug of choice and most (96%) were totally abstinent from all drugs. One hundred and seven eligible subjects were

CISS discriminant validity

Outcome was first assessed on the basis of drug use during the month preceding follow-up. Forty-eight subjects reported no drug use during that month and were classified as having a good outcome, their mean CISS score was 2.9 (SD=1.9). Forty-two subjects reported drug use and were classified as having a poor outcome, their mean CISS score was 10.6 (SD=4.3). A two sample t-test revealed these scores to be significantly different (t (54.3)=10.7, P<0.001).

When assessing outcome on the basis of

CISS reliability

The item alpha coefficient of internal consistency was 0.74, the test-retest coefficient was 0.82, and the inter-rater coefficient was 0.82. The inter-rater coefficient increased to 0.91 when retests were conducted the same day. These figures indicated the scale to be of satisfactory reliability.

CISS validity and comparison scores

The CISS demonstrated good face validity as its items were quite acceptable to workers and clients. Participating workers stated that they preferred it relative to other instruments which were less

Acknowledgements

With thanks to Amber Astron, Bob Blizard, Richard Leachman and the Ceres Randal Trust, Dr Christine Franey of the Centre for Research on Drugs and Health Behaviour, John Gordon-Smith and the staff of Hammersmith and Fulham Social services alcohol and drugs team, and Shereen Sadiq of Ealing, Hammersmith & Hounslow Health Authority. Thanks also to the staff and clients of the following London based services: Response Royal Free Drug Service, Camden and Islington Community Trust Alcohol Advisory

References (24)

  • S. Darke et al.

    Development and validation of a multi-dimensional instrument for assessing outcome of treatment among opiate users: the Opiate Treatment Index

    Br. J. Addict.

    (1992)
  • S. Darke et al.

    Estimating drug consumption in opioid users: reliability and validity of a ‘recent use’ episodes method

    Br. J. Addict.

    (1991)
  • Cited by (28)

    • Economic evaluation: A comparison of methadone versus buprenorphine for opiate substitution treatment

      2013, Drug and Alcohol Dependence
      Citation Excerpt :

      In line with the clinical paper, covariate adjustment variables were those baseline variables that were originally considered to have a possibility of being different between the two treatment groups (p ≤ 0.2; Pinto et al., 2010), were used in the models to adjust costs. The covariate variables comprised: gender of the participant, if they had resident children, presence of a physical health diagnosis, presence of a psychological health diagnosis, the number of prescription medications, taking of any psychotropic medications, brief symptom inventory (BSI) (Royse and Drude, 1984) global severity index score, BSI positive symptom distress index score, BSI positive symptom total score, previously injected opiates, currently injecting opiates, any previous methadone prescriptions, any previous buprenorphine prescriptions, previously enrolled in a UK drug interventions programme, partner is drug user, CHRISTO baseline score (Christo et al., 2000) as an overall measure of social functioning, number of illicit drugs currently used, positive test for cocaine at baseline, and positive test for cannabis at baseline. To assess the overall effects of the MST and BST programmes on costs, the four cost categories were statistically analysed for effects of treatment programme choice, success of being retained in the study, and the treatment programme choice by retention interaction effect utilising the complete 361 participant dataset.

    • The SUMMIT Trial:. A field comparison of buprenorphine versus methadone maintenance treatment.

      2010, Journal of Substance Abuse Treatment
      Citation Excerpt :

      To eliminate this discrepancy and prevent possible bias, it was decided at the outset to obtain copies of the first urine toxicology report each month only from the clinical notes. Christo inventories (a measure of social functioning, addictive and risk behavior, and treatment engagement; Christo, Spurrell, & Alcorn, 2000) were completed by key workers at baseline, 3 months, and 6 months. Additional self-report data on alcohol use (brief Alcohol Use Disorder Identification Test, brief AUDIT), illicit drug use (adapted Office for National Statistics questionnaire), and psychological health (Brief Symptom Inventory [BSI]) were collected at baseline, 3 months, and 6 months.

    • Validation of the Chinese version of the Opiate Addiction Severity Inventory (OASI) and the Severity of Dependence Scale (SDS) in non-institutionalized heroin users in China

      2008, Addictive Behaviors
      Citation Excerpt :

      The present study reports on the validity and reliability of the 5-item Chinese version of the SDS. A number of studies showed that quality of life was associated with the severity of drug dependence (Brogly, Mercier, Bruneau, Palepu, & Franco, 2003; Christo et al., 2000). The QOL-DA was used in this study (Fang, 2000).

    View all citing articles on Scopus
    View full text