This is the final of the three supplementary papers updating the NZ ASD Guideline, that I have read and summarised for my blog. Please refer back to the first post for more information on the NZ ASD Guideline itself (published in 2008), and for links to the other two papers.
This third paper was published in April 2011. The New Zealand Guidelines Group (NZGG) reviewed instruments for the screening, diagnosis and assessment of autism and Aspergers disorder. They have released both the 7 page summary, and the main 30 page report. The 7 page summary serves as just that – a summary – that raised a lot of questions for me; so I found it necessary to go back to the full report for the answers. In effect then, this post is my own summary of the main report. For the sake of briefness and ease of accessing the key information, I will variously refer to either report in this post as suits (with supporting links for clarity).
The main report begins by pointing out that these instruments are an important aid to diagnosis; not a substitute for clinical expertise.
The NZGG reviewed 8 instruments for the screening and diagnosis of autism. These 8 were selected according to particular criteria: That they are “listed as appropriate in the New Zealand Autism Spectrum Disorder Guideline (Ministry of Health, 2008); and,
listed in the ‘Practice parameters for the screening and diagnosis of autism’ by the American Academy of Neurology and the Child Neurology Society (Filipek et al, 2000), current as of July 2006 (National Guidelines Clearinghouse, nd).”
The 4 instruments they considered for Aspergers disorder, were selected on slightly different criteria: “listed as appropriate in the New Zealand Autism Spectrum Disorder Guideline (Ministry of Health, 2008); and reviewed in the Mental Measurement Yearbook Tests Online, from the Buros Institute at the University of Nebraska; the Yearbook provides independent, expert testing and review of instruments.”
This total then of 12 instruments, was reviewed using the following considerations: Administrative issues; comparison sample (looking at their trialling and development on people with and without ASD); performance statistics, including sensitivity (detecting someone with ASD), specificity (excluding those without ASD), reliability (including inter-rater reliability (whether different assessors get consistent results), and temporal stability (consistency across time), and internal consistency (the consistency of different items within the test)), and instrument validity (including content validity as to how appropriately it measures ASD, and construct validity as to how it reflects theories of ASD, and discriminant validity, as to whether it can discriminate between 2 distinct phenomena); and licensing arrangements. (After many re-typings, I’m just crossing my fingers that I got all the brackets and semi-colons in the right place, check out the top of page 4 of the main report if you want the slightly lengthier bullet-point approach.)
The main report then proceeds to review all 12 instruments in detail. For ease of cross reference if you’d like to look them each up for yourself, I have included the original paragraph references in bold in the following list of the instruments. First, the 8 autism instruments: 2.1 ADI-R; 2.2 ADOS; 2.3 CARS, 2.4 GARS-2; 2.5 SCQ; 2.6 SRS; 2.7 3di; 2.8 DISCO. And the 4 Aspergers disorder instruments: 2.9 GADS; 2.10 ASDS; 2.11 KADI (which particularly distinguishes Aspergers from high functioning autism); and 2.12 ASSQ.
They then consider particular combinations of the instruments for clinical use, selected on the basis of strength of research, usefulness, suitability for a wide range of disciplines, and allowing for both interviews and observation based assessments. (For a more detailed explanation of their criteria, see page 20 of the main report.)
For autism, CARS and SCQ were the preferred screening instruments: “They are both brief and easy to administer and the research into the reliability and validity is strong. Neither instrument requires training, and both instruments can be used by a wide range of health and educational professionals CARS claims validity for children as young as two years of age, and SCQ for children aged four years or more. The SCQ may be slightly preferable on the basis that it is a newer tool and reflects more recent DSM classifications.”
For autism diagnosis, ADI-R and 3di were the preferred interview-based instruments: “Research indicates sound reliability and validity for both. The ADI-R may be slightly preferable because of its larger research base. Training is required for both instruments, but they can be used by a wide range of health and educational professionals. For both instruments, interviews are time-consuming for both assessor and family. However, the 3di can be abbreviated.”
ADOS is the preferred observation-based instrument for autism diagnosis: “On the criterion that instrument-based assessment should include clinician observation data, the ADOS, as the only clinician observation-based instrument, is preferred (as long as used in combination with information from other sources – see section 2.2.1). Available research findings suggest that it has sound reliability and validity. However, the ADOS can be difficult to administer and training of users is required.”
They specifically note that there is a controversy about using CARS as an observational tool for diagnosis: Despite being recommended for such use by other groups, CARS was developed for screening, and from outdated diagnostic criteria, and doesn’t include some constructs considered important for diagnosis of autism. This needs to be kept in mind for their table of suggested combinations of instruments for the screening and diagnosis of autism.
That table of combinations for autism, is listed as table 2 on page 22 of the main report, and as table 3 on page 4 of the summary. The strongest preferences for combinations are at the top of the chart (the least preferred at the bottom).
The combination of instruments for Aspergers disorder is non-existent essentially, since only KADI qualified as a screening instrument on their criteria, but none of the instruments qualified as diagnostic tools: “The KADI appears to warrant consideration as a preferred instrument for screening for Asperger’s disorder, although initial data only is available. It is brief and relatively easy to administer, and its publishers do not specify requirements as regards user qualifications. In common with all reviewed measures for screening for Asperger’s disorder, further research is warranted… No instruments in this review are validated for making a definitive diagnosis of Asperger’s disorder (see section 2).”
The report concludes by saying that professional bodies that include members who screen for and diagnose autism and Aspergers disorder, should direct their membership towards the review.
I am going to end with the important reminder that these instruments are tools to aid diagnosis; they are not a substitute for clinical expertise.