Comparing the Effects of Sampling Designs on the Diagnostic Accuracy of Eight Promising Screening Algorithms for Pediatric Bipolar Disorder

BACKGROUND Past studies have used markedly different inclusion and exclusion criteria to form samples used to evaluate diagnostic tests, making it difficult to compare results across studies. The present investigation compared eight screening algorithms in the same sample but under two different design strategies. METHODS The DSM-IV diagnoses were based on a semi-structured diagnostic interview (KSADS) with the parent and youth sequentially. Raters were blind to index test scores. Participants were 216 youths with bipolar spectrum diagnoses and 284 youths with other Axis I diagnoses or no diagnosis. T-tests evaluated whether areas under the curve (AUC) from receiver operating characteristic analyses differed under the two design conditions. RESULTS All of the instruments discriminated bipolar cases better when inclusion and exclusion criteria duplicated those used in phenomenological research studies selecting narrow phenotypic cases (AUCs ranging from .90 to .81). The measures performed less well when more heterogeneous clinical presentations were included [AUCs ranging from .86 to .69, t(8) = 4.99, p = 001]. CONCLUSIONS Results indicate that checklists perform less well discriminating pediatric bipolar disorder under conditions that more closely resemble clinical practice. Test users must consider whether the sampling strategies and participant characteristics used to evaluate tests match the characteristics of their own patients.

[1]  Xiao-Hua Zhou,et al.  Statistical Methods in Diagnostic Medicine , 2002 .

[2]  Rebecca Tillman,et al.  A brief screening tool for a prepubertal and early adolescent bipolar disorder phenotype. , 2005, The American journal of psychiatry.

[3]  M. Monuteaux,et al.  The CBCL predicts DSM bipolar disorder in children: a receiver operating characteristic curve analysis. , 2005, Bipolar disorders.

[4]  J. Calabrese,et al.  Comparing the diagnostic accuracy of six potential screening instruments for bipolar disorder in youths aged 5 to 17 years. , 2004, Journal of the American Academy of Child and Adolescent Psychiatry.

[5]  S. Faraone,et al.  Depression in attention deficit hyperactivity disorder (ADHD) children: "true" depression or demoralization? , 1998, Journal of affective disorders.

[6]  J. Calabrese,et al.  Toward an Evidence-Based Assessment of Pediatric Bipolar Disorder , 2005, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[7]  S. Faraone,et al.  A preliminary meta-analysis of the child behavior checklist in pediatric bipolar disorder , 2003, Biological Psychiatry.

[8]  M. Fristad,et al.  Defining and identifying early onset bipolar spectrum disorder , 2004, Current psychiatry reports.

[9]  M. Gould,et al.  Aggregating data from multiple informants in child psychiatry epidemiological research. , 1992, Journal of the American Academy of Child and Adolescent Psychiatry.

[10]  E. Leibenluft,et al.  Defining clinical phenotypes of juvenile mania. , 2003, The American journal of psychiatry.

[11]  M. DelBello,et al.  DSM-IV mania symptoms in a prepubertal and early adolescent bipolar disorder phenotype compared to attention-deficit hyperactive and normal controls. , 2002, Journal of child and adolescent psychopharmacology.

[12]  R. Gibbons,et al.  Preliminary studies of the reliability and validity of the children's depression rating scale. , 1984, Journal of the American Academy of Child Psychiatry.

[13]  N. Ryan,et al.  Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. , 1997, Journal of the American Academy of Child and Adolescent Psychiatry.

[14]  D. Papolos.  The Bipolar Child: The Definitive and Reassuring Guide to Childhood's Most Misunderstood Disorder , 2000 .

[15]  M. Dulcan,et al.  Psychiatric disorders in youth in juvenile detention. , 2002, Archives of general psychiatry.

[16]  T. Achenbach.  Manual for the child behavior checklist/4-18 and 1991 profile , 1991 .

[17]  S. Faraone,et al.  CBCL clinical scales discriminate prepubertal children with structured interview-derived diagnosis of mania from those with ADHD. , 1995, Journal of the American Academy of Child and Adolescent Psychiatry.

[18]  Phil A. Silva,et al.  DSM-III disorders in preadolescent children. Prevalence in a large sample from the general population. , 1987, Archives of general psychiatry.

[19]  J. Calabrese,et al.  Rapid, continuous cycling and psychiatric co-morbidity in pediatric bipolar I disorder. , 2001, Bipolar disorders.

[20]  J. Calabrese,et al.  Comparing diagnostic checklists for pediatric bipolar disorder in academic and community mental health settings. , 2005, Bipolar disorders.

[21]  Helena Chmura Kraemer,et al.  Evaluating Medical Tests: Objective and Quantitative Guidelines , 1992 .

[22]  M. Williams,et al.  Prepubertal and young adolescent bipolarity versus ADHD: assessment and validity using the WASH-U-KSADS, CBCL and TRF. , 1998, Journal of affective disorders.

[23]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. , 2003, Clinical chemistry.

[24]  J. Calabrese,et al.  Discriminative validity of parent report of hypomanic and depressive symptoms on the General Behavior Inventory. , 2001, Psychological assessment.

[25]  M. Kovács,et al.  Difficulties in diagnosing manic disorders among children and adolescents. , 1992, Journal of the American Academy of Child and Adolescent Psychiatry.

[26]  R. C. Young,et al.  A Rating Scale for Mania: Reliability, Validity and Sensitivity , 1978, British Journal of Psychiatry.

[27]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[28]  E. Youngstrom,et al.  Bipolar Diagnoses in Community Mental Health: Achenbach Child Behavior Checklist Profiles and Patterns of Comorbidity , 2005, Biological Psychiatry.

[29]  D. Sackett.  Evidence-based medicine: how to practice and teach EBM: 2nd ed , 2000 .

[30]  L. Rohde,et al.  Juvenile bipolar disorder in Brazil clinical and treatment findings , 2003, Biological Psychiatry.

[31]  L. Lengua,et al.  Rationally and empirically derived dimensions of children's symptomatology: expert ratings and confirmatory factor analyses of the CBCL. , 2001, Journal of consulting and clinical psychology.

[32]  M. DelBello,et al.  Bipolar Disorder in Childhood and Early Adolescence , 2003 .

[33]  James Robert Brašić,et al.  A children's global assessment scale (CGAS). , 1983, Archives of general psychiatry.

[34]  E. Nottelmann.  National Institute of Mental Health research roundtable on prepubertal bipolar disorder. , 2001, Journal of the American Academy of Child and Adolescent Psychiatry.

[35]  PL Hazell,et al.  Confirmation that Child Behavior Checklist clinical scales discriminate juvenile mania from attention deficit hyperactivity disorder , 1999, Journal of paediatrics and child health.

[36]  D. Miklowitz,et al.  Childhood mania, attention deficit hyperactivity disorder and conduct disorder: a critical review of diagnostic dilemmas. , 2002, Bipolar disorders.

[37]  G. Guyatt,et al.  Users' guides to the medical literature. , 1993, JAMA.

[38]  J. Calabrese,et al.  Clinical decision-making using the General Behavior Inventory in juvenile bipolarity. , 2002, Bipolar disorders.

[39]  J. Calabrese,et al.  Employing parent, teacher, and youth self-report checklists in identifying pediatric bipolar spectrum disorders: an examination of diagnostic accuracy and clinical utility. , 2003, Journal of child and adolescent psychopharmacology.