Perspective Enabling a learning healthcare system with automated computer protocols that produce replicable and personalized clinician actions Perspective

Clinical decision-making is based on knowledge, expertise, and authority, with clinicians approving almost every intervention—the starting point for delivery of “All the right care, but only the right care,” an unachieved healthcare quality improvement goal. Unaided clinicians suffer from human cognitive limitations and biases when decisions are based only on their training, expertise, and experience. Electronic health records (EHRs) could improve healthcare with robust decision-support tools that reduce unwarranted variation of clinician decisions and actions. Current EHRs, focused on results review, documentation, and accounting, are awkward, time-consuming, and contribute to clinician stress and burnout. Decision-support tools could reduce clinician burden and enable replicable clinician decisions and actions that personalize patient care. Most current clinical decisionsupport tools or aids lack detail and neither reduce burden nor enable replicable actions. Clinicians must provide subjective interpretation and missing logic, thus introducing personal biases and mindless, unwarranted, variation from evidence-based practice. Replicability occurs when different clinicians, with the same patient information and context, come to the same decision and action. We propose a feasible subset of therapeutic decisionsupport tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). eActions enable different clinicians to make consistent decisions and actions when faced with the same patient input data. eActions embrace good everyday decision-making informed by evidence, experi-ence, EHR data, and individual patient status. eActions can reduce unwarranted variation, increase quality of clinical care and research, reduce EHR noise, and could enable a learning healthcare system.

[1]  Guilherme Del Fiol,et al.  Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout? , 2020, J. Am. Medical Informatics Assoc..

[2]  J. Henrich The WEIRDest People in the World: How the West Became Psychologically Peculiar and Particularly Prosperous , 2021 .

[3]  D. Ashcroft,et al.  Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar , 2020, BMJ.

[4]  A. Butte,et al.  The Case for Algorithmic Stewardship for Artificial Intelligence and Machine Learning Technologies. , 2020, JAMA.

[5]  Brian C. Stagg,et al.  Using clinical decision support systems to bring predictive models to the glaucoma clinic. , 2020, Ophthalmology. Glaucoma.

[6]  "Pseudodoxia Pediatrica". , 2020, JAMA.

[7]  C. Hassager,et al.  Performance of a guideline-recommended algorithm for prognostication of poor neurological outcome after cardiac arrest , 2020, Intensive Care Medicine.

[8]  T. Hoffmann,et al.  Waste in covid-19 research , 2020, BMJ.

[9]  Julie K. Taitsman,et al.  Commercial Influences on Electronic Health Records and Adverse Effects on Clinical Decision Making. , 2020, JAMA internal medicine.

[10]  D. DeMets,et al.  When Can Intermediate Outcomes Be Used as Surrogate Outcomes? , 2020, JAMA.

[11]  Lawrence O Gostin,et al.  US Emergency Legal Responses to Novel Coronavirus: Balancing Public Health and Civil Liberties. , 2020, JAMA.

[12]  D. Berwick,et al.  Enhancing Private Sector Health System Preparedness for 21st-Century Health Threats: Foundational Principles From a National Academies Initiative. , 2020, JAMA.

[13]  O. Fix,et al.  Faculty Opinions recommendation of Physician time spent using the electronic health record during outpatient encounters: A descriptive study. , 2020 .

[14]  J. Overhage,et al.  Physician Time Spent Using the Electronic Health Record During Outpatient Encounters , 2020, Annals of Internal Medicine.

[15]  Marzyeh Ghassemi,et al.  Challenges to the Reproducibility of Machine Learning Models in Health Care. , 2020, JAMA.

[16]  Jodyn E. Platt,et al.  An Analysis of the Learning Health System in Its First Decade in Practice: Scoping Review (Preprint) , 2019 .

[17]  Michael E Matheny,et al.  Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. , 2019, JAMA.

[18]  S. Greenfield Can expert bias be reduced in medical guidelines? , 2019, BMJ.

[19]  Paul P. Lee,et al.  Waste in the US Health Care System-Insights For Vision Health. , 2019, JAMA ophthalmology.

[20]  Philipp Berens,et al.  On the ethics of algorithmic decision-making in healthcare , 2019, Journal of Medical Ethics.

[21]  Edward R Melnick,et al.  The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians. , 2019, Mayo Clinic proceedings.

[22]  W. Shrank,et al.  Waste in the US Health Care System: Estimated Costs and Potential for Savings. , 2019, JAMA.

[23]  D. Berwick Elusive Waste: The Fermi Paradox in US Health Care. , 2019, JAMA.

[24]  M. Mcclellan,et al.  Toward Evidence-Based Policy Making to Reduce Wasteful Health Care Spending. , 2019, JAMA.

[25]  H. Bauchner,et al.  Waste in the US Health Care System. , 2019, JAMA.

[26]  N. Tangri,et al.  Quality of Chronic Kidney Disease Management in Canadian Primary Care , 2019, JAMA network open.

[27]  Philip J. Kroth,et al.  Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout , 2019, JAMA network open.

[28]  J. Bhattacharya,et al.  Association of residency work hour reform with long term quality and costs of care of US physicians: observational study , 2019, BMJ.

[29]  Gloria P. Lipori,et al.  MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery , 2019, Annals of surgery.

[30]  Owen Johnson,et al.  A framework for analysing learning health systems: Are we removing the most impactful barriers? , 2019, Learning health systems.

[31]  W. Self,et al.  Bag‐Mask Ventilation during Tracheal Intubation of Critically Ill Adults , 2019, The New England journal of medicine.

[32]  A. Jha,et al.  Eliminating Wasteful Health Care Spending-Is the United States Simply Spinning Its Wheels? , 2019, JAMA cardiology.

[33]  R. Heesen Why the Reward Structure of Science Makes Reproducibility Problems Inevitable , 2018 .

[34]  J. Orme,et al.  Lower Glucose Target Is Associated With Improved 30‐Day Mortality in Cardiac and Cardiothoracic Patients , 2018, Chest.

[35]  R. Moreau,et al.  Albumin infusion in spontaneous bacterial peritonitis: another brick off the wall? , 2018, Annals of Intensive Care.

[36]  S. Rees,et al.  An Open-Loop, Physiologic Model–Based Decision Support System Can Provide Appropriate Ventilator Settings , 2018, Critical care medicine.

[37]  Jesse M. Ehrenfeld,et al.  Balanced Crystalloids versus Saline in Critically Ill Adults , 2018, Journal Club AINS.

[38]  A. Morris Human Cognitive Limitations. Broad, Consistent, Clinical Application of Physiological Principles Will Require Decision Support. , 2018, Annals of the American Thoracic Society.

[39]  William Marsh,et al.  Learning health systems: the research community awareness challenge , 2018, BMJ Health & Care Informatics.

[40]  S. McDonough,et al.  Education of healthcare professionals for preventing pressure ulcers (Protocol) , 2015 .

[41]  M. Paul Antibiotic Prescribing for Nonbacterial Acute Upper Respiratory Infections in Elderly Persons , 2017, Annals of Internal Medicine.

[42]  Carol E. Nicholson,et al.  Variability in Usual Care Mechanical Ventilation for Pediatric Acute Respiratory Distress Syndrome: Time for a Decision Support Protocol?* , 2017, Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.

[43]  Carol E. Nicholson,et al.  Potential Acceptability of a Pediatric Ventilator Management Computer Protocol* , 2017, Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.

[44]  Adam Wierman,et al.  Thinking Fast and Slow , 2017, SIGMETRICS Perform. Evaluation Rev..

[45]  Clotilde Rambaud-Althaus,et al.  A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): A randomized, controlled non-inferiority trial , 2017, PLoS medicine.

[46]  Ziad Obermeyer,et al.  Lost in Thought - The Limits of the Human Mind and the Future of Medicine. , 2017, The New England journal of medicine.

[47]  Jessica Nelson,et al.  Facing Addiction in America: The Surgeon General’s Report on Alcohol, Drugs, and Health: A Commentary , 2017 .

[48]  C P Friedman,et al.  Toward an Information Infrastructure for Global Health Improvement , 2017, Yearbook of Medical Informatics.

[49]  V. Nadkarni,et al.  Tight Glycemic Control in Critically Ill Children , 2017, The New England journal of medicine.

[50]  Brendan Delaney,et al.  The science of Learning Health Systems: Foundations for a new journal , 2016, Learning health systems.

[51]  J. Flier,et al.  Irreproducibility of published bioscience research: Diagnosis, pathogenesis and therapy , 2016, Molecular metabolism.

[52]  S. Mehta,et al.  An Official American Thoracic Society/American College of Chest Physicians Clinical Practice Guideline: Liberation from Mechanical Ventilation in Critically Ill Adults. Rehabilitation Protocols, Ventilator Liberation Protocols, and Cuff Leak Tests , 2017, American journal of respiratory and critical care medicine.

[53]  Mariarosaria Taddeo,et al.  The ethics of algorithms: Mapping the debate , 2016, Big Data Soc..

[54]  T. Rice,et al.  Randomized Trial of Video Laryngoscopy for Endotracheal Intubation of Critically Ill Adults* , 2016, Critical care medicine.

[55]  Donald M. Berwick,et al.  Era 3 for Medicine and Health Care. , 2016, JAMA.

[56]  John P. A. Ioannidis,et al.  Why Most Clinical Research Is Not Useful , 2016, PLoS medicine.

[57]  L. Fox,et al.  Inpatient dermatology consultative programs: A continued need, tools for needs assessment for curriculum development, and a call for new methods of teaching. , 2016, Journal of the American Academy of Dermatology.

[58]  Peter J Pronovost,et al.  Re-examining high reliability: actively organising for safety , 2016, BMJ Quality & Safety.

[59]  Alan H. Morris,et al.  An in silico method to identify computer-based protocols worthy of clinical study: An insulin infusion protocol use case , 2016, J. Am. Medical Informatics Assoc..

[60]  Jane L. Risen Believing what we do not believe: Acquiescence to superstitious beliefs and other powerful intuitions. , 2016, Psychological review.

[61]  Thorsten Gerber,et al.  Judgment And Decision Making An Interdisciplinary Reader , 2016 .

[62]  Andrew M. Rosenfield,et al.  NOISE: How to overcome the high, hidden cost of inconsistent decision making , 2016 .

[63]  D. Mukamana,et al.  The “treatment gap” in global mental health reconsidered: sociotherapy for collective trauma in Rwanda , 2015, European journal of psychotraumatology.

[64]  M. Samore,et al.  Evaluation of Outpatient Parenteral Antimicrobial Therapy at a Veterans Affairs Hospital , 2015, Infection Control & Hospital Epidemiology.

[65]  J. Kaufman The concept of risk in comparative effectiveness research. , 2015, The New England journal of medicine.

[66]  G. Bernard,et al.  Chlorhexidine bathing and health care-associated infections: a randomized clinical trial. , 2015, JAMA.

[67]  William W. Stead,et al.  Toward a science of learning systems: a research agenda for the high-functioning Learning Health System , 2014, J. Am. Medical Informatics Assoc..

[68]  Kevin Taaffe,et al.  Human Performance Engineering , 2015 .

[69]  Chris Arney Nudge: Improving Decisions about Health, Wealth, and Happiness , 2015 .

[70]  S. Manolagas,et al.  Reproducibility of Results in Preclinical Studies: A Perspective From the Bone Field , 2014, Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research.

[71]  Robert S Saunders,et al.  Engineering a better health care system: a report from the President's Council of Advisors on Science and Technology. , 2014, JAMA.

[72]  David C. Kale,et al.  Algorithms to Estimate PaCO2 and pH Using Noninvasive Parameters for Children with Hypoxemic Respiratory Failure , 2014, Respiratory Care.

[73]  I. Kohane,et al.  Finding the missing link for big biomedical data. , 2014, JAMA.

[74]  J. Orme,et al.  Coefficient of glucose variation is independently associated with mortality in critically ill patients receiving intravenous insulin , 2014, Critical Care.

[75]  Manish K. Mishra,et al.  Integrating systems engineering practice with health-care delivery , 2014 .

[76]  OHRP and standard-of-care research. , 2014, The New England journal of medicine.

[77]  Published Online Biomedical research: increasing value, reducing waste , 2014 .

[78]  D. Dreyfuss,et al.  Reply: continuous infusion of amphotericin B deoxycholate for the treatment of life-threatening Candida infections. , 2013, American journal of respiratory and critical care medicine.

[79]  M. Chassin Improving the quality of health care: what's taking so long? , 2013, Health affairs.

[80]  M. Chassin,et al.  High-Reliability Health Care: Getting There from Here , 2013, The Milbank quarterly.

[81]  Pat Croskerry,et al.  From mindless to mindful practice--cognitive bias and clinical decision making. , 2013, The New England journal of medicine.

[82]  V. Stodden,et al.  Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals , 2013, PloS one.

[83]  R. Saunders,et al.  Best Care at Lower Cost: The Path to Continuously Learning Health Care in America , 2013 .

[84]  G. Stoddard,et al.  Moderate glucose control is associated with increased mortality compared with tight glucose control in critically ill patients without diabetes. , 2013, Chest.

[85]  D. Dreyfuss,et al.  Amphotericin B deoxycholate for candidiasis in intensive care unit patients revisited: medical, ethical, and financial implications. , 2013, American journal of respiratory and critical care medicine.

[86]  P. Yong,et al.  The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary , 2013 .

[87]  S. Cantor,et al.  Decision aids: when 'nudging' patients to make a particular choice is more ethical than balanced, nondirective content. , 2013, Health affairs.

[88]  Steven N Goodman,et al.  An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. , 2013, The Hastings Center report.

[89]  K. Chung,et al.  Methodologically Rigorous Clinical Research , 2012, Plastic and reconstructive surgery.

[90]  Steen Andreassen,et al.  Decision support for optimized blood glucose control and nutrition in a neurotrauma intensive care unit: preliminary results of clinical advice and prediction accuracy of the Glucosafe system , 2012, Journal of Clinical Monitoring and Computing.

[91]  Alan H. Morris,et al.  The evolution of eProtocols that enable reproducible clinical research and care methods , 2012, Journal of Clinical Monitoring and Computing.

[92]  Gerd Gigerenzer,et al.  Heuristic decision making in medicine , 2012, Dialogues in clinical neuroscience.

[93]  T. Hoff Deskilling and adaptation among primary care physicians using two work innovations , 2011, Health care management review.

[94]  G. Rubenfeld,et al.  Clinical protocols and trainee knowledge about mechanical ventilation. , 2011, JAMA.

[95]  M. Thirunavukarasu Closing the treatment gap , 2011, Indian journal of psychiatry.

[96]  B. McKinley,et al.  Computer protocol facilitates evidence-based care of sepsis in the surgical intensive care unit. , 2011, The Journal of trauma.

[97]  Patricia Malone Switch: How to Change Things When Change Is Hard , 2011 .

[98]  J. Wennberg,et al.  Time to tackle unwarranted variations in practice , 2011, BMJ : British Medical Journal.

[99]  B. Jasny,et al.  Data replication & reproducibility. Again, and again, and again .... Introduction. , 2011, Science.

[100]  A. Fraser,et al.  On the impossibility of being expert , 2010, BMJ : British Medical Journal.

[101]  Steen Andreassen,et al.  Prospective evaluation of a decision support system for setting inspired oxygen in intensive care patients. , 2010, Journal of critical care.

[102]  Ian J. Deary,et al.  What Intelligence Tests Miss: The Psychology of Rational Thought , 2010 .

[103]  T. Browne Pseudodoxia Epidemica: Or, Enquiries Into Very Many Received Tenents, and Commonly Presumed Truths , 2010 .

[104]  Converting venous acid–base and oxygen status to arterial in patients with lung disease , 2009, European Respiratory Journal.

[105]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[106]  Melanie Mitchell,et al.  Complexity - A Guided Tour , 2009 .

[107]  Clayton M. Christensen,et al.  The Innovator's Prescription: A Disruptive Solution for Health Care , 2008 .

[108]  M. Loeffler,et al.  Practice and perception—A nationwide survey of therapy habits in sepsis* , 2008, Critical care medicine.

[109]  H. Warner,et al.  An Electronic Protocol for Translation of Research Results to Clinical Practice: A Preliminary Report , 2008, Journal of diabetes science and technology.

[110]  C. O'connor,et al.  Heart Failure Care in the Outpatient Cardiology Practice Setting: Findings From IMPROVE HF , 2008, Circulation. Heart failure.

[111]  D. Sorenson,et al.  A frame-based representation for a bedside ventilator weaning protocol , 2008, J. Biomed. Informatics.

[112]  Eliotte Hirshberg,et al.  Blood glucose control in critically ill adults and children: a survey on stated practice. , 2008, Chest.

[113]  Gordon R Bernard,et al.  A replicable method for blood glucose control in critically Ill patients , 2008, Critical care medicine.

[114]  Alan H. Morris,et al.  Multicenter Validation of a Computer-Based Clinical Decision Support Tool for Glucose Control in Adult and Pediatric Intensive Care Units , 2008, Journal of diabetes science and technology.

[115]  M. Huehn Paradigms in Management , 2008 .

[116]  Charlene R. Weir,et al.  Clinicians' perceptions about use of computerized protocols: A multicenter study , 2008, Int. J. Medical Informatics.

[117]  Malgorzata E. Wilinska,et al.  Intensive insulin therapy: enhanced Model Predictive Control algorithm versus standard care , 2008, Intensive Care Medicine.

[118]  Elizabeth A McGlynn,et al.  The quality of ambulatory care delivered to children in the United States. , 2007, The New England journal of medicine.

[119]  D. Asch,et al.  Harnessing the power of default options to improve health care. , 2007, The New England journal of medicine.

[120]  Frank Dobbin,et al.  The Global Diffusion of Public Policies: Social Construction, Coercion, Competition, or Learning? , 2007 .

[121]  K. Emmons,et al.  How can we increase translation of research into practice? Types of evidence needed. , 2007, Annual review of public health.

[122]  C. McDonald,et al.  Effects of computerized guidelines for managing heart disease in primary care , 2003, Journal of General Internal Medicine.

[123]  Bram W. Smith,et al.  Using physiological models and decision theory for selecting appropriate ventilator settings , 2006, Journal of Clinical Monitoring and Computing.

[124]  M. Davidson,et al.  Effective diabetes care by a registered nurse following treatment algorithms in a minority population. , 2006, The American journal of managed care.

[125]  Jaap E Tulleken,et al.  Towards a feasible algorithm for tight glycaemic control in critically ill patients: a systematic review of the literature , 2006, Critical care.

[126]  A. H. Wilcock,et al.  A Systems Approach to Biomedicine , 1970, Medical and biological engineering.

[127]  Steen Andreassen,et al.  A method for calculation of arterial acid-base and blood gas status from measurements in the peripheral venous blood , 2006, Comput. Methods Programs Biomed..

[128]  C. Palmer Cost of Poor Quality or Waste in Integrated Delivery System Settings , 2006 .

[129]  Louis C. Sheppard,et al.  Computer control of the infusion of vasoactive drugs , 2006, Annals of Biomedical Engineering.

[130]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[131]  Katrina M. Krause,et al.  Is There Time for Management of Patients With Chronic Diseases in Primary Care? , 2005, The Annals of Family Medicine.

[132]  J Marc Overhage,et al.  Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. , 2005, Health services research.

[133]  G. Robert,et al.  Diffusion of innovations in service organizations: systematic review and recommendations. , 2004, The Milbank quarterly.

[134]  Alan Shiell,et al.  Complex interventions: how “out of control” can a randomised controlled trial be? , 2004, BMJ : British Medical Journal.

[135]  J. Marini,et al.  Ventilatory management of acute respiratory distress syndrome: A consensus of two , 2004, Critical care medicine.

[136]  A. Knight Principles of critical care , 1988, Intensive Care Medicine.

[137]  P. Basch Quality of health care delivered to adults in the United States. , 2003, The New England journal of medicine.

[138]  Karen J. Chan,et al.  Pulmonary function and health-related quality of life in survivors of acute respiratory distress syndrome. , 2003, American journal of respiratory and critical care medicine.

[139]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[140]  Christopher Martyn,et al.  Complexity and Healthcare: An Introduction , 2003, BMJ : British Medical Journal.

[141]  Kensaku Kawamoto,et al.  Clinical Decision Support Provided within Physician Order Entry Systems: A Systematic Review of Features Effective for Changing Clinician Behavior , 2003, AMIA.

[142]  D. Cook,et al.  The design of randomized clinical trials in critically ill patients. , 2002, Chest.

[143]  B C James,et al.  Making it easy to do it right. , 2001, The New England journal of medicine.

[144]  T D East,et al.  Computerized decision support for mechanical ventilation of trauma induced ARDS: results of a randomized clinical trial. , 2001, The Journal of trauma.

[145]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[146]  A D Kaluzny,et al.  Assessing the Implementation of Clinical Process Innovations: A Cross‐case Comparison , 2000, Journal of healthcare management / American College of Healthcare Executives.

[147]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[148]  Alan Morris Developing and Implementing Computerized Protocols for Standardization of Clinical Decisions , 2000, Annals of Internal Medicine.

[149]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[150]  M. Cabana,et al.  Why don't physicians follow clinical practice guidelines? A framework for improvement. , 1999, JAMA.

[151]  J. Kruger Lake Wobegon be gone! The "below-average effect" and the egocentric nature of comparative ability judgments. , 1999, Journal of personality and social psychology.

[152]  Alan H. Morris,et al.  Efficacy of computerized decision support for mechanical ventilation: results of a prospective multi-center randomized trial , 1999, AMIA.

[153]  G. Wells,et al.  A Canadian survey of transfusion practices in critically ill patients. Transfusion Requirements in Critical Care Investigators and the Canadian Critical Care Trials Group. , 1998, Critical care medicine.

[154]  Tor Nørretranders,et al.  The User Illusion: Cutting Consciousness Down to Size , 1998 .

[155]  Chris Sauer,et al.  Research Paper: Information Technology in Complex Health Services: Organizational Impediments to Successful Technology Transfer and Diffusion , 1997, J. Am. Medical Informatics Assoc..

[156]  A Jackson-Smale,et al.  Protocols for clinical care. , 1995, Computer methods and programs in biomedicine.

[157]  Bill Bell,et al.  Help-System Development Tools , 1995 .

[158]  R A Cowley,et al.  Randomized clinical trial of pressure-controlled inverse ration ventilation and extra corporeal CO2 removal for ARDS. , 1995, American journal of respiratory and critical care medicine.

[159]  T P Clemmer,et al.  A successful computerized protocol for clinical management of pressure control inverse ratio ventilation in ARDS patients. , 1992, Chest.

[160]  M. Singer,et al.  Critical Care Algorithms , 1992 .

[161]  Dean F. Sittig,et al.  A strategy for development of computerized critical care decision support systems , 1991, International journal of clinical monitoring and computing.

[162]  J. Reggia,et al.  HELP: A Dynamic Hospital Information System , 1991, Computers and Medicine.

[163]  B Kleinmuntz,et al.  Why we still use our heads instead of formulas: toward an integrative approach. , 1990, Psychological bulletin.

[164]  Bruce I. Blum,et al.  Implementing Health Care Information Systems , 1989, Computers and Medicine.

[165]  Dean F. Sittig,et al.  Implementation of a computerized patient advice system using the HELP clinical information system. , 1989, Computers and biomedical research, an international journal.

[166]  R. Dawes,et al.  Heuristics and Biases: Clinical versus Actuarial Judgment , 2002 .

[167]  J W Williamson,et al.  Health science information management and continuing education of physicians. A survey of U.S. primary care practitioners and their opinion leaders. , 1989, Annals of internal medicine.

[168]  G A Diamond,et al.  Clinician decisions and computers. , 1987, Journal of the American College of Cardiology.

[169]  H. Sox,et al.  Clinical prediction rules. Applications and methodological standards. , 1985, The New England journal of medicine.

[170]  C Z Margolis,et al.  Uses of clinical algorithms. , 1983, JAMA.

[171]  W. Edwards Deming,et al.  Out of the Crisis , 1982 .

[172]  R. Haack Education and the Good Life , 1981, Philosophy.

[173]  H. Arkes,et al.  Impediments to accurate clinical judgment and possible ways to minimize their impact. , 1981, Journal of consulting and clinical psychology.

[174]  L. Bertalanffy A Systems View of Man , 1981 .

[175]  R S Newbower,et al.  Preventable anesthesia mishaps: a study of human factors* , 1978, Anesthesiology.

[176]  B. Fischhoff,et al.  Knowing with Certainty: The Appropriateness of Extreme Confidence. , 1977 .

[177]  J. Hoyt,et al.  Algorithm for management of pulmonary complications in burn patients , 1977, Critical care medicine.

[178]  N. Kouchoukos,et al.  Automation of measurements and interventions in the systematic care of postoperative cardiac surgical patients. , 1977, Medical instrumentation.

[179]  W. Shoemaker A patient care algorithm for cardiac arrest , 1976, Critical care medicine.

[180]  W B Schwartz,et al.  Medicine and the computer. The promise and problems of change. , 1970, The New England journal of medicine.

[181]  Lotfi A. Zadeh,et al.  General System Theory , 1962 .

[182]  Sarah Williams,et al.  Computer applications , 1988 .

[183]  H J ATKINS,et al.  Three Pillars of Clinical Research , 1959, British medical journal.

[184]  E. L. Kelly Clinical versus statistical prediction: A theoretical analysis and review of the evidence. , 1955 .

[185]  Reginald G. Bickford,et al.  Automatic electroencephalographic control of general anesthesia , 1950 .

[186]  A. Whitehead An Introduction to Mathematics , 1949, Nature.

[187]  W. Weaver Science and complexity. , 1948, American scientist.

[188]  Berard Vogt The opus Majus of Roger Bacon , 1929 .