Assessing the accuracy of legal implementation readiness decisions

Software engineers regularly build systems that are required to comply with laws and regulations. To this end, software engineers must determine which requirements have met or exceeded their legal obligations and which requirements have not. Requirements that have met or exceeded their legal obligations are legally implementation ready, whereas requirements that have not met or exceeded their legal obligations need further refinement. Research is needed to better understand how to support software engineers in making these determinations. In this paper, we describe a case study in which we asked graduate-level software engineering students to assess whether a set of software requirements for an electronic health record system met or exceeded their corresponding legal obligations as expressed in regulations created pursuant to the U.S. Health Insurance Portability and Accountability Act (HIPAA). We compare the assessment made by graduate students with an assessment made by HIPAA compliance subject matter experts. Additionally, we contrast these results with those generated by a legal requirements triage algorithm. Our findings suggest that the average graduate-level software engineering student is ill-prepared to write legally compliant software with any confidence and that domain experts are an absolute necessity. Our findings also indicate the potential utility of legal requirements metrics in aiding software engineers as they make legal compliance decisions.

[1]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[2]  John Mylopoulos,et al.  First International Workshop on Requirements Engineering and Law (RELAW) , 2008 .

[3]  Aaron K. Massey,et al.  Triage for legal requirements , 2010 .

[4]  David L. Olson,et al.  Advanced Data Mining Techniques , 2008 .

[5]  Fabio Massacci,et al.  Using a security requirements engineering methodology in practice: The compliance with the Italian data protection legislation , 2005, Comput. Stand. Interfaces.

[6]  Annie I. Antón,et al.  Evaluating existing security and privacy requirements for legal compliance , 2009, Requirements Engineering.

[7]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[8]  Luo Si,et al.  A statistical model for scientific readability , 2001, CIKM '01.

[9]  Raymond P. L. Buse,et al.  A metric for software readability , 2008, ISSTA '08.

[10]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[11]  Andrew Meneely,et al.  Software metrics validation criteria: A systematic literature review , 2010 .

[12]  Annie I. Antón,et al.  The production rule framework: developing a canonical set of software requirements for compliance with law , 2010, IHI.

[13]  Annie I. Antón,et al.  Prioritizing Legal Requirements , 2009, 2009 Second International Workshop on Requirements Engineering and Law.

[14]  Jane Cleland-Huang,et al.  A machine learning approach for tracing regulatory codes to product specific requirements , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[15]  J. Peter Kincaid,et al.  Derivation and Validation of the Automated Readability Index for Use with Technical Materials , 1970 .

[16]  Helen Nissenbaum,et al.  Privacy and contextual integrity: framework and applications , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).

[17]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[18]  Annie I. Antón,et al.  Addressing Legal Requirements in Requirements Engineering , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[19]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[20]  R. Flesch.  A new readability yardstick. , 1948, The Journal of applied psychology.

[21]  Annie I. Antón,et al.  Developing Production Rule Models to Aid in Acquiring Requirements from Legal Texts , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[22]  Jane Cleland-Huang,et al.  Towards automated requirements prioritization and triage , 2009, Requirements Engineering.

[23]  Geoffrey J. McLachlan,et al.  Analyzing Microarray Gene Expression Data , 2004 .

[24]  R. Gunning.  The Technique of Clear Writing. , 1968 .

[25]  Annie I. Antón,et al.  Analyzing Regulatory Rules for Privacy and Security Requirements , 2008, IEEE Transactions on Software Engineering.

[26]  Insup Lee,et al.  Privacy APIs: access control techniques to analyze and verify legal privacy policies , 2006, 19th IEEE Computer Security Foundations Workshop (CSFW'06).

[27]  Annie I. Antón,et al.  Checking Existing Requirements for Compliance with Law Using a Production Rule Model , 2009, 2009 Second International Workshop on Requirements Engineering and Law.

[28]  Annie I. Antón,et al.  Validating Existing Requirements for Compliance with Law Using a Production Rule Model , 2009 .

[29]  G. Harry McLaughlin,et al.  SMOG Grading - A New Readability Formula. , 1969 .

[30]  Andrew Stellman,et al.  Applied software project management , 2005 .

[31]  V. Basili.  Software modeling and measurement: the Goal/Question/Metric paradigm , 1992 .