From ZULU to RERS - Lessons Learned in the ZULU Challenge

This paper summarizes our experience with the ZULU challenge on active learning without equivalence queries, presents our winning solution, investigates the character of ZULU's rating approach, and discusses how this approach can be taken further to establish a framework for the systematic investigation of domain-specific, scalable learning solutions for practically relevant application. In particular, it discusses the RERS initiative, which provides a community platform together with a learning framework that allows users to interactively compose complex learning solutions on the basis of libraries for various learning components, system connectors, and other auxiliary functionality. This framework will be the backbone for an extended challenge on learning in 2011.

[1]  Roland Groz,et al.  Inferring Mealy Machines , 2009, FM.

[2]  Mihalis Yannakakis,et al.  Black Box Checking , 1999, FORTE.

[3]  Hongyang Qu,et al.  Assume-Guarantee Verification for Probabilistic Systems , 2010, TACAS.

[4]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[5]  T. Kanade.  Model-Based Testing of Reactive Systems , 2005 .

[6]  Frits W. Vaandrager,et al.  Inference and Abstraction of the Biometric Passport , 2010, ISoLA.

[7]  Bengt Jonsson,et al.  Inferring Compact Models of Communication Protocol Entities , 2010, ISoLA.

[8]  Tiziana Margaria,et al.  Efficient test-based model generation for legacy reactive systems , 2004, Proceedings. Ninth IEEE International High-Level Design Validation and Test Workshop (IEEE Cat. No.04EX940).

[9]  Amir Pnueli,et al.  On the learnability of infinitary regular sets , 1991, COLT '91.

[10]  Benedikt Bollig,et al.  Smyle: A Tool for Synthesizing Distributed Models from Scenarios by Learning , 2008, CONCUR.

[11]  Hardi Hungar,et al.  Domain-Specific Optimization in Automata Learning , 2003, CAV.

[12]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[13]  Ana Cavalcanti,et al.  FM 2009: Formal Methods, Second World Congress, Eindhoven, The Netherlands, November 2-6, 2009. Proceedings , 2009, FM.

[14]  Keqin Li,et al.  Learning Parameterized State Machine Model for Integration Testing , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[15]  Bengt Jonsson,et al.  On the Correspondence Between Conformance Testing and Regular Inference , 2005, FASE.

[16]  Colin de la Higuera,et al.  Zulu: An Interactive Learning Competition , 2009, FSMNLP.

[17]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems: Advanced Lectures (Lecture Notes in Computer Science) , 2005 .

[18]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[19]  Tiziana Margaria,et al.  Dynamic testing via automata learning , 2007, International Journal on Software Tools for Technology Transfer.

[20]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[21]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[22]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[23]  Hardi Hungar,et al.  Model Generation by Moderated Regular Extrapolation , 2002, FASE.

[24]  Howard Barringer,et al.  Learning to divide and conquer: applying the L* algorithm to automate assume-guarantee reasoning , 2008, Formal Methods Syst. Des..

[25]  T. Margaria,et al.  Efficient Regression Testing of CTI-Systems : Testing a complex Call-Center Solution , 2001 .

[26]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.

[27]  Perdita Stevens,et al.  Modelling Recursive Calls with UML State Diagrams , 2003, FASE.

[28]  Tiziana Margaria,et al.  LearnLib: a framework for extrapolating behavioral models , 2009, International Journal on Software Tools for Technology Transfer.

[29]  Marsha Chechik,et al.  CONCUR 2008 - Concurrency Theory, 19th International Conference, CONCUR 2008, Toronto, Canada, August 19-22, 2008. Proceedings , 2008, CONCUR.

[30]  José L. Balcázar,et al.  Algorithms for Learning Finite Automata from Queries: A Unified View , 1997, Advances in Algorithms, Languages, and Complexity.

[31]  Corina S. Pasareanu,et al.  Learning Assumptions for Compositional Verification , 2003, TACAS.

[32]  A. Nerode,et al.  Linear automaton transformations , 1958 .