Incremental analysis of real programming languages

A major research goal for compilers and environments is the automatic derivation of tools from formal specifications. However, the formal model of the language is often inadequate; in particular, LR(k) grammars are unable to describe the natural syntax of many languages, such as C++ and Fortran, which are inherently non-deterministic. Designers of batch compilers work around such limitations by combining generated components with ad hoc techniques (for instance, performing partial type and scope analysis in tandem with parsing). Unfortunately, the complexity of incremental systems precludes the use of batch solutions. The inability to generate incremental tools for important languages inhibits the widespread use of language-rich interactive environments.We address this problem by extending the language model itself, introducing a program representation based on parse dags that is suitable for both batch and incremental analysis. Ambiguities unresolved by one stage are retained in this representation until further stages can complete the analysis, even if the reaolution depends on further actions by the user. Representing ambiguity explicitly increases the number and variety of languages that can be analyzed incrementally using existing methods.To create this representation, we have developed an efficient incremental parser for general context-free grammars. Our algorithm combines Tomita's generalized LR parser with reuse of entire subtrees via state-matching. Disambiguation can occur statically, during or after parsing, or during semantic analysis (using existing incremental techniques); program errors that preclude disambiguation retsin multiple interpretations indefinitely. Our representation and analyses gain efficiency by exploiting the local nature of ambiguities: for the SPEC95 C programs, the explicit representation of ambiguity requires only 0.5% additional space and less than 1% additional time during reconstruction.

[1]  Luigi Petrone.  Reusing Batch Parsers as Incremental Parsers , 1995, FSTTCS.

[2]  Mark Johnson.  The Computational Complexity of GLR Parsing , 1991 .

[3]  Wilf R. LaLonde,et al.  Regular right part grammars and their parsers , 1977, CACM.

[4]  Bjarne Stroustrup,et al.  The Annotated C++ Reference Manual , 1990 .

[5]  Paul Klint,et al.  The syntax definition formalism SDF—reference manual— , 1989, SIGP.

[6]  Jean H. Gallier,et al.  Building friendly parsers , 1982, POPL '82.

[7]  M. Just,et al.  Working memory constraints on the processing of syntactic ambiguity , 1992, Cognitive Psychology.

[8]  Masaru Tomita,et al.  Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems , 1985 .

[9]  Masaru Tomita,et al.  Efficient parsing for natural language , 1985 .

[10]  Neil D. Jones,et al.  Attribute-Influenced LR Parsing , 1980 .

[11]  David A. Watt,et al.  Rule splitting and attribute-directed parsing , 1980, Semantics-Directed Compiler Generation.

[12]  Marc Lankhorst,et al.  An imperial comparison of generalized LR tables , 1991 .

[13]  Ikuo Nakata,et al.  Rie, a compiler generator based on a one‐pass‐type attribute grammar , 1995, Softw. Pract. Exp..

[14]  Susan L. Graham,et al.  Efficient self-versioning documents , 1997, Proceedings IEEE COMPCON 97. Digest of Papers.

[15]  J. Rekers,et al.  Parser Generation for Interactive Environments , 1992 .

[16]  Paul Hudak,et al.  A gentle introduction to Haskell , 1992, SIGP.

[17]  William Harry Maddox,et al.  Incremental Static Semantic Analysis , 1998 .

[18]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.

[19]  Alfred V. Aho,et al.  Deterministic parsing of ambiguous grammars , 1973, POPL.

[20]  Jean-Marie Larchevêque.  Optimal incremental parsing , 1995, TOPL.

[21]  Manuel Vilares Ferro,et al.  Efficient incremental parsing for context-free languages , 1994, Proceedings of 1994 IEEE International Conference on Computer Languages (ICCL'94).

[22]  Tim A. Wagner,et al.  Efficient and flexible incremental parsing , 1998, TOPL.

[23]  Robert Giegerich.  Considerate Code Selection , 1991, Code Generation.

[24]  Eelco Visser,et al.  A Case Study in Optimizing Parsing Schemata by Disambiguation Filters , 1997, IWPT.

[25]  Eelco Visser,et al.  Scannerless Generalized-LR Parsing , 1997 .

[26]  Eelco Visser,et al.  Using Filters for the Disambiguation of Context-free Grammars , 1994 .

[27]  Rahman Nozohoor-Farshi.  GLR Parsing for ε-Grammers , 1991 .

[28]  M. Just,et al.  Working Memory Constraints on the Resolution of Lexical Ambiguity: Maintaining Multiple Interpretations in Neutral Contexts , 1994 .