From semi-syntactic lexical analyzer to a new compiler model

This paper introduces the "semi-syntactic lexical analyzer", a new concept of lexical analyzer which is capable of recognizing program fragments. The significant results of such an analyzer are that some syntactic analysis can be omitted, better intermediate or object codes can be generated and a new compiler model is thus derived. The main difference among currently used models and our new model is that currently used models separate input symbol stream into tokens, while our new model separates them into "meaningful units" (simple expressions or even part of an expression) if possible. We found that separating them into meaningful units is much easier for further analysis, translation and optimization.