next up previous
Next: Availability Up: a Previous: Look Ahead

Comparison with plex

The only other scanner generator which produces output in Prolog that we are aware of is plex by Peter Reintjes. An important advantage of Elex over plex is the fact that Elex is multilingual. Arguments in favor of multilingual scanner generators have been given in the introduction.

A further striking difference with plex is the fact that plex not only produces Prolog code, but is implemented in Prolog as well. The installation requires Quintus Prolog. In contrast, Elex requires Perl5 and C++ (both of which are available for free).2 Related to this may be the observation that scanner generation in plex is rather slow; the documentation mentions scanner generation times of several minutes. In contrast, Elex requires only a few seconds for all examples we tried (including some scripts we use for practical programming work).

The plex documentation is not entirely clear about how conflicts in the scanner are solved, and how non-determinism is treated. It seems that plex implements in a deterministic way a look-ahead of a single token. In this sense, the resulting Prolog code might be more efficient (and more complicated) then our simple implementation of look-ahead given above, but less flexible (it cannot treat cases which require a larger look-ahead). A newer version of Elex in which non-determinism is treated in the automaton construction part will be both efficient, simple and flexible.

The code produced by plex uses a similar technique for indexing as we implemented for Elex(Prolog). A compact version, as we provide too, is mentioned in [6] as an interesting alternative which is not part of plex.

Two further arguments are given in favor of plex in [6]: the possibility to call other tokenizers recursively, and the possibility to construct tokenizers on the fly. The latter possibility is of course not provided in Elex(Prolog) simply because the language in which Elex is implemented is different from the output language. The first possibility is available in Elex(Prolog) too by using the module prefixes.


next up previous
Next: Availability Up: a Previous: Look Ahead
Noord G.J.M. van
1998-09-25