Evaluation of the NLP Components of the OVIS2 Spoken
Dialogue System

Gert Veldhuijzen van Zanten¹
Gosse Bouma²
Khalil Sima'an³
Gertjan van Noord²
Remko Bonnema⁴

¹IPO Eindhoven
²Rijksuniversiteit Groningen
³Universiteit Utrecht
⁴Universiteit van Amsterdam

Abstract:

The NWO Priority Programme Language and Speech Technology is a 5-year research programme aiming at the development of spoken language information systems. In the Programme, two alternative natural language processing (NLP) modules are developed in parallel: a grammar-based (conventional, rule-based) module and a data-oriented (memory-based, stochastic, DOP) module. In order to compare the NLP modules, a formal evaluation has been carried out three years after the start of the Programme. This paper describes the evaluation procedure and the evaluation results. The grammar-based component performs much better than the data-oriented one in this comparison.