CALLISTO is a text summarizer that searches through a space of possible configurations for the best one. This is different from other systems since it allows CALLISTO 1) to choose adequate components based on results obtained on the training data (and thus, to choose a configuration better adapted to the problem) and 2) to allow different texts to be summarized in different ways. The purpose of this talk is to show how the initial space CALLISTO explores can be modified to improve the overall quality of the summaries produced.

We will explain how we evaluated the first arbitrary design choices made in the system, through a fully automated framework based on a content measure proposed by Lin and Hovy. In particular, we will present experiments with learning algorithms and evaluation methods different from the original ones (C5.0 and Key Phrases in Abstract). We have shown that using Naive Bayes with a weighted unigram and bigram F-Score significantly improves the quality of our summaries.