Papers for Relevance Assessment by Enrique Amigó

Research Question: Automatic evaluation of summaries and automatic metaevaluation of metrics. How to combine and meta-evaluate similarity metrics to measure the proximity from an automatic summary to a set of models.

Paper ID
(Link to PDF)

Title

Author(s)

C02-1073

Meta-evaluation of Summaries in a Cross-lingual Environment using Content-based Metrics

Horacio Saggion; Dragomir Radev; Simone Teufel; Wai Lam

W05-0907

Evaluating DUC 2004 Tasks with the QARLA Framework

Enrique Amigo; Julio Gonzalo; Anselmo Penas; Felisa Verdejo

W05-0905

Evaluating Automatic Summaries of Meeting Recordings

Gabriel Murray; Steve Renals; Jean Carletta; Johanna Moore

2

CESTA: Machine Translation Evaluation Campaign [Work-in-Progress Project Report]

Widad Mustafa El Hadi, Marianne Dabbadie, Ismaïl Timimi, Martin Rajman, Philippe Langlais, Antony Hartley and Andrei Popescu Belis

W02-0406

Manual and automatic evaluation of summaries

Chin-Yew Lin; Eduard Hovy

I05-2014

BLEU in Characters: Towards Automatic MT Evaluation in Languages without Word Delimiters

Etienne Denoual; Yves Lepage

W05-0909

METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments

Satanjeev Banerjee; Alon Lavie

W00-1401

Evaluation Metrics for Generation

Srinivas Bangalore; Owen Rambow; Steve Whittaker

Santos

Automatic Evaluation of Summaries Using Document Graphs

Eugen Santos Jr., Ahmed A. Mohamed and Qunhua Zhao

H05-2007

Pattern Visualization for Machine Translation Output

Adam Lopez; Philip Resnik

215_pdf_2-col

Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics

Chin-Yew Lin and Franz Josef Och

194_Paper

Using N-Grams To Understand the Nature of Summaries

Michele Banko and Lucy Vanderwende

H05-1093

BLANC: Learning Evaluation Metrics for MT

Lucian Lita; Monica Rogati; Alon Lavie

Lin

ROUGE: A Package for Automatic Evaluation of Summaries

Chin-Yew Lin

W05-0833

Hybrid Example-Based SMT: the Best of Both Worlds?

Declan Groves; Andy Way