Course pages 2013–14
Statistical Machine Translation
Principal lecturers: Dr Stephen Clark, Prof Bill Byrne
Taken by: MPhil ACS, Part III
Hours: 16 (10 lectures + extended practical session covering 6 sessions)
Prerequisites: L100 Introduction to Natural Language Processing
This module provides an in-depth introduction to statistical machine translation (MT), the dominant approach to providing large-scale, robust translation applicable to many language pairs (and the approach currently used by Google).
- Overview: [2 lectures]: Translation as an economic, political, and cultural activity. Machine translation as a problem in natural language processing. Syntax and morphology in translation. Translation memories; example and rule-based based MT. Interlingua.
- Alignment: automatic translations in text [2 lectures]: Parallel texts and their role in building translation systems and measuring translation quality. Document and sentence alignment: models and algorithms. Word and phrase alignment: models and algorithms. Techniques for automatic measurement of alignment quality. Webcrawling for parallel text.
- Weighted finite state transducers: algorithms for natural language processing and MT [2 lectures]
- SMT systems [4 lectures]: Extraction of translation rules from parallel text. Phrase-based, Hiero, syntax-based MT. Techniques for automatic measurement of translation quality. Minimum error rate training. Language models for SMT: simple back-off, MapReduce. MT system combination. Practical issues in SMT: true casing; source text pre-processing; handling morphology; system building procedure.
All lectures will be given by Dr Clark or Dr Byrne.
On completion of this module, students should understand:
- the role of parallel text in MT;
- how alignment models can be estimated from parallel text;
- how alignment models capture divergent language properties such as word order;
- the use of WFSTs in translation and some other basic NLP tasks;
- the extraction of translation rules from parallel text;
- various phrase-based translation architectures, including Hiero;
- parameter optimization procedures for SMT;
- the role of language models in SMT;
- the evaluation of SMT systems using automatic metrics;
- system combination techniques for SMT.
There will be two substantial practical exercises associated with this module.
- Practical 1: 2 sessions. Parallel text, alignment models and WFSTs.
- Practical 2: 4 sessions. SMT system construction and evaluation.
- Written report covering the practical worth 35% of the marks.
- One final take-home test covering all the material. Final take-home test will contribute 65% to the final mark. Questions set and marked by Dr Clark and Dr Byrne.
Jurafsky, D. & Martin, J. (2008). Speech and language
processing. Prentice Hall (2nd ed.). Chapter 25 on Machine Translation.
Brown, P.F., Della Pietra, V.J. & Della Pietra, S.A. (1993). The mathematics of statistical machine translation: parameter estimation. Computational linguistics.
Chiang, D. (2007). Hierarchical phrase-based translation. Computational linguistics, 33(2):201–228.