Diarmuid Ó Séaghdha
MPhil Project Suggestions 2013-14
Using Natural Language Processing to investigate how spoken sentences are processed in the human brain (with Anna Korhonen and Barry Devereux)
The study of language has been a central activity across many disciplines, including psychology, linguistics, computer science, and cognitive neuroscience. However, to date there has been surprisingly little cross-disciplinary research integrating the powerful analytical tools of computational linguistics into the study of how language is processed in the human brain. In this project, you will contribute to the emerging field of computational neurolinguistics, by developing quantitative measures of specific aspects of sentence processing, and then evaluating these measures against state-of-the-art neuroimaging data.
Understanding a spoken sentence has several different processing components, involving different regions of the brain. The incoming speech must be acoustically and phonetically processed, and lexical information for each word must be activated and integrated through syntactic computations to produce a final representation of the utterance. Moreover, because the speech signal unfolds over time, temporary ambiguities arise where multiple candidate syntactic representations are consistent with the currently available input, and these ambiguities must be subsequently resolved for successful understanding. For example, the phrase ,landing planes is locally syntactically ambiguous; it could be a noun phrase in which landing modifies planes (e.g. the full sentence might be landing planes are noisy) or it could be a gerundive clause, where planes is the object of landing (e.g. landing planes is difficult). Lexicalist accounts of sentence processing propose that lexico-syntactic knowledge associated with each word guides activation of candidate parses and is therefore influential in the ambiguity resolution process (Tyler and Marslen-Wilson, 1977; Marslen-Wilson et al., 1988; MacDonald et al., 1994). Such proposals are also supported by recent neuroimaging evidence (e.g. Shetreet et al., 2007; Tyler et al., 2013).
One important kind of lexico-syntactic knowledge is knowledge of selectional preference, the phenomenon by which verbs and other linguitic predicates are more likely to take certain semantic classes as arguments than others. For example, the direct object of drink is more likely to be a liquid than a human, but the subject of drink is more likely to be a human than a beverage. In this project, you will investigate how models of verb selectional preferences can be used to make predictions about the parsing preferences people that have when they hear locally ambiguous phrases. A variety of such models have been proposed in the NLP literature (e.g., Ó Séaghdha 2010, Ó Séaghdha and Korhonen, 2012) and the project will involve evaluating a representative selection. Does knowledge of selectional preferences create expectations regarding how phrases such as "landing planes" get disambiguated, and can we determine the neural correlates of such expectations in the brain? The explanatory power of the model you develop will be evaluated against high-temporal-resolution neuroimaging data acquired as human subjects listened to spoken sentences (Tyler et al 2013).
- Anna Korhonen, Yuval Krymolowski and Ted Briscoe. 2006. A large subcategorization lexicon for Natural Language Processing applications. In Proceedings of LREC-06. Genova, Italy.
- Maryellen C. MacDonald, Neal J. Pearlmutter and Mark S. Seidenberg. 1994. The lexical nature of syntactic ambiguity resolution. Psychological Review 101(4):676-703.
- William Marslen-Wilson, Colin M. Brown and Lorraine K Tyler. 1988. Lexical representations in spoken language comprehension. Language and Cognitive Processes 3(1): 1-16.
- Diana McCarthy. 2001. Lexical Acquisition at the Syntax-Semantics Interface: Diathesis Alternations, Subcategorization Frames and Selectional Preferences. DPhil Thesis, University of Sussex
- Diarmuid Ó Séaghdha. 2010. Latent variable models of selectional preference. In Proceedings of ACL-10. Uppsala, Sweden.
- Diarmuid Ó Séaghdha and Anna Korhonen. 2012. Modelling selectional preferences in a lexical hierarchy. In Proceedings of *SEM-12. Montreal, Canada.
- Einat Shetreet, Dafna Palti, Naama Friedmann and Uri Hadar. 2007. Cortical representation of verb processing in sentence comprehension: Number of complements, subcategorization, and thematic frames. Cerebral Cortex 17(8):1958-1969.
- Lorraine K. Tyler, Teresa P. L. Cheung, Barry J. Devereux and Alex Clarke. 2013. Syntactic computations in the language network: Characterizing dynamic network properties using representational similarity analysis. Frontiers in Language Sciences 4:271.
- Lorraine K. Tyler and William D. Marslen-Wilson. 1977. The on-line effects of semantic context on syntactic processing. Journal of Verbal Learning and Verbal Behavior 16(6):683-692
Identifying Causes and Effects
Reasoning about the causal links between events is a fundamental task in general artificial intelligence and in natural language understanding. As part of the SemEval 2012 exercise in semantic evaluation, Gordon et al. (2012) presented the Choice of Plausible Alternatives (COPA) shared task in which systems must select either the most likely of two possible outcomes caused by a trigger event or the most likely to two possible triggers leading to an outcome event. An example item is the following:
- Premise
- The man lost his balance on the ladder. What happened as a result?
- Alternative 1
- He fell off the ladder.
- Alternative 2
- He climbed up the ladder.
Only one system participated in the SemEval competition (Goodwin et al., 2012) and it has not received much attention since, so there is plenty left to do with this dataset! The project would involve comparing a number of methods in a controlled fashion, potentially including event chains in the manner of Chambers and Jurafsky (2008), insights from the psychology of causal reasoning (Griffiths and Tenenbaum, 2005) and supervised models of causality trained on data annotated for another SemEval shared task by Hendrickx et al. (2010).
- Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised Learning of Narrative Event Chains. In Proceedings of ACL 2008.
- Travis Goodwin, Bryan Rink, Kirk Roberts, Sanda Harabagiu. 2012. UTDHLT: COPACETIC System for Choosing Plausible Alternatives. In Proceedings of SemEval 2012.
- Andrew S. Gordon, Zornitsa Kozareva and Melissa Roemmle. 2012. SemEval-2012 Task 7: Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. In Proceedings of SemEval 2012.
- Thomas L. Griffiths and Joshua B. Tenenbaum. 2005. Structure and strength in causal induction. Cognitive Psychology 51(4):334-384.
- Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano and Stan Szpakowicz. 2010. SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations between Pairs of Nominals. In Proceedings of SemEval 2010.
Tracking Word Meaning Through Time
Modelling word meaning is a fundamental task in NLP and as such has received a lot of attention from researchers. The distributional approach to lexical semantics gives us a powerful way to model the meanings of words by observing their patterns of co-occurrence in text corpora (Turney and Pantel, 2010); in particular, probabilistic latent-variable models have been shown to produce robust and informative representations (Ó Séaghdha, 2010). It would also be interesting to track how a word changes its meaning over time; for example, the rise of computing and the Internet has given us new senses for many words including mouse, surf and tweet. Previous work has used corpora to track changes in topic prominence, lexical frequency and syntactic preferences over time (Blei and Lafferty, 2006; Hall et al, 2008; Michel et al, 2010, Danescu-Niculescu-Mizil et al, 2013), and machine learning researchers have developed models that learn diachronic variations of topic structure (Blei and Lafferty, 2006; Wang and McCallum, 2006). Applying insights and methods from this body of research to problems in lexical semantics has great potential to help us detect and analyse how and where words change their meanings. There has been some previous work in this area (e.g., Sagi et al, 2009) but it remains underexplored.
The goal of this project is to investigate methods for modelling how the distributional aspects of a word's meaning change over time in a chronologically partitioned corpus. Depending on the student's interests, this corpus may come from digitised books, scientific articles or social media. Evaluation experiments will involve quantitative and qualitative evaluations, potentially including user studies by domain experts.
This project would be a good fit for students interested in lexical semantics, lexicography, computational sociolinguistics, machine learning and/or digital humanities.
- David Blei and John Lafferty. 2006. Dynamic topic models. In Proceedings of ICML 2006.
- Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec and Christopher Potts. 2013. No country for old members: User lifecycle and linguistic change in online communities. In Proceedings of WWW-13.
- David Hall, Daniel Jurafsky and Christopher D. Manning. 2008. Studying the History of Ideas Using Topic Models. In Proceedings of EMNLP 2008.
- Jean-Baptiste Michel, et al. 2010. Quantitative analysis of culture Using millions of digitized books. Science 331(6014): 176--182.
- Diarmuid Ó Séaghdha. 2010. Latent variable models of selectional preference. In Proceedings of ACL 2010.
- Eyal Sagi, Stefan Kaufmann and Brady Clark. 2009. Semantic Density Analysis: Comparing Word Meaning across Time and Phonetic Space. In Proceedings of GEMS 2009.
- Peter Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research 37: 141-188.
- Xuerui Wang and Andrew McCallum. 2006. Topics over Time: A non-Markov continuous time model of topical trends. In Proceedings of KDD 2006.