This project aims at extending multi-task learning frameworks for sequence labeling [6, 7] in order to model the relationships between tasks. Most current multi-task learning approaches learn shared representations at the bottom layers of a network, with task-specific classifiers at the top layers [1]. However, such approaches may be vulnerable to under-transfer or even negative transfer as they do not explicitly model the similarities / dissimilarities between tasks. In this project, we will focus on explicit modeling of task relationships [1, 2, 3, 4] for sequence labeling and extend our neural sequence labeling framework [6] through the investigation of matrix-variate (normal) priors [2] and (sparse, non-negative) regularisers [5] to guide learning, among others.
[1] Zhang Y, Yang Q: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 2017.
[2] Zhang Y, Yeung DY: A regularization approach to learning task relationships in multitask learning. ACM Transactions on Knowledge Discovery from Data (TKDD) 2014, 8 (3):12.
[3] Zhang Y, Yeung DY: Learning high-order task relationships in multi-task learning. In Twenty-Third International Joint Conference on Artificial Intelligence 2013.
[4] Goncalves AR, Von Zuben FJ, Banerjee A: Multi-task sparse structure learning with gaussian copula models. The Journal of Machine Learning Research 2016, 17:1205–1234.
[5] Lee G, Yang E, Hwang S: Asymmetric multi-task learning based on task relatedness and loss. In International Conference on Machine Learning 2016:230–2.
[6] https://github.com/marekrei/sequence-labeler
[7] Rei M and Yannakoudakis H. 2017. Auxiliary Objectives for Neural Error Detection Models. In Proceedings of the 12th NAACL Workshop on Innovative Use of Natural Language Processing for Building Educational Applications.
In this project, we will focus on automatically generating feedback with respect to discourse coherence in writing produced by non-native learners of English. We will explore a number of different approaches for modelling discourse coherence (see references below) and build a tool that visualises different aspects of discourse (e.g., referential coherence, relational coherence, misuse of cohesive devices, etc.) with the aim of providing feedback for improving the quality of one's writing.
[1] https://www.aclweb.org/anthology/P01-1014.pdf
[2] https://www.aclweb.org/anthology/W02-0211.pdf
[3] https://www.aclweb.org/anthology/W16-3615.pdf
[4] https://www.aclweb.org/anthology/W19-2720.pdf
[5] https://www.aclweb.org/anthology/C14-1090.pdf
[6] https://www.cis.upenn.edu/~elenimi/lrecpaper.pdf