Senseval and SemEval datasets have come to be the de facto gold standards for much of the work in computational semantics. The events that spawned these datasets have enabled comparison of systems, fostered a large community of researchers and spawned interest in a wide variety of semantic tasks. In my talk, I will discuss the origins of these events in the evaluation of word sense disambiguation systems and I will describe a sample of the tasks that have arisen as SemEval has evolved. These `evaluation exercises' have brought many benefits to the field in terms of algorithms and data as well as new players and tasks. Nevertheless, various issues have arisen from the very inception of Senseval and need consideration when interpreting results and designing tasks. One issue that I will particularly discuss is representation of semantic phenomena and the pros and cons of tasks that are representation independent.
Dr. Diana McCarthy is an affiliated lecturer at the University of Cambridge, UK. She has been active in the field of computational linguistics for over 20 years and specialises in the field of computational lexical semantics. Along with various collaborators, she won best paper awards for her work on automatic detection of sense predominance and for compositionality modelling of compound nouns with distributional semantics. Prior to her affiliation at Cambridge, she held a Dorothy Hodgkin Research Fellowship from the UK Royal Society at the University of Sussex from 2005 until 2009. From 2009 until 2012 she worked as a director and computational linguist for the research-led company 'Lexical Computing Ltd' whose main product is the Sketch Engine. Her work includes evaluation of computational models of lexical semantics and she has been involved in the Senseval (now SemEval) community since the first event in 1998 as both participant and task organiser.
The Alan Turing Institute
1st floor of the British Library, 96 Euston Road, London, NW1 2DB