The Alan Turing Institute

Computational Semantics Evaluation: The Origins of Senseval and Evolution of SemEval

Learn more Register now Add to Calendar 07/16/2018 10:30 AM 07/16/2018 12:00 PM Europe/London Computational Semantics Evaluation: The Origins of Senseval and Evolution of SemEval Location of the event
Monday 16 Jul 2018
Time: 10:30 - 12:00

Research area

Event type




About the event

Senseval and SemEval datasets have come to be the de facto gold standards for much of the work in computational semantics. The events that spawned these datasets have enabled comparison of systems, fostered a large community of researchers and spawned interest in a wide variety of semantic tasks. In my talk, I will discuss the origins of these events in the evaluation of word sense disambiguation systems and  I will describe a sample of the tasks that have arisen as SemEval has evolved.  These `evaluation exercises' have brought many benefits to the field  in terms of algorithms and data as well as new players and  tasks. Nevertheless, various issues  have arisen from the very inception of Senseval and need consideration when interpreting results and designing tasks. One issue that I will particularly discuss  is representation of semantic phenomena and the pros and cons of  tasks that are representation independent.


Speaker bio:

Dr. Diana McCarthy is an affiliated lecturer at the University of Cambridge, UK. She has been active in the field of computational linguistics for over 20 years and specialises in the field of computational lexical semantics. Along with various collaborators, she won best paper awards for her work on automatic detection of sense predominance and for compositionality modelling of compound nouns with distributional semantics. Prior to her affiliation at Cambridge, she held a Dorothy Hodgkin Research Fellowship from the UK Royal Society at the University of Sussex from 2005 until 2009. From 2009 until 2012 she worked as a director and computational linguist for the research-led company 'Lexical Computing Ltd' whose main product is the Sketch Engine. Her work includes evaluation of computational models of lexical semantics and she has been involved in the Senseval (now SemEval) community since the first event in 1998 as both participant and task organiser.



The Alan Turing Institute

1st floor of the British Library, 96 Euston Road, London, NW1 2DB

51.5297753, -0.12665390000006