EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition
EVALITA, the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language, was organised for the fifth time in 2016. Six tasks, covering both re-reruns as well as completely new tasks, and an IBM-sponsored challenge, attracted a total of 34 submissions. An innovative...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Accademia University Press
2017-06-01
|
| Series: | IJCoL |
| Online Access: | https://journals.openedition.org/ijcol/441 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846134093974077440 |
|---|---|
| author | Pierpaolo Basile Viviana Patti Francesco Cutugno Malvina Nissim Rachele Sprugnoli |
| author_facet | Pierpaolo Basile Viviana Patti Francesco Cutugno Malvina Nissim Rachele Sprugnoli |
| author_sort | Pierpaolo Basile |
| collection | DOAJ |
| description | EVALITA, the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language, was organised for the fifth time in 2016. Six tasks, covering both re-reruns as well as completely new tasks, and an IBM-sponsored challenge, attracted a total of 34 submissions. An innovative aspect at this edition was the focus on social media data, especially Twitter, and the use of shared data across tasks, yielding a test set with layers of annotation concerning PoS tags, sentiment information, named entities and linking, and factuality information. Differently from the previous edition(s), many systems relied on a neural architecture, and achieved best results when used. From the experience and success of this edition, also in terms of dissemination of information and data, and in terms of collaboration between organisers of different tasks, we collected some reflections and suggestions that prospective EVALITA chairs might be willing to take into account for future editions. |
| format | Article |
| id | doaj-art-3b915099016748c9804f2e43f9174a23 |
| institution | Kabale University |
| issn | 2499-4553 |
| language | English |
| publishDate | 2017-06-01 |
| publisher | Accademia University Press |
| record_format | Article |
| series | IJCoL |
| spelling | doaj-art-3b915099016748c9804f2e43f9174a232024-12-09T13:21:59ZengAccademia University PressIJCoL2499-45532017-06-01319312710.4000/ijcol.441EVALITA Goes Social: Tasks, Data, and Community at the 2016 EditionPierpaolo BasileViviana PattiFrancesco CutugnoMalvina NissimRachele SprugnoliEVALITA, the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language, was organised for the fifth time in 2016. Six tasks, covering both re-reruns as well as completely new tasks, and an IBM-sponsored challenge, attracted a total of 34 submissions. An innovative aspect at this edition was the focus on social media data, especially Twitter, and the use of shared data across tasks, yielding a test set with layers of annotation concerning PoS tags, sentiment information, named entities and linking, and factuality information. Differently from the previous edition(s), many systems relied on a neural architecture, and achieved best results when used. From the experience and success of this edition, also in terms of dissemination of information and data, and in terms of collaboration between organisers of different tasks, we collected some reflections and suggestions that prospective EVALITA chairs might be willing to take into account for future editions.https://journals.openedition.org/ijcol/441 |
| spellingShingle | Pierpaolo Basile Viviana Patti Francesco Cutugno Malvina Nissim Rachele Sprugnoli EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition IJCoL |
| title | EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition |
| title_full | EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition |
| title_fullStr | EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition |
| title_full_unstemmed | EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition |
| title_short | EVALITA Goes Social: Tasks, Data, and Community at the 2016 Edition |
| title_sort | evalita goes social tasks data and community at the 2016 edition |
| url | https://journals.openedition.org/ijcol/441 |
| work_keys_str_mv | AT pierpaolobasile evalitagoessocialtasksdataandcommunityatthe2016edition AT vivianapatti evalitagoessocialtasksdataandcommunityatthe2016edition AT francescocutugno evalitagoessocialtasksdataandcommunityatthe2016edition AT malvinanissim evalitagoessocialtasksdataandcommunityatthe2016edition AT rachelesprugnoli evalitagoessocialtasksdataandcommunityatthe2016edition |