Stage-oe-small.jpg

Article3200: Unterschied zwischen den Versionen

Aus Aifbportal
Wechseln zu:Navigation, Suche
K
K
Zeile 24: Zeile 24:
 
{{Publikation Details
 
{{Publikation Details
 
|Abstract=In recent years, named entity linking (NEL) tools were primarily developed in terms of a general approach, whereas today numerous tools are focusing on specific domains such as e. g. the mapping of persons and organizations only, or the annotation of locations or events in microposts. However, the available benchmark datasets necessary for the evaluation of NEL tools do not reflect this focalizing trend. We have analyzed the evaluation process applied in the NEL benchmarking framework GERBIL [37,30] and all its benchmark datasets. Based on these insights we have extended the GERBIL framework to enable a more fine grained evaluation and in depth analysis of the available benchmark datasets with respect to different emphases.This paper presents the implementation of an adaptive filter for arbitrary entities and customized benchmark creation as well as the automated determination of typical NEL benchmark dataset properties, such as the extent of content-related ambiguity and diversity. These properties are integrated on different levels, which also enables to tailor customized new datasets out of the existing ones by remixing documents based on desired emphases. Besides a new system library to enrich provided NIF [11] datasets with statistical information, best practices for dataset remixing are presented, and an in depth analysis of the performanceof entity linking systems on special focus datasets is presented.
 
|Abstract=In recent years, named entity linking (NEL) tools were primarily developed in terms of a general approach, whereas today numerous tools are focusing on specific domains such as e. g. the mapping of persons and organizations only, or the annotation of locations or events in microposts. However, the available benchmark datasets necessary for the evaluation of NEL tools do not reflect this focalizing trend. We have analyzed the evaluation process applied in the NEL benchmarking framework GERBIL [37,30] and all its benchmark datasets. Based on these insights we have extended the GERBIL framework to enable a more fine grained evaluation and in depth analysis of the available benchmark datasets with respect to different emphases.This paper presents the implementation of an adaptive filter for arbitrary entities and customized benchmark creation as well as the automated determination of typical NEL benchmark dataset properties, such as the extent of content-related ambiguity and diversity. These properties are integrated on different levels, which also enables to tailor customized new datasets out of the existing ones by remixing documents based on desired emphases. Besides a new system library to enrich provided NIF [11] datasets with statistical information, best practices for dataset remixing are presented, and an in depth analysis of the performanceof entity linking systems on special focus datasets is presented.
|Download=Extending_GERBIL_2_3__Final_Revision_.pdf
+
|Download=Remixing Entity Linking Evaluation Datasets for Focused Benchmarking.pdf
 
|Link=https://www.fiz-karlsruhe.de/fileadmin/redaktion/Forschung/ISE/Extending_GERBIL_2_3__Final_Revision_.pdf
 
|Link=https://www.fiz-karlsruhe.de/fileadmin/redaktion/Forschung/ISE/Extending_GERBIL_2_3__Final_Revision_.pdf
 
|DOI Name=10.3233/SW-180334
 
|DOI Name=10.3233/SW-180334
 
|Forschungsgruppe=Information Service Engineering
 
|Forschungsgruppe=Information Service Engineering
 
}}
 
}}

Version vom 17. November 2022, 14:41 Uhr


Remixing Entity Linking Evaluation Datasets for Focused Benchmarking


Remixing Entity Linking Evaluation Datasets for Focused Benchmarking



Veröffentlicht: 2018 Dezember

Journal: Semantic Web Journal
Nummer: 2
Seiten: 385-412

Volume: 10


Referierte Veröffentlichung

BibTeX

Tags:Entity LinkingGERBILEvaluationBenchmark


Kurzfassung
[[Abstract::In recent years, named entity linking (NEL) tools were primarily developed in terms of a general approach, whereas today numerous tools are focusing on specific domains such as e. g. the mapping of persons and organizations only, or the annotation of locations or events in microposts. However, the available benchmark datasets necessary for the evaluation of NEL tools do not reflect this focalizing trend. We have analyzed the evaluation process applied in the NEL benchmarking framework GERBIL [37,30] and all its benchmark datasets. Based on these insights we have extended the GERBIL framework to enable a more fine grained evaluation and in depth analysis of the available benchmark datasets with respect to different emphases.This paper presents the implementation of an adaptive filter for arbitrary entities and customized benchmark creation as well as the automated determination of typical NEL benchmark dataset properties, such as the extent of content-related ambiguity and diversity. These properties are integrated on different levels, which also enables to tailor customized new datasets out of the existing ones by remixing documents based on desired emphases. Besides a new system library to enrich provided NIF [11] datasets with statistical information, best practices for dataset remixing are presented, and an in depth analysis of the performanceof entity linking systems on special focus datasets is presented.]]

Download: Media:Remixing Entity Linking Evaluation Datasets for Focused Benchmarking.pdf
Weitere Informationen unter: Link
DOI Link: 10.3233/SW-180334



Forschungsgruppe

Information Service Engineering


Forschungsgebiet