Inproceedings3603: Unterschied zwischen den Versionen
Nk6388 (Diskussion | Beiträge) |
Fw7309 (Diskussion | Beiträge) |
||
| (2 dazwischenliegende Versionen von einem anderen Benutzer werden nicht angezeigt) | |||
| Zeile 19: | Zeile 19: | ||
|Title=Knowledge Guided Attention and Inference for Describing Images Containing Unseen Objects | |Title=Knowledge Guided Attention and Inference for Describing Images Containing Unseen Objects | ||
|Year=2018 | |Year=2018 | ||
| + | |Month=Juni | ||
|Booktitle=The Semantic Web. Latest Advances and New Domains. 15th Extended Semantic Web Conference (ESWC), Crete, Greece. | |Booktitle=The Semantic Web. Latest Advances and New Domains. 15th Extended Semantic Web Conference (ESWC), Crete, Greece. | ||
|Publisher=Springer International Publishing. | |Publisher=Springer International Publishing. | ||
}} | }} | ||
{{Publikation Details | {{Publikation Details | ||
| − | |Abstract=Images on the Web encapsulate diverse knowledge about | + | |Abstract=Images on the Web encapsulate diverse knowledge about varied abstract concepts. They cannot be sufficiently described with models learned from image-caption pairs that mention only a small number of visual object categories. In contrast, large-scale knowledge graphs contain many more concepts that can be detected by image recognition models. Hence, to assist description generation for those images which contain visual objects unseen in image-caption pairs, we propose a two-step process by leveraging large-scale knowledge graphs. In the first step, a multi-entity recognition model is built to annotate images with concepts not mentioned in any caption. In the second step, those annotations are leveraged as external semantic attention and constrained inference |
| − | + | in the image description generation model. Evaluations show that our models outperform most of the prior work on out-of-domain MSCOCO image description generation and also scales better to broad domains with more unseen objects. | |
| − | are leveraged as external semantic attention and constrained inference in the image description generation model. Evaluations show that our models outperform most of the prior work on out-of-domain MSCOCO image description generation and also scales better to broad domains with more unseen objects. | + | |Download=Eswc2018-camera-ready.pdf,Presentation-ImageCapUnknownObj-Rettinger-ESWC18.pdf, |
|Forschungsgruppe=Web Science | |Forschungsgruppe=Web Science | ||
}} | }} | ||
Aktuelle Version vom 4. Juni 2018, 13:51 Uhr
Knowledge Guided Attention and Inference for Describing Images Containing Unseen Objects
Knowledge Guided Attention and Inference for Describing Images Containing Unseen Objects
Published: 2018
Juni
Buchtitel: The Semantic Web. Latest Advances and New Domains. 15th Extended Semantic Web Conference (ESWC), Crete, Greece.
Verlag: Springer International Publishing.
Nicht-referierte Veröffentlichung
BibTeX
Kurzfassung
Images on the Web encapsulate diverse knowledge about varied abstract concepts. They cannot be sufficiently described with models learned from image-caption pairs that mention only a small number of visual object categories. In contrast, large-scale knowledge graphs contain many more concepts that can be detected by image recognition models. Hence, to assist description generation for those images which contain visual objects unseen in image-caption pairs, we propose a two-step process by leveraging large-scale knowledge graphs. In the first step, a multi-entity recognition model is built to annotate images with concepts not mentioned in any caption. In the second step, those annotations are leveraged as external semantic attention and constrained inference
in the image description generation model. Evaluations show that our models outperform most of the prior work on out-of-domain MSCOCO image description generation and also scales better to broad domains with more unseen objects.
Download: Media:Eswc2018-camera-ready.pdf,Media:Presentation-ImageCapUnknownObj-Rettinger-ESWC18.pdf