Refine
Document Type
- Doctoral thesis (1)
Has Fulltext
- yes (1)
Is part of the Bibliography
- no (1)
Year of publication
- 2018 (1)
Language
- English (1)
Keywords
- Computer Vision (1)
- Computer vision (1)
- Computerlinguistik (1)
- Deep Learning (1)
- Linguistic Interpretation (1)
- Linguistische Interpretation (1)
- Machine Learning (1)
- Maschinelles Lernen (1)
- Maschinelles Sehen (1)
- Textur Beschreibung (1)
Institute
The main part of the research outlined in this thesis is to develop Deep Learning models for the linguistic interpretation of the visual contents. This part is split into two research problems: interactive region segmentation and captioning, and selective texture labeling. In the first attempt, we proposed a novel hybrid Deep Learning architecture whereby the user is able to specify an arbitrary region of the image that should be highlighted and described. The proposed model alternates the bounding box indications of the standard object localization process with the output of a deep interactive segmentation module to achieve a better understanding of the dense image captioning and improve the object localization accuracy. The idea of the next part is to establish a bidirectional correlation between deep texture representation and its linguistic description via a hybrid CNN-RNN model that enables end-to-end learning of the selective texture labeling. This novel architecture provides new opportunities to describe, search, and also retrieve texture images from their linguistic descriptions. To be able to train such a model, we generated a multi-label texture dataset that covers color, material, and pattern labeling simultaneously. Our contribution to the automatic generation of texture descriptions provides an excellent opportunity to enrich the existing vocabulary of the image captioning. Such a conceptual extension can be used for fine-grained captioning applicable in geology, meteorology and other natural sciences where fine-grained image structures are of importance to understand complicated patterns. Apart from Deep Learning technologies, in the final section of the thesis, we proposed a novel approach to define mathematical morphology on color images. To this end, we converted common RGB-values of the color images into a new biconal color space and then combined two approaches of mathematical morphology to give meaning to the maximum and the minimum of the matrix field data and formulate our novel strategy.