Localization of Rocaille ornaments in art images using Convolutional Neural Networks

This thesis implements a convolutional neural network to find Rocaille lines in art images. We show that U-Net can be applied to art images and provide a first baseline for the localization of Rocaille lines.

Description

Annotating images is a cumbersome task for art historians. The objective of this thesis was to eliminate the manual annotation step needed when analyzing art images by applying neural networks to the task of line segmentation. First, an art historian provided 17 annotated images which were converted into black and white masks. Since CNNs are not able to process images bigger than a few hundred pixels, the original images and masks were split into segments of size 224x224. The small number of 17 images made it necessary to use data augmentation techniques like shifts, flips and rotations. The next step was to design an architecture for the neural network. U-Net has shown good results in domains where not enough images are available. In order to improve the performance, U-Net was used in combination with DenseNet as a feature extractor. Furthermore, a special loss function based on focal and smooth dice loss was designed to stabilize the gradient flow.

Results

This thesis provided a first baseline to the task of line segmentation in art images. The neural network is capable of recognizing roughly the location of the lines and achieves about 40% hard dice coefficient. However, the lack of images reduced the performance significantly and caused fluctuations when training. Further research is needed to see whether transfer learning can be applied more extensively to improve the dice coefficient.

Files

Full version of the bachelor thesis (German only)

License

This original work is copyright by University of Bremen.
Any software of this work is covered by the European Union Public Licence v1.2. To view a copy of this license, visit eur-lex.europa.eu.
The Thesis provided above (as PDF file) is licensed under Attribution-NonCommercial-NoDerivatives 4.0 International.
Any other assets (3D models, movies, documents, etc.) are covered by the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit creativecommons.org.
If you use any of the assets or software to produce a publication, then you must give credit and put a reference in your publication.
If you would like to use our software in proprietary software, you can obtain an exception from the above license (aka. dual licensing). Please contact zach at cs.uni-bremen dot de.