Our work is aimed at the processing of image data, collected during material research. We concentrate ourselves on solutions to following problems

• mutual geometrical alignement of multimodal data - visual, ultraviolet, electronic microscope, infrared
• segmentation of regions of interest (ROIs) - individual samples, layers, distinctive seeds and other sample parts
• data combination (fusion) - all acquired images are depicted in one, fused image, providing all important details in one image
• descriptiopn of regions of interest (ROIs) - texture based and shape based description of individual sample parts for further processing such as classification, image retrieval
• classification - an identification of used materials and possibly determination of the author and/or place of the origin
• image-based retrieval - an efficient way for the look up of art pieces/samples with similar structure as the sample in question

Art conservators make use of new sensors and modern techniques to study, conserve, and restore old and often damaged artworks. The key issue of the art restoration is the material research – an identification of the used materials. Its aim is the location and the classification of inorganic and organic compounds using microanalytical methods, and description of painting layers and their morphology, where the layer is defined as consistent and distinguishable part of the painting profile. Our proposed system Nephele tries to facilitate the work of restorers in this field.

Several of the following image processing modules are incorporated in the Nephele system. They were designed for processing input image data acquired during the material research: microscopic images of minute surface samples (0.3mm in diameter). They are taken off of the selected areas, embedded in a polyester resin, and grounded at a right angle to the surface plane to expose the painting layers of the artwork. The microscopic images are taken in several modalities. Stratigraphy (learning about painting layers) is usually studied in VIS and UV images, where the UV analysis makes use of the luminescence. Different materials have different luminescence, which can help distinguish materials not resolvable otherwise.

 

Data acquisition

Acquired image data can be of low quality due to the manipulation errors and wrong camera setting. Using the image deconvolution techniques image quality can be improved, as it is apparent on the data below. 

Image preprocessing

The ultimate goal of the image preprocessing is the identification and description of the individual material layers. Before the layer localization can start, the multimodal input data have to be brought into geometric alignment, because the VIS and UV image pairs of the sample are often geometrically misaligned due to manipulation errors etc. They can be mutually shifted and rotated in the scanning plane. The proposed image registration method solves the spatial alignment of the image pairs.

After the image rectification, the color layers can be estimated. The segmentation module performs segmentation of the cross-section from the noisy background and also preliminary layer segmentation based on both VIS and UV images. The construction of the full and correct segmentation turns out to be a very complex task, because expert knowledge is often necessary (certain materials cannot be neighbors, others are always together, etc.). Individual seeds and distinctive parts of the cross-section can be segmented, too, for further material description. After the layer segmentation, we have the set of base structures, which are homogenous and can be further described, analyzed and used for more sophisticated tasks such as image based retrieval or material classification.

Image retrieval

For better functionality of the Nephele database, effective tools are implemented to look-up relevant reports. One of them is content-based image retrieval (CBIR), which is recently very popular and is used as a part of multimedia systems in art galleries. The image retrieval exploits similarity of the query sample to the images contained in the archived reports. The visual similarity can point to the same author, used material, or technique. Image below: the blue rectangle marks query images, the rest of images are the most similar samples from the database.

Publications: 

• Zitová Barbara, Beneš Miroslav, Hradilová J., Hradil David: Analysis of painting materials on multimodal microscopic level , Proceedings of SPIE IS&T/SPIE Electronic Imaging 2010 , Eds: Stork D. G., Coddington J., Bentkowska-Kafel A., IS&T/SPIE Electronic Imaging 2010, (San Jose, US, 17.01.2010–21.01.2010)
• Beneš Miroslav, Zitová Barbara, Hradilová J., Hradil David: Image processing in material analyses of artworks, Proceedings of VISAPP 2008, Eds: Ranchordas A. K. N., Araujo H. J., VISAPP Interantional Conference on Computer Vision Theory and Applications, (Funchal, Madeira, PT, 22.01.2008-25.01.2008)
• Beneš Miroslav, Zitová Barbara, Hradilová J., Hradil David: The Material Description and Classification in Nephele System for Artwork Restoration, Proceedings of the 14th International Conference on Virtual Systems and Multimedia, Eds: Ioannides M., Addison A., Georgopoulos A., Kalisperis L., VSMM International Conference on Virtual Systems and Multimedia, (Limassol, CY, 20.10.2008-25.10.2008)
• Zitová Barbara, Beneš Miroslav, Hradil David, Hradilová J.: Nephele:Electronic Database of Materials Research of Paintings, The Future of Heritage, The 3rd Annual Ename International Colloquium, (Gent, BE, 21.03.07-24.03.07)
• Hradilová J., Beneš Miroslav, Zitová Barbara: Projekt elektronické databáze výsledků materiálového průzkumu malířských děl, Technologia Artis, p. 64-69, Nové metody a témata v analýze malířských výtvarných děl, (Praha, CZ, 07.11.2006-07.11.2006)
• Beneš Miroslav, Zitová Barbara: Wavelets for Texture Description in Nephele System, WDS'06 Proceedings of Contributed Papers, p. 151-156, Eds: Šafránková J., Pavlů J., Annual Conference of Doctoral Students /15./ WDS´06 - Week of doctoral students 2006, (Praha, CZ, 06.06.2006-09.06.2006)
• Beneš Miroslav, Zitová Barbara: Nephele: Databáze restaurátorských zpráv s možností vyhledávání podle textové a obrazové informace, ÚTIA AV ČR, (Praha 2005), Internal Publication 2005/43