Reading: "Hyper3D: 3D Graphics Software for Examining Cultural Artifacts"

Computer Graphics y paper

Hyperspectral Imaging : collects and processes information from across the electromagnetic spectrum. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifying materials, or detecting processes. (from wikipedia)

Abstract
Many digital imaging technologies have become valuable tools in physical art conversation. Yet, using such high-tech digital data is difficult for beginners in terms of complexities of user interfaces and limitation of digital data formats. The software we present attempts to solve these problems, allowing conservators to access different forms of information and to view a variety of image types simultaneously.

[気になった文: The field of computer graphics has developed many open-source frameworks for processing 3D geometric models [Cignoni et al. 2008; Levy 2008; Mobius and Kobbelt 2010]]


Contributions

  • The integrated visualization of various types of imaging data for art restoration in a single software package.
  • A coherent system for saving and retrieving user-generated metadata and annotations alongside the images displayed.
  • A publicly accessible open-source framework for extensive future modification.

Previous Software Frameworks

  • ImageJ : for image processing; doesn't deal with 3D data
  • GIMP: for image manipulation; doesn't deal with 3D data
  • MeshLab: for mesh processing; can handle various types of 3D mesh data and enables various geometric operations
  • Graphite: for research in geometric processing that can handle various types of 3D data and includes implementations of various geometry processing algorithms
  • OpenFlipper: geometric processing toolkit
  • OsiriX: an open source software that imports data from multiple image modalities (focuses on volumetric scan data)
  • ImaNote: : an open-source Web-based viewing and annotation tool

Basic Structure of the Software Package

  • 3 main Application Programming Interfaces:
    • For graphics/visualization engine: the Visualization Toolkit(VTK graphics API) and the Insight Segmentation and Registration Toolkit(ITK); VTK = object-oriented wrapper for the OpenGL pipeline. ITK = provides basic image processing modules.
    • For user interface: Qt API; Qt has been commonly used in the previous geometric processing softwares.
    • For linear algebra operation: Armadillo API; this wraps BLAS(Basic Linear Algebra Subprogram) and LAPACK(Linear Algebra Package) operations

(The window objects are mainly rendered by the Qt API, and the mouse interaction and events are controlled by both Qt and VTK API. Use Qt API to implement the digital annotation and metadata system.)

Software Functions

  • provides common rendering modes: points, wireframe, and surface
  • delivers conventional information about the model or image, such as the polygon count, current polygon index, surface normal of the selected polygon, world coordinates, texture file information.

ITK : Convert source data (CT data) into 3D voxel values, and pass them to VTK.
VTK : Use each module to process the data (Mapper - > Actor -> Renderer), render them, and pass them to QT.
QT: Visualize the rendered data.

Strength and Weakness

  • Integrate the viewing capabilities of previous projects for 2D and 3D images and 3D geometric objects
  • Specifically, capable to visualize hyperspectral textures or to display volumetric data
  • Allows the user to document and restore the visualization of an object(bookmarking)
  • Cannot conduct geometric processing needed for each type of data (relies on specialized software for this part)

Ideas for Extention

  • I thought that this software can be used not only for art conservation but also for police investigation of a crime scene. The software's key features such as accessible user interface, hyperspectral visual inspection, and detailed annotation/metadata documenting system seem to be helpful for initial (quick) investigation of a crime scene, although detailed and accurate investigation would require forensic teams.
  • Seems useful if all the documentations and image data can be stored online so that users can access the data from everywhere, which might encourage collaborative work with others.

Open Questions

  • The idea of online storage reminds me of the Photo Tuorism project, which introduces a system for 3D reconstruction from photographs (of a historical monument) that are accumulated through Internet photo sharing sites. If these historical monuments are also the target for art conservation, what ways can we think of integrating these two systems
  • It was interesting to see how 3D graphics software is made. What are other ways to use graphics API / What are other ways to develop these graphic softwares ? (such as Sculptris, Blender, etc..)