Benchmark

We plan to create a benchmark related to the use of geospatial data in recording cultural heritage. There are two types of datasets that may be considered for the benchmark: (1) an emphasis in data acquisition techniques, in which participants may use their own methods to process different datasets of a particular heritage object, or (2) focus on the data processing part, e.g., a benchmark for semantic segmentation/AI, point cloud processing, or HBIM automation in which the emphasis is on testing the algorithms developed by participants.

Furthermore, two approaches to the development of the benchmark may also be considered. First, a new benchmark with a specific task may be created from existing data from interested members. Secondly, we may also look for an existing benchmark and support its development with. The second approach is preferred in order to avoid redundancy, particularly when the benchmark’s objectives align with our interests. A possible path for this second approach is to reach out to institutions with a good track record in documenting cultural heritage in 3D, such as CyArk.

The benchmark will therefore consist of a known dataset to be proposed to be shared among academic participants, research institutes and companies. With it, the intended objective is to be able to evaluate the performance of different processes, methods, programs, etc. when surveying complex cultural heritage scenarios, either based on image analysis, range-based clouds, or both.

Once the benchmark has been chosen, we will establish the general topic, resolutions, and other characteristics of it, and we will communicate it to the ISPRS so that it can be announced on the website.