Aim of the benchmark

Commissions

The aim of the project is to assess the accuracy and reliability of current methods for calibration and orientation of images acquired by different platforms as well as their integration for image matching and dense point cloud generation. In current research especially the question on how large changes of perspective and scale differences need to be tackled in image orientation and (dense) image matching is not approached systematically.By providing a new benchmark dataset consisting of state-of-the-art sensor data and covering different relevant tasks and scenarios the current status of research is identified and further works will be stimulated.

The DENSE IMAGE MATCHING (DIM) benchmark is performed in cooperation with the EuroSDR’s Scientific Initiative “Benchmark on High Density Image Matching for DSM Computation”, joining both the two available datasets on Dortmund (Germany) and Zurich (Switzerland). The efforts of both the initiatives are contributing to set up a more complete and challenging dataset, considering flights with different features in terms of GSD size and overlap.

Two different typologies of data are available for participants:
1) Nadir and oblique aerial images both on Zurich and Dortmund (Zeche Zollern)
2) Combined block (UAV and terrestrial images) on Dortmund (Zeche Zollern)

Participants' results will be evaluated according to the methodology described by Cavegn et al. (2014). The necessary images have been already oriented for the data delivery, and no GCPs or CPs will be provided.

For more information about the delivered data on Dortmund (Zeche Zollern) and Zurich, please refer to the dedicated webpages.

The IMAGE ORIENTATION (IO) benchmark aims to assess the state-of-the-art in multi-platform and multi-view image orientation and calibration. One current problem is to retrieve reliable and efficient matches when large different perspectives and scales are available, such as in modern multi-camera oblique systems, or when UAV-based images are to be combined with images taken from the ground.

For this benchmark again two image sets are available:
1) Nadir and oblique aerial images Dortmund (Centre)
2) Combined block (UAV and terrestrial images) on Dortmund (Centre)

We will provide some selected ground control points and evaluate the bundle block adjustment result through residuals at check points and tie point quality at planar faces. Details are described in Nex et al. (2015).

For further information about the acquired data in Dortmund, see the description below.

Terms of use

The data users should include the following acknowledgement in any publication resulting from the dataset: "The authors would like to acknowledge the provision of the datasets by ISPRS and EuroSDR, released in conjuction with the ISPRS scientific initiative 2014 and 2015, lead by ISPRS ICWG I/Vb."

The benchmark team would appreciate if participants could cite the following paper reporting the aims of the benchmark and the data detailed description:

Nex, F., Gerke, M., Remondino, F., Przybilla H.-J., Bäumker, M., Zurhorst, A., 2015. ISPRS Benchmark for Multi-Platform Photogrammetry. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. II-3/W4, pp.135-142.

Acknowledgements

Data acquisition and pre-processing was made feasible through the funds provided by ISPRS (Scientific Initiative) and EuroSDR. Further we are grateful to Aerowest GmbH for sponsoring part of the PentaCam flight. We thank Pix4D for providing us a research license of Pix4Dmapper.

General description of the acquired data in Dortmund:

Two different test areas have been acquired over the city of Dortmund. These areas have been chosen according to different criteria, such as: (i) different typologies of buildings; (ii) security for people and instruments and easy accessibility to the whole area; (iii) technical support and availability of local surveying companies; (iv) easiness in getting flight permissions (for UAV flights).

1) City center – Dortmund: it is a dense urban area with tall modern buildings as well as historical façades (churches and public buildings) as shown in Figure 1. The yellow area defines the area acquired by the airborne oblique image system (IGI PentaCam) while the red area shows the areas acquired by UAV platforms and terrestrial images as well as by TLS. This second area contains besides others the Dortmund town hall and public offices.

Figure 1 - City center test area

Data acquired on that area are listed below:

a) Oblique images: 1260 images (252 images for each camera) were acquired with PentaCam IGI (flown by AeroWest - Dortmund) on May 19, 2014, around approximately 10-12h CET. The GSD is 10 cm in nadir images and varies from 8 to 12cm in the oblique views. The overlap is 75% / 80% (along / across-track directions) for the nadir images while it is 80% / 80% for oblique images.

b) UAV: A multi-rotor DIJ S800 (provided by ÖbVI Zurhorst) was used for the data acquisition on June 8, 2014: 1073 images were acquired with this drone over the “Rathaus (city hall)”, the “Stadthaus” as well as the obelisk and the surrounding square area. The images were acquired during 4 different flights and an image GSD ranging from 1 to 3 cm was adopted in each flight. Both nadir and oblique images were acquired on the area. The camera used with the drone was a Sony Nex-7 (the same as used for terrestrial acquistition, see below). Note that because of bad weather the UAV flight needed to be done some weeks after the terrestrial acquisition. Targets at the buildings were no more available then!

c) Terrestrial images: The whole area was surveyed using three different cameras on May 11, 2014:Ricoh GXR (1724 images), Sony Nex-7 (2858 images) and Canon 600D (1452 images). The images were acquired considering different distances from the analyzed object and using convergent and rotated views in order to enable the complete object reconstruction.

d) Terrestrial Laser Scanner (TLS): X scans were acquired by Hochschule Bochum (person in charge Prof. Przybilla) using a ZF 5010C scanner, a scanner with 1 mm accuracy on May 11, 2014 as well. The two main buildings and the surrounding square were acquired using a 2 mm resolution step. More than xx millions of points where acquired.

e) Aerial Laser Scanner (ALS): Point clouds over the test area were provided by the Municipality of Dortmund for free. The point clouds have a 4 pts/m2 resolution.

f) Total station (and GNSS RTK in the red area): A topographic network and a GNSS survey were finally set up by Hochschule Bochum (Prof. Przybilla) for the geo-localization of the data. This data was acquired on the same day with the terrestrial images and TLS. 17 ground control points with a good distribution, as well as 91 target coordinates on the building façades where surveyed as GCPs and CKs. Two GNSS Leica-1200 and Leica total stations were used for this task. All points were transformed to UTM, Zone 32N. The TLS scans were registered in that system, as well.

2) Museum area -Zeche Zollern: there are several detached houses and the ex-mining area (museum) with historical buildings (Figure 2). The area acquired by oblique image system is reported in yellow while the area surveyed by UAV, terrestrial images as well as TLS is defined by the red rectangle.

Figure 2 - Zeche Zollern test area

Data acquired on that area are listed below:

a) Oblique images: 905 images acquired by AeroWest (Dortmund) with the PentaCam IGI system. The image GSDs and the overlaps are the same of the former test area; the images were taken during the same flight (i.e. on May 19, 2014).

b) UAV: Two different UAVs have been used for the image acquisition task: the DIJ S800 multi-rotor platform and the Mavinci fixed wing airplane. Both the platforms were provided by ÖbVI Zurhorst. A Sony Nex-7 was installed onboard the multi-rotor system, while a (lighter) Panasonic GX-1 was mounted on the Mavinci UAV (584 images). The multi-rotor UAV allowed to acquire both nadir and oblique image views of the Zeche museum areas (1305 images). The fixed-wing platform allowed to acquire 584 nadir images on a wider area (i.e. the museum as well as the surrounding buildings).

c) Terrestrial images: Two different cameras were used for the data acquisition (Canon D600 with Voigtländer fix focus lens, Nex-7). Three different buildings (“lohnhalle”, “pferdestall” and “verwaltung”) were acquired with both the cameras. Canon D600 captured 720 images while Nex-7 968 images. A sub-centimeter GSD was assured in all the acquisitions.

d) Terrestrial Laser Scanner (TLS): Some existing point clouds of the Zeche Zollern (with a ZF 5010c) were provided by Hochschule Bochum.

e) Aerial Laser Scanner (ALS): Point clouds over the test area were provided by the Municipality of Dortmund for free. The point clouds have a 4 pts/m2 resolution.

f) Total station: A topographic network was realized using Leica instruments: 176 targets on the building façades and 46 points on the ground of the museum area and the surrounding car-park were acquired as GCP / CP.

g) GNSS acquisitions: Static acquisitions were performed on several points in the museum area to assure a better acquisition. Several levelling measures were also performed in order to increase the quality in the height measurements. Two Leica-1200 instruments were used for this task.

Figure 3 - Example of the acquired terrestrial (a), zoom to show the targets (b) UAV (c) and airborne oblique images (d)