Results of benchmark test on indoor modelling

The quality of submitted models is measured per dataset in terms of completeness, correctness and accuracy only for wall elements. To see the results for each dataset click the dataset name.

Completeness is defined as the area of intersection between the source and reference surfaces for a buffer size b over the total area of reference surfaces.

Correctness is defined as the area of intersection between the source and reference surfaces for a buffer size b over the total area of source surfaces.

Accuracy is measured as the median absolute distance between sample points on the reference surfaces and the closest surface in the source model as a function of a cut-off distance r. Lower values represent higher accuracies.

A more detailed description of the quality measures can be found in this paper and this paper.

The tables show the measures for b = 10 cm and r = 10 cm. The figures show the measures across a range of b and from 0 to 15 cm. To see the full resolution figures click the corresponding image.

To see the complete evaluation results for each individual submission click the authors hyperlink in the tables.

Authors Affiliation Completeness
@ 10 cm
Correctness
@ 10 cm
Accuracy (cm)
@ 10 cm
Tran et al. University of Melbourne 0.84 0.46 1.34
Ochmann et al. University of Bonn 0.93 0.55 1.82
Previtali et al. Politecnico di Milano 0.77 0.50 2.23
Maset et al. Udine University 0.83 0.48 1.80
Tran & Khoshelham University of Melbourne 0.82 0.78 2.71
Su et al. Wuhan University 0.69 0.64 3.11
Ahmed et al. Hong Kong Polytechnic 0.57 0.55 4.92
Lim et al. Korea University 0.85 0.80 2.34
Bassier & Vergauwen KU Leuven 0.89 0.65 1.74

 

Authors Affiliation Completeness
@ 10 cm
Correctness
@ 10 cm
Accuracy (cm)
@ 10 cm
Tran et al. University of Melbourne 0.84 0.57 2.12
Ochmann et al. University of Bonn 0.82 0.61 2.75
Maset et al. Udine University 0.62 0.52 4.16
Cui et al. Shenzhen University 0.53 0.56 5.99
Su et al. Wuhan University 0.57 0.74 2.94
Ahmed et al. Hong Kong Polytechnic 0.76 0.92 3.06
Ai Wuhan University 0.66 0.83 2.42
Lim et al. Korea University 0.66 0.78 2.15
Bassier & Vergauwen KU Leuven 0.75 0.48 3.47

 

Authors Affiliation Completeness
@ 10 cm
Correctness
@ 10 cm
Accuracy (cm)
@ 10 cm
Tran et al. University of Melbourne 0.94 0.38 1.41
Ochmann et al. University of Bonn 0.57 0.24 2.79
Maset et al. Udine University 0.63 0.36 4.84
Tran & Khoshelham University of Melbourne 0.78 0.35 2.59
Ai Wuhan University 0.44 0.32 3.04
Lim et al. Korea University 0.82 0.50 1.96
Bassier & Vergauwen KU Leuven 0.53 0.14 3.49

 

Authors Affiliation Completeness
@ 10 cm
Correctness
@ 10 cm
Accuracy (cm)
@ 10 cm
Tran et al. University of Melbourne 0.87 0.64 0.51
Ochmann et al. University of Bonn 0.54 0.38 4.46
Maset et al. Udine University 0.44 0.24 4.63
Tran & Khoshelham University of Melbourne 0.58 0.49 6.66
Ai Wuhan University 0.52 0.45 5.16
Lim et al. Korea University 0.65 0.51 4.03
Bassier & Vergauwen KU Leuven 0.54 0.18 3.77

 

Authors Affiliation Completeness
@ 10 cm
Correctness
@ 10 cm
Accuracy (cm)
@ 10 cm
Tran et al. University of Melbourne 0.87 0.77 0.92
Ochmann et al. University of Bonn 0.72 0.41 3.06
Maset et al. Udine University 0.72 0.52 3.10
Tran & Khoshelham University of Melbourne 0.73 0.77 3.77
Su et al. Wuhan University 0.60 0.73 4.36
Lim et al. Korea University 0.60 0.70 2.81
Bassier & Vergauwen KU Leuven 0.69 0.49 3.70

 

Logo

The International Society for Photogrammetry and Remote Sensing is a non-governmental organization devoted to the development of international cooperation for the advancement of photogrammetry and remote sensing and their applications. The Society operates without any discrimination on grounds of race, religion, nationality, or political philosophy.

Our Contact

ISPRS
c/o
Leibniz University Hannover
Institute of Photogrammetry and GeoInformation
Nienburger Str. 1
D-30167 Hannover
GERMANY
Email: isprs-sg@isprs.org