Results of benchmark test on indoor modelling

The quality of submitted models is measured per dataset in terms of completeness, correctness and accuracy only for wall elements. To see the results for each dataset click the dataset name.

Completeness is defined as the area of intersection between the source and reference surfaces for a buffer size b over the total area of reference surfaces.

Correctness is defined as the area of intersection between the source and reference surfaces for a buffer size b over the total area of source surfaces.

Accuracy is measured as the median absolute distance between sample points on the reference surfaces and the closest surface in the source model as a function of a cut-off distance r. Lower values represent higher accuracies.

A more detailed description of the quality measures can be found in this paper and this paper.

The tables show the measures for b = 10 cm and r = 10 cm. The figures show the measures across a range of b and r from 0 to 15 cm. To see the full resolution figures click the corresponding image.

To see the complete evaluation results for each individual submission click the authors hyperlink in the tables.

TUB1

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Tran et al.	University of Melbourne	0.84	0.46	1.34
Ochmann et al.	University of Bonn	0.93	0.55	1.82
Previtali et al.	Politecnico di Milano	0.77	0.50	2.23
Maset et al.	Udine University	0.83	0.48	1.80
Tran & Khoshelham	University of Melbourne	0.82	0.78	2.71
Su et al.	Wuhan University	0.69	0.64	3.11
Ahmed et al.	Hong Kong Polytechnic	0.57	0.55	4.92
Lim et al.	Korea University	0.85	0.80	2.34
Bassier & Vergauwen	KU Leuven	0.89	0.65	1.74

 

TUB2

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Tran et al.	University of Melbourne	0.84	0.57	2.12
Ochmann et al.	University of Bonn	0.82	0.61	2.75
Maset et al.	Udine University	0.62	0.52	4.16
Cui et al.	Shenzhen University	0.53	0.56	5.99
Su et al.	Wuhan University	0.57	0.74	2.94
Ahmed et al.	Hong Kong Polytechnic	0.76	0.92	3.06
Ai	Wuhan University	0.66	0.83	2.42
Lim et al.	Korea University	0.66	0.78	2.15
Bassier & Vergauwen	KU Leuven	0.75	0.48	3.47

 

Fire Brigade

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Tran et al.	University of Melbourne	0.94	0.38	1.41
Ochmann et al.	University of Bonn	0.57	0.24	2.79
Maset et al.	Udine University	0.63	0.36	4.84
Tran & Khoshelham	University of Melbourne	0.78	0.35	2.59
Ai	Wuhan University	0.44	0.32	3.04
Lim et al.	Korea University	0.82	0.50	1.96
Bassier & Vergauwen	KU Leuven	0.53	0.14	3.49

 

UVigo

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Tran et al.	University of Melbourne	0.87	0.64	0.51
Ochmann et al.	University of Bonn	0.54	0.38	4.46
Maset et al.	Udine University	0.44	0.24	4.63
Tran & Khoshelham	University of Melbourne	0.58	0.49	6.66
Ai	Wuhan University	0.52	0.45	5.16
Lim et al.	Korea University	0.65	0.51	4.03
Bassier & Vergauwen	KU Leuven	0.54	0.18	3.77

 

UoM

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Tran et al.	University of Melbourne	0.87	0.77	0.92
Ochmann et al.	University of Bonn	0.72	0.41	3.06
Maset et al.	Udine University	0.72	0.52	3.10
Tran & Khoshelham	University of Melbourne	0.73	0.77	3.77
Su et al.	Wuhan University	0.60	0.73	4.36
Lim et al.	Korea University	0.60	0.70	2.81
Bassier & Vergauwen	KU Leuven	0.69	0.49	3.70

 

GM

Authors	Affiliation	Completeness @ 10 cm	Correctness @ 10 cm	Accuracy (cm) @ 10 cm
Ahmed et al.	Hong Kong Polytechnic	0.29	0.71	4.03
Bassier & Vergauwen	KU Leuven	0.65	0.49	3.70