Benchmark Test

In order to compare algorithms and methods with the same data sets and with high quality data from airborne campaigns, WGI/4 provides benchmarking data set located in Catalonia, Spain

  • Comparison of DEM generation algorithmsThe goal is to give interested scientists of the ISPRS community the opportunity to test their algorithms on DEM generation, to see how they match with the reference data and to compare their results within the scientific community. Also the comparison of the results using stereo data of different spatial resolution should be part of the analysis. 
  • Develop methodology for DEM quality analysisA second goal is to develop further methodology for a common DEM quality analysis with qualitative and quantitative measures. Several proposals exist already and the working group is going to publish them on their website and all research teams may send proposals for that to the working group chair and co-chairs. Still there is a need for more standardized methodologies to quantify the quality even for cases where no (better) reference is available. The main goal is to analyze and discuss pros and cons of the DEM generation methods as well as to trigger the process of establishing standardized DEM quality figures and procedures. 
  • Region and Datasets. The region selected is an area in Catalonia, Spain, including city areas (Terrassa), rural areas and forests in flat and medium undulated terrain as well as steep mountainous terrain. In this archive you find following datasets:
    • 2.5 meter class: Cartosat-1 (later ALOS-PRISM)
    • 0.5 meter class: Worldview-1 (later GeoEye-1 and maybe Worldview-2)
    • ground truth data generated by the Institut Cartogràfic de Catalunya (ICC) and are provided as reference for comparison:
      • Orthoimages from airborne campaigns
      • Digital Elevation Models (DEM) produced by airborne DMC stereo
      • Digital Surface Models from Lidar
  • Using the dataset and reportingA short report is expected from each group stating the methodology used for DEM production, analysis and comparison. A special session will be organized for the next WG events. The WG chairs would expect that everybody who is downloading the data is registering himself or his group by sending an e-mail to peter.reinartz@dlr.de.
    You can download the datasets mentioned above via
    ftp://isprs-dem-benchmarking:p-2PhDTj@ftp.dlr.de/download
    Please upload your results to a folder with your name and the name of your institution to
    ftp://isprs-dem-benchmarking:p-2PhDTj@ftp.dlr.de/work
    Regarding any publication of the data or products thereof, the following acknowledgement has to be stated (accordingly which data have been used):
    Acknowledgement: Special thanks are given to the data providers for the provision of the stereo data sets, namely: Euromap for the Cartosat-1 data, Digital Globe for the Worldview-1 data, ESA/JAXA for the ALOS-PRISM data and ICC Catalunya for the reference data.