LuoJiaNET

LuoJiaNET is a dedicated deep learning framework for Remote Sensing (RS) image interpretation. Its design particularly takes the remote sensing data characteristics into consideration, e.g., large image size, large-scale change, multiple channels, and imbedded geographic knowledge. LuoJiaNET supports various Artificial Intelligence (AI) hardware, i.e., CPUs, GPUs, and Ascend NPUs. It employs a Graph-Model-Spectral collaboration core layer that converts a front-end model into Directional Acyclic Graphs (DAGs) and optimizes the graph via intermediate representations (IRs). Model with remote sensing data characteristics is scalable in memory and optimized in the data channel. Furthermore, the core layer boosts the spectral performance through geographic knowledge extraction, such as normalized difference vegetation index (NDVI), morphological building index (MBI), and polarization salinity index (PSI). Thus, it solves the problems of applying computer vision network to the RS image interpretation, such as ultra-large remote sensing image classification, multi-spectral image channel selection.

LuoJiaNET provides a model bank for RS image processing. Five typical RS interpretation tasks are currently included, i.e., the scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction. The supported networks includes the VGG-Net, SENet, ResNet, MobileNet, and DenseNet in the scene classification module, the Faster-RCNN and Mask-RCNN in the object detection module, the U-Net, HRNet, FreeNet, DeepLabV3, and DeepLabV3+ in the land-use classification module, the DTCDSCN, DSIFN, and Building_CD in the binary change detection module, 3D reconstruction models such as MVSNet and GCNet in the multi-view 3D reconstruction module. In addition, existing models from other deep learning frameworks such as PyTorch and TensorFlow can be converted into LuoJiaNet conveniently.

A front-end platform for data processing and visualization is also provided in the LuoJiaNET to support various applications. It has a visual modeling tool for online model services that transfers the complex neural network structure into a form of comprehensible DAGs diagram representation. With this modeling tool, users can grasp the workflow of LuoJiaNET’s model training, validation and online deployment. It facilitates non-professional users to use DL in their applications, without knowing complicated parallel-computation programming.

LuoJiaNET provides the user's guide, tutorials, model examples, API references, and educational curricula at its homepage. Users can view and search the online map (temporarily provides Bing Maps, Tiandi Maps, and Gaode Maps) to search for and select the training data and typical models from LuoJiaNET. In addition, users can upload their own satellite dataset for model training, inference, and visualization.

LuoJiaNET also supports the latest OGC TrainingDML-AI standard (https://www.ogc.org/projects/groups/trainingdmlswg) API drafted by an Open AI/ML Training Data Hub, namely LuoJiaSET (http://58.48.42.237/luojiaSet), for developing and sharing Earth AI algorithms and products on benchmark datasets for RS image interpretation.

LuoJiaNET is an open source project. All source code can be found at https://github.com/WHULuoJiaTeam/luojianet. Users can sign up for the LuoJiaNET at http://58.48.42.237/luojiaNet.