Semantic 3D Map Change Detection and Update based on Smartphone Visual Positioning System

Accurate localization and 3D maps are increasingly needed for various artificial intelligence based IoT applications such as augmented reality, intelligent transportation, crowd monitoring, robotics, etc. This article proposes a novel semantic 3D map change detection and update based on a smartphone visual positioning system (VPS) for the outdoor and indoor environments. The proposed method presents an alternate solution to SLAM for map update in terms of efficiency, cost, availability, and map reuse. Building on existing 3D maps of recent years, a system is designed to use artificial intelligence to identify high-level semantics in images for positioning and map change detection. Then, a virtual LIDAR that estimates the depth of objects in the 3D map is used to generate a compact point cloud to update changes in the scene. We present an excellent performance of localization with respect to other state-of-the-art smartphone positioning solutions to accurately update semantic 3D maps. It is shown that the proposed solution can position users within 1.9m, and update objects with an average error of 2.1m.

Semantic-Based VPS for Smartphone Localization in Challenging Urban Environments

Accurate smartphone-based outdoor localization system in deep urban canyons are increasingly needed for various IoT applications such as augmented reality, intelligent transportation, etc. The recently developed feature-based visual positioning system (VPS) by Google detects edges from smartphone images to match with pre-surveyed edges in their map database. As smart cities develop, the building information modeling (BIM) becomes widely available, which provides an opportunity for a new semantic-based VPS. This article proposes a novel 3D city model and semantic-based VPS for accurate and robust pose estimation in urban canyons where global navigation satellite system (GNSS) tends to fail. In the offline stage, a material segmented city model is used to generate segmented images. In the online stage, an image is taken with a smartphone camera that provides textual information about the surrounding environment. The approach utilizes computer vision algorithms to rectify and hand segment between the different types of material identified in the smartphone image. A semantic-based VPS method is then proposed to match the segmented generated images with the segmented smartphone image. Each generated image holds a pose that contains the latitude, longitude, altitude, yaw, pitch, and roll. The candidate with the maximum likelihood is regarded as the precise pose of the user. The positioning results achieves 2.0m level accuracy in common high rise along street, 5.5m in foliage dense environment and 15.7m in alleyway. A 45% positioning improvement to current state-of-the-art method. The estimation of yaw achieves 2.3{\deg} level accuracy, 8 times the improvement to smartphone IMU.