Project / 10 · Applied Research
Object Recognition from Point Cloud
Recognizing and classifying objects within 3D point cloud data.
What Who is the 3D scanner?
Environmental 3D scanning is increasingly carried out using mobile robotic platforms, including drones, autonomous vehicles, and, more recently, quadruped robots. Colleagues from Saxion research group Mechatronics provided a Spot robot dog to test the on board scanning data output and its potential uses. As the quadruped moves through space, it creates a map to help it navigate and avoid obstacles. The generated 3D map served as the basis for subsequent processing and analysis.
Data processing
The point cloud data from Spot was imported into the Rhino environment and subjected to meshing. Further segmentation and plane averaging were used to create a base for a bridge design to test whether the data provides sufficient and accurate enough detail to design infrastructure over it.
3D concrete printing and fitting on location
Conclusion
Spot’s integrated scanning equipment produces discretised point cloud data which is too inaccurate for direct use in designing infrastructure. However, it can serve as a first pass scanning tool, allowing an algorithm to identify objects of interest and guide more detailed scans where needed. For this, the project focused on exploration of point cloud object recognition directly within the Rhino + Grasshopper environment, opening up possibilities for automation of the analysis of the scanned data.
A variety of shapes and objects
The following scanning session aimed to capture a range of common primitive geometries and more complex forms. An indoor setting became Spot’s playground for generating point cloud data enriched with contextual information used in the next stages of the project. The exploratory research focused on investigating different methods for processing point cloud data to identify and recognise shapes and objects.
Object recognition from image point cloud intersection
An image recognition model was trained on a Grasshopper generated database of primitive shapes. Using a combination of animated sliders, a training set was prepared and sent out to a model for learning. The images were designed to imitate point cloud data where profiles might be incomplete or have a lot of noise.
2D to 3D
After testing 2D image processing and shape identification, the next step was to identify the workflow needed to process 3D data directly. Object recognition from 3D point cloud data was done using PointNet, a pre trained Neural Network on office furniture.
