To run the example, please install Open3D with pip install open3d-python.
cd /path/to/MinkowskiEngine python -m examples.indoor
Segmentation of a hotel room¶
When you run the example, you will see a hotel room and semantic segmentation of the room. You can interactively rotate the visualization when you run the example. First, we load the data.
def load_file(file_name): pcd = o3d.read_point_cloud(file_name) coords = np.array(pcd.points) colors = np.array(pcd.colors) return coords, colors, pcd
You can provide a quantized coordinates that ensures there would be only one point per voxel, or you can use the new
MinkowskiEngine.TensorField that does not require quantized coordinates to process point clouds. However, since it does the quanization in the main training process instead of delegating the quantization to the data loading processes, it could slow down the training.
Next, you should create a batched coordinates by calling
# Create a batch, this process is done in a data loader during training in parallel. in_field = ME.TensorField( features=torch.from_numpy(colors).float(), coordinates=ME.utils.batched_coordinates([coords / voxel_size], dtype=torch.float32), quantization_mode=ME.SparseTensorQuantizationMode.UNWEIGHTED_AVERAGE, minkowski_algorithm=ME.MinkowskiAlgorithm.SPEED_OPTIMIZED, device=device, )
Finally, we feed-forward the sparse tensor into the network and get the predictions.
# Convert to a sparse tensor sinput = in_field.sparse() # Output sparse tensor soutput = model(sinput) # get the prediction on the input tensor field out_field = soutput.slice(in_field)
After doing some post-processing. We can color the labels and visualize the input and the prediction side-by-side.
The weights are downloaded automatically once you run the example and the weights are currently the top-ranking algorithm on the Scannet 3D segmentation benchmark.
Please refer to the complete indoor segmentation example for more detail.