Semantic Segmentation ===================== To run the example, please install `Open3D `_ with `pip install open3d-python`. .. code-block:: shell cd /path/to/MinkowskiEngine python -m examples.indoor Segmentation of a hotel room ---------------------------- When you run the example, you will see a hotel room and semantic segmentation of the room. You can interactively rotate the visualization when you run the example. First, we load the data. .. code-block:: python def load_file(file_name): pcd = o3d.read_point_cloud(file_name) coords = np.array(pcd.points) colors = np.array(pcd.colors) return coords, colors, pcd You can provide a quantized coordinates that ensures there would be only one point per voxel, or you can use the new :attr:`MinkowskiEngine.TensorField` that does not require quantized coordinates to process point clouds. However, since it does the quanization in the main training process instead of delegating the quantization to the data loading processes, it could slow down the training. Next, you should create a batched coordinates by calling :attr:`MinkowskiEngine.utils.batched_coordinates`. .. code-block:: python # Create a batch, this process is done in a data loader during training in parallel. in_field = ME.TensorField( features=torch.from_numpy(colors).float(), coordinates=ME.utils.batched_coordinates([coords / voxel_size], dtype=torch.float32), quantization_mode=ME.SparseTensorQuantizationMode.UNWEIGHTED_AVERAGE, minkowski_algorithm=ME.MinkowskiAlgorithm.SPEED_OPTIMIZED, device=device, ) Finally, we feed-forward the sparse tensor into the network and get the predictions. .. code-block:: python # Convert to a sparse tensor sinput = in_field.sparse() # Output sparse tensor soutput = model(sinput) # get the prediction on the input tensor field out_field = soutput.slice(in_field) After doing some post-processing. We can color the labels and visualize the input and the prediction side-by-side. .. image:: ../images/segmentation.png The weights are downloaded automatically once you run the example and the weights are currently the top-ranking algorithm on the `Scannet 3D segmentation benchmark `_. Please refer to the `complete indoor segmentation example `_ for more detail.