Home
Posts
Tag
About
Associatively Segmenting Instances and Semantics in Point Clouds
Associatively Segmenting Instances and Semantics in Point Clouds
2019-03-27 15:44:31
#paper
* Authors：Xinlong Wang / Chunhua Shen / Jiaya Jia * Abstract * real scene / precisely and intuitively / diversified elements / informative 3D scene * instance segmentation benefit from semantic segmentation through learning semantic-aware point-level instance embedding * semantic features of the points belonging to the same instances are fused together to make more accurate per-point * http://github.com/WXinlong/ASIS * 1.Introduction * be parsed to / panoptic segmentation / raw point / step-wise paradigm / sub-optimal and inefficient / tailor * The former one distinguishes different instances of the same class clearly， while the latter one wants them to have the same label. * two sub-optimal approachs: (1) 获取语义分割的结果，在每个语义分割获取的类上进行实例分割 (2) 获取实例分割的结果，并对实例分割结果进行识别，确认不同的实例应该属于哪一类 * ASIS (Associatively Segmenting Instances and Semantica) * 2.Related Work * Instance Segmentation * inspired by R-CNN, segment instances by proposing segment candidates * based on bounding box proposals * Mask R-CNN * using the learned associative embedding * discriminative loss function which enables to learn the pixel-level instance embedding efficiently * sub-grouping problems * similarity matrix of a point cloud SGPN * Semantic Segementation * 3D-FCNN * Pointnet * SPG * Deep Learning on Point Clouds * multiview rendering images part of contextual information in point cloud is left behind during the projection process * volumetric representations both computationally and memory intensive due to the sparsity of point clouds and the heavy computation of 3D convolutions * raw point PointNet/PointNet++/RSNet/DGCNN/PointCNN * 3.Our Method * 3.1.A Simple Baseline * a shaed encoder two parallel decoders * Point cloud —>feature encoder—>feature matrix N_P—> * semantic segmentation—>decode the feature martix to N_P*N_F—>feature matrix F_SEM—>semantic predictions P_SEM N_P*N_C * instance segmentation—>decode the feature martix to N_P*N_F—>feature matrix F_INS—>predict per-point instance embedding E_INS N_P*N_E * Loss Function is tailored from [6] * 3.2.Mutual Aid * Semantic-aware Instance Segmentation * F_SINS = F_INS + FC(F_SEM) * 将突出的语义特征加入到实例分割的特征矩阵中，使得矩阵对图像的语义特征敏感 * Instance-fused Semantic Segmentation * K nearest neighbor * channel-wise max aggregation * 利用K近邻算法，将原始数据中对不同类的隶属关系(亲切程度)通过N_P*K表现出来，然后跟语义分割数据放在一起形成三维特征空间，并根据信道最大聚合，将三维特征空间中的最大隶属度提取出来，进行语义分割。 * 4.Experiments * 4.1.Experiment Settings * Darasets * S3DIS * 9-dim feature vector XYZ，RGB，normalized coordinates as to the room * 6 areas 272 room 13 categories * 3D scans Matterport Scanners * ShapeNet * 3-dim XYZ * 16881 16 categories * 3D shapes from 16 categories * Evaluation Metrics * Semantic segmentation * over-all accuracy OACC * mean accurac mACC * mean IoU mIoU * Instance segmentation * CoV * WcoV * mean precision mPrec * mean recall mRec * Training and Inference Details * vanilla * RePr * 4.2.S3DIS Results * 4.2.1.Baseline Method * efffective and efficient * 4.2.2.ASIS * the impact of the ASIS on evaluation metrics are analysised * 4.2.3.Analsis * Ablative Analysis * Category-based Analysis * 4.2.4.Qualitative Results * 4.2.5.Computation Time * 4.3.ShapeNet Results * 5.Conclusion * hoping the novel design provides insights to future works on segmentation tasks panoptic segmentation and beyondRe * References * [6] Semantic instance segmentation with a discriminative loss function