PC2WF: 3D WIREFRAME RECONSTRUCTION FROM RAW POINT CLOUDS

Abstract

We introduce PC2WF, the first end-to-end trainable deep network architecture to convert a 3D point cloud into a wireframe model. The network takes as input an unordered set of 3D points sampled from the surface of some object, and outputs a wireframe of that object, i.e., a sparse set of corner points linked by line segments. Recovering the wireframe is a challenging task, where the numbers of both vertices and edges are different for every instance, and a-priori unknown. Our architecture gradually builds up the model: It starts by encoding the points into feature vectors. Based on those features, it identifies a pool of candidate vertices, then prunes those candidates to a final set of corner vertices and refines their locations. Next, the corners are linked with an exhaustive set of candidate edges, which is again pruned to obtain the final wireframe. All steps are trainable, and errors can be backpropagated through the entire sequence. We validate the proposed model on a publicly available synthetic dataset, for which the ground truth wireframes are accessible, as well as on a new real-world dataset. Our model produces wireframe abstractions of good quality and outperforms several baselines.

1. INTRODUCTION

Many practical 3D sensing systems, like stereo cameras or laser scanners, produce unstructured 3D point clouds. That choice of output format is really just a "smallest common denominator", the least committal representation that can be reliably generated with low-level signal processing. Most users would prefer a more efficient and more intuitive representation that describes the scanned object's geometry as a compact collection of geometric primitives, together with their topological relations. Specifically, many man-made objects are (approximately) polyhedral and can be described by corners, straight edges and/or planar surfaces. Roughly speaking there are two ways to abstract a point cloud into a polyhedral model: either find the planar surfaces and intersect them to find the edges and corners, e.g., Schnabel et al. (2007) ; Fang et al. (2018) ; Coudron et al. (2018) ; or directly find the salient corner and/or edges, e.g., Jung et al. (2016) ; Hackel et al. (2016) . A wireframe is a graph representation of an object's shape, where vertices correspond to corner points (with high Gaussian curvature) that are linked by edges (with high principal curvature). Wireframes are a good match for polyhedral structures like mechanical parts, furniture or building interiors. In particular, since wireframes focus on the edge structure, they are best suited for piece-wise smooth objects with few pronounced crease edges (whereas they are less suitable for smooth objects without defined edges or for very rough ones with edges everywhere). We emphasise that wireframes are not only a "compression technique" to save storage. Their biggest advantage in many applications is that they are easy to manipulate and edit, automatically or interactively in CAD software, because they make the salient contours and their connectivity explicit. Reconstructed wireframes can drive and help to create 3D CAD models for manufacturing parts, metrology, quality inspection, as well as visualisation, animation, and rendering. Inferring the wireframe from a noisy point cloud is a challenging task. We can think of the process as a sequence of steps: find the corners, localise them accurately (as they are not contained in the point cloud), and link them with the appropriate edges. However, these steps are intricately correlated. For example, corner detection should "know" about the subsequent edge detection: curvature is affected by noise (as any user of an interest point detector can testify), so to qualify as a corner a 3D location should also be the plausible meeting point of multiple, non-collinear edges. Here we introduce PC2WF, an end-to-end trainable architecture for point cloud to wireframe conversion. PC2WF is composed of a sequence of feed-forward blocks, see Fig. 1 for the individual steps of the pipeline. First a "backbone" block extracts a latent feature encoding for each point. The next block identifies local neighbourhoods ("patches") that contain corners and regresses their 3D locations. The final block links the corners with an overcomplete set of plausible candidate edges, collects edge features by pooling point features along each edge, and uses those to prune the candidate set to a final set of wireframe edges. All stages of the architecture that have trainable parameters support back-propagation, such that the model can be learned end-to-end. We validate PC2WF with models from the publicly available ABC dataset of CAD models, and with a new furniture dataset collected from the web. The experiments highlight the benefit of explicit, integrated wireframe prediction: on the one hand, the detected corners are more complete and more accurate than with a generic corner detector; on the other hand, the reconstructed edges are more complete and more accurate than those found with existing (point-wise) edge detection methods.

2. RELATED WORK

Relatively little work exists on the direct reconstruction of 3D wireframes. We review related work for the subtasks of corner and edge detection, as well as works that reconstruct wireframes in 2D or polyhedral surface models and geometric primitives in 3D. Corner Detection. Corner points play a fundamental role in both 2D and 3D computer vision. A 3D corner is the intersection point of ≥3 (locally) non-collinear surfaces. Corners have a well-defined location and a local neighbourhood with non-trivial shape (e.g., for matching). Collectively the set of corners often is sufficient to tightly constrain the scale and shape of an object. Typical local methods for corner detection are based on the 2 nd -moment matrix of the points (resp. normals) in a local neighbourhood (Głomb, 2009; Sipiran & Bustos, 2010; 2011) , a straight-forward generalisation of the 2D Harris operator first proposed for 2D images (Harris et al., 1988) . Also other 2D corner detectors like Susan (Smith & Brady, 1997) have been generalised to 3D (Walter et al., 2009) . Point-level Edge Point Detection. A large body of work is dedicated to the detection of 3D points that lie on or near edges. The main edge feature is high local curvature (Yang & Zang, 2014) . Related features, such as symmetry (Ahmed et al., 2018) , change of normals (Demarsin et al., 2007) , and eigenvalues (Bazazian et al., 2015; Xia & Wang, 2017) can also be used for edge detection. Hackel et al. (2016; 2017) use a combination of several features as input for a binary edge-point classifier. After linking the points into an adjacency graph, graph filtering can be used to preserve only edge points (Chen et al., 2017) . EC-Net (Yu et al., 2018) is the first method we know of that detects edge points in 3D point clouds with deep learning. Learning Structured Models. Many methods have been proposed to abstract point clouds into more structured 3D models, e.g., triangle meshes (Lin et al., 2004; Remondino, 2003; Lin et al., 2004) , polygonal surfaces (Fang et al., 2018; Coudron et al., 2018) , simple parametric surfaces (Schnabel et al., 2007; 2008; Li et al., 2019) . From the user perspective, the ultimate goal would be to reverseengineer a CAD model (Gonzalez-Aguilera et al., 2012; Li et al., 2017b; Durupt et al., 2008) . Several works have investigated wireframe parsing in 2D images (Huang et al., 2018; Huang & Gao, 2019) . LCNN (Zhou et al., 2019a ) is an end-to-end trainable system that directly outputs a vectorised wireframe. The current state-of-the-art appears to be the improved LCNN of Xue et al. (2020) , which uses a holistic "attraction field map" to characterise line segments. Zou et al. (2018) estimate the 3D layout of an indoor scene from a single perspective or panoramic image with an encoder-decoder architecture, while Zhou et al. (2019b) obtain a compact 3D wireframe representation from a single image by exploiting global structural regularities. Jung et al. (2016) reconstruct 3D wireframes of building interiors under a Manhattan-world assumption. Edges are detected heuristically by projecting the points onto 2D occupancy grids, then refined via least-squares fitting under parallelism/orthogonality constraints. Wireframe extraction in 2D images is simplified by the regular pixel grid. For irregular 3D points the task is more challenging and there has been less work. Another related direction of research is 3D shape decomposition into sets of simple, parameterised primitives (Tulsiani et al., 2017; Li et al., 2017a; Zou et al., 2017; Sharma et al., 2018; Du et al., 2018) . Most approaches use a coarse volumetric occupancy grid as input for the 3D data, which  N ⇥ 3 < l a t e x i t s h a 1 _ b a s e 6 4 = " w Y C u v V A t P b A o K y 8 m C X s X Z b D M z o Y = " > A A A C J 3 i c b V D L S g N B E O y N r x h f i R 6 9 D A b B U 9 h V Q Y 9 B L 5 4 k g n l I s o T Z y S Q Z M j O 7 z M y K Y c l X e N U P 8 G u 8 i R 7 9 E y f Z P b j R g o a i q p v u r i D i T B v X / X I K K 6 t r 6 x v F z d L W 9 s 7 u X r m y 3 9 J h r A h t k p C H q h N g T T m T t G m Y 4 b Q T K Y p F w G k 7 m F z P / f Y j V Z q F 8 t 5 M I + o L P J J s y A g 2 V n q 4 7 R k m q E Z n / X L V r b k L o L / E y 0 g V M j T 6 F a f Y G 4 Q k F l Q a w r H W X c + N j J 9 g Z R j h d F b q x Z p G m E z w i H Y t l d j u 8 Z P F x T N 0 b J U B G o b K l j R o o f 6 e S L D Q e i o C 2 y m w G e t l b y 7 + 5 3 V j M 7 z 0 E y a j 2 F B J 0 k X D m C M T o v n 7 a M A U J Y Z P L c F E M X s r I m O s M D E 2 p N y W Q O R + S J 7 S 0 3 N a K t n s v O W k / p L W a c 1 z a 9 7 d e b V + l a V Y h E M 4 g h P w 4 A L q c A M N a A I B A c / w A q / O m / P u f D i f a W v B y W Y O I A f n + w c Q x K Y e < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " w Y C u v V A t P b A o K y 8 m C X s X Z b D M z o Y = " > A A A C J 3 i c b V D L S g N B E O y N r x h f i R 6 9 D A b B U 9 h V Q Y 9 B L 5 4 k g n l I s o T Z y S Q Z M j O 7 z M y K Y c l X e N U P 8 G u 8 i R 7 9 E y f Z P b j R g o a i q p v u r i D i T B v X / X I K K 6 t r 6 x v F z d L W 9 s 7 u X r m y 3 9 J h r A h t k p C H q h N g T T m T t G m Y 4 b Q T K Y p F w G k 7 m F z P / f Y j V Z q F 8 t 5 M I + o L P J J s y A g 2 V n q 4 7 R k m q E Z n / X L V r b k L o L / E y 0 g V M j T 6 F a f Y G 4 Q k F l Q a w r H W X c + N j J 9 g Z R j h d F b q x Z p G m E z w i H Y t l d j u 8 Z P F x T N 0 b J U B G o b K l j R o o f 6 e S L D Q e i o C 2 y m w G e t l b y 7 + 5 3 V j M 7 z 0 E y a j 2 F B J 0 k X D m C M T o v n 7 a M A U J Y Z P L c F E M X s r I m O s M D E 2 p N y W Q O R + S J 7 S 0 3 N a K t n s v O W k / p L W a c 1 z a 9 7 d e b V + l a V Y h E M 4 g h P w 4 A L q c A M N a A I B A c / w A q / O m / P u f D i f a W v B y W Y O I A f n + w c Q x K Y e < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " w Y C u v V A t P b A o K y 8 m C X s X Z b D M z o Y = " > A A A C J 3 i c b V D L S g N B E O y N r x h f i R 6 9 D A b B U 9 h V Q Y 9 B L 5 4 k g n l I s o T Z y S Q Z M j O 7 z M y K Y c l X e N U P 8 G u 8 i R 7 9 E y f Z P b j R g o a i q p v u r i D i T B v X / X I K K 6 t r 6 x v F z d L W 9 s 7 u X r m y 3 9 J h r A h t k p C H q h N g T T m T t G m Y 4 b Q T K Y p F w G k 7 m F z P / f Y j V Z q F 8 t 5 M I + o L P J J s y A g 2 V n q 4 7 R k m q E Z n / X L V r b k L o L / E y 0 g V M j T 6 F a f Y G 4 Q k F l Q a w r H W X c + N j J 9 g Z R j h d F b q x Z p G m E z w i H Y t l d j u 8 Z P F x T N 0 b J U B G o b K l j R o o f 6 e S L D Q e i o C 2 y m w G e t l b y 7 + 5 3 V j M 7 z 0 E y a j 2 F B J 0 k X D m C M T o v n 7 a M A U J Y Z P L c F E M X s r I m O s M D E 2 p N y W Q O R + S J 7 S 0 3 N a K t n s v O W k / p L W a c 1 z a 9 7 d e b V + l a V Y h E M 4 g h P w 4 A L q c A M N a A I B A c / w A q / O m / P u f D i f a W v B y W Y O I A f n + w c Q x K Y e < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " w Y C u v V A t P b A o K y 8 m C X s X Z b D M z o Y = " > A A A C J 3 i c b V D L S g N B E O y N r x h f i R 6 9 D A b B U 9 h V Q Y 9 B L 5 4 k g n l I s o T Z y S Q Z M j O 7 z M y K Y c l X e N U P 8 G u 8 i R 7 9 E y f Z P b j R g o a i q p v u r i D i T B v X / X I K K 6 t r 6 x v F z d L W 9 s 7 u X r m y 3 9 J h r A h t k p C H q h N g T T m T t G m Y 4 b Q T K Y p F w G k 7 m F z P / f Y j V Z q F 8 t 5 M I + o L P J J s y A g 2 V n q 4 7 R k m q E Z n / X L V r b k L o L / E y 0 g V M j T 6 F a f Y G 4 Q k F l Q a w r H W X c + N j J 9 g Z R j h d F b q x Z p G m E z w i H Y t l d j u 8 Z P F x T N 0 b J U B G o b K l j R o o f 6 e S L D Q e i o C 2 y m w G e t l b y 7 + 5 3 V j M 7 z 0 E y a j 2 F B J 0 k X D m C M T o v n 7 a M A U J Y Z P L c F E M X s r I m O s M D E 2 p N y W Q O R + S J 7 S 0 3 N a K t n s v O W k / p L W a c 1 z a 9 7 d e b V + l a V Y h E M 4 g h P w 4 A L q c A M N a A I B A c / w A q / O m / P u f D i f a W v B y W Y O I A f n + w c Q x K Y e < / l a t e x i t > backbone features N ⇥ C < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 B h c e f w i h 3 r i d U c G w v s T b X F j W 7 0 = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i y e J Y B 6 Y L G F 2 M k m G z M 4 u M 7 N i W P I X X v U D / B p v k q t f 4 i S 7 B z e x o K G o 6 q a 7 y 4 8 E 1 8 Z x 5 m h j c 2 t 7 Z 7 e w V 9 w / O D w 6 L p V P W j q M F W V N G o p Q d X y i m e C S N Q 0 3 g n U i x U j g C 9 b 2 J / W F 3 3 5 h S v N Q P p l p x L y A j C Q f c k q M l Z 4 f e o Y H T O N 6 s V + q O F V n C b x O 3 I x U I E O j X 0 a F 3 i C k c c C k o Y J o 3 X W d y H g J U Y Z T w W b F X q x Z R O i E j F j X U k n s I i 9 Z n j z D F 1 Y Z 4 G G o b E m D l + r f i Y Q E W k 8 D 3 3 Y G x I z 1 q r c Q / / O 6 s R n e e g m X U W y Y p O m i Y S y w C f H i f z z g i l E j p p Y Q q r i 9 F d M x U Y Q a m 1 J u i x / k f k h e 0 9 N z W i r Z 7 N z V p N Z J 6 6 r q O l X V P W j q M F W V N G o p Q d X y i m e C S N Q 0 3 g n U i x U j g C 9 b 2 J / W F 3 3 5 h S v N Q P p l p x L y A j C Q f c k q M l Z 4 f e o Y H T O N 6 s V + q O F V n C b x O 3 I x U I E O j X 0 a F 3 i C k c c C k o Y J o 3 X W d y H g J U Y Z T w W b F X q x Z R O i E j F j X U k n s I i 9 Z n j z D F 1 Y Z 4 G G o b E m D l + r f i Y Q E W k 8 D 3 3 Y G x I z 1 q r c Q / / O 6 s R n e e g m X U W y Y p O m i Y S y w C f H i f z z g i l E j p p Y Q q r i 9 F d M x U Y Q a m 1 J u i x / k f k h e 0 9 N z W i r Z 7 N z V p N Z J 6 6 r q O l X V P W j q M F W V N G o p Q d X y i m e C S N Q 0 3 g n U i x U j g C 9 b 2 J / W F 3 3 5 h S v N Q P p l p x L y A j C Q f c k q M l Z 4 f e o Y H T O N 6 s V + q O F V n C b x O 3 I x U I E O j X 0 a F 3 i C k c c C k o Y J o 3 X W d y H g J U Y Z T w W b F X q x Z R O i E j F j X U k n s I i 9 Z n j z D F 1 Y Z 4 G G o b E m D l + r f i Y Q E W k 8 D 3 3 Y G x I z 1 q r c Q / / O 6 s R n e e g m X U W y Y p O m i Y S y w C f H i f z z g i l E j p p Y Q q r i 9 F d M x U Y Q a m 1 J u i x / k f k h e 0 9 N z W i r Z 7 N z V p N Z J 6 6 r q O l X V P W j q M F W V N G o p Q d X y i m e C S N Q 0 3 g n U i x U j g C 9 b 2 J / W F 3 3 5 h S v N Q P p l p x L y A j C Q f c k q M l Z 4 f e o Y H T O N 6 s V + q O F V n C b x O 3 I x U I E O j X 0 a F 3 i C k c c C k o Y J o 3 X W d y H g J U Y Z T w W b F X q x Z R O i E j F j X U k n s I i 9 Z n j z D F 1 Y Z 4 G G o b E m D l + r f i Y Q E W k 8 D 3 3 Y G x I z 1 q r c Q / / O 6 s R n e e g m X U W y Y p O m i Y S y w C f H i f z z g i l E j p p Y Q q r i 9 F d M x U Y Q a m 1 J u i x / k f k h e 0 9 N z W i r Z 7 N z V p N Z J 6 6 r q O l X + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C  A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > M ⇥ C < l a t e x i t A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > ... < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7  + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V 5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7  + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y / D Y r Y J q d a R n h k 0 W z W K I 9 g = " > A A A C I X i c b V D L S s N A F L 1 T X 7 W + W l 2 6 G S y C q 5 C I o M u i G 5 c V 7 Q P a U C b T a T t 0 M g k z E 7 G E f o J b / Q C / x p 2 4 E 3 / G a Z O F q R 6 4 c D j n X u 6 9 J 4 g F 1 8 Z 1 v 1 B p b X 1 j c 6 u 8 X d n Z 3 d s / q N Y O 2 z p K F G U t G o l I d Q O i m e C S t Q w 3 g n V j x U g Y C N Y J p j c L v / P I l O a R f D C z m P k h G U s + 4 p Q Y K 9 0 7 j j O o 1 l 3 H X Q L / J V u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d  u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d  u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d  u F Q + a e k w V p Q 1 a S h C 1 f G J Z o J L 1 j T c C N a J F C O B L 1 j b n 9 Q X f v u F K c 1 D + W S m E f M C M p J 8 y C k x V n p + 6 B k e M I 3 r x X 6 p 4 l S d J f A 6 c T N S g Q y N f h k V e o O Q x g G T h g q i d d d 1 I u M l R B l O B Z s V e 7 F m E a E T M m J d S y W x i 7 x k e f I M X 1 h l g I e h s i U N X q p / J x I S a D 0 N f N s Z E D P W q 9 5 C / M / r x m Z 4 6 y V c R r F h k q a L h r H A J s S L / / G A K 0 a N m F p C q O L 2 V k z H R B F q b E q 5 L X 6 Q + y F 5 T U / P a a l k s 3 N X k 1 o n r a u q 6 1 T d works well for shape primitive fitting but only allows for rather inaccurate localisation of edges and vertices. Li et al. (2019) propose a method for fitting 3D primitives directly to raw point clouds. A limiting factor of their approach is that only a fixed maximum number of primitive instances can be detected. This upper bound is hard-coded in the architecture and prevents the network from fitting more complex structures. Nan & Wonka (2017) propose Polyfit to extract polygonal surfaces from a point cloud. Polyfit first selects an exhaustive set of candidate surfaces from the raw point cloud using RANSAC, and then refines the surfaces. Similarly to our PC2WF, Polyfit works best with objects made of straight edges and planar surfaces. The concurrent work Wang et al. (2020) proposed a deep neural network that is trained to identify a collection of parametric edges. In summary, most 3D methods are either hand-crafted for restricted, schematic settings, or only learn to label individual "edge points". Here we aim for an end-to-end learnable model that directly maps a 3D point cloud to a vectorised wireframe.

3. METHOD

Our full architecture is depicted in Fig. 1 . In the following we describe each block and, where necessary, discuss training and inference procedures. We denote a point cloud of N 3D points with P = {p i } N i=1 where p i ∈ R 3 represents the 3D coordinates of point i. The 3D wireframe model of the point cloud P is denoted by W = (V, E) and it is defined, analogously to an undirected graph, by a set of vertices V which are connected by a set of edges E ⊆ V × V. As the vertices represent points in the 3D space we can think of them as being described by 3D coordinates v i ∈ V ⊂ R 3 . Overall Pipeline. Our feed-forward pipeline consists of four blocks: (1) a convolutional backbone that extracts per-point features; (2) a vertex detector that examines patches of the point cloud and decides which ones contain vertices; (3) a vertex localiser that outputs the location of the vertex within a patch; (4) an edge detector that instantiates candidate edges and prunes them to a wireframe. Backbone. PC2WF first extracts a feature vector per 3D point that encodes its geometric context. Our pipeline is compatible with any (hand-coded or learned) function that outputs a fixed-size descriptor vector per point. We use Fully Convolutional Geometric Features (FCGF, Choy et al., 2019b) , which are compact and capture broad spatial context. The backbone operates on sparse tensors (Choy et al., 2019a) and efficiently computes 32-dimensional features in a single, 3D fully-convolutional pass. Vertex Detector. This block performs binary classification of local patches (neighbourhoods in the point cloud) into those (few) that contain a corner and those that do not. Only patches detected to contain a corner are passed on to the subsequent block. While one could argue that detecting the presence of a vertex and inferring its 3D coordinates are two facets of the same problem, we found it advantageous to separate them. Detecting first whether a patch is likely to contain a corner, then constructing its 3D coordinates reduces the computational cost, as we discard patches without corners early. Empirically it also reduces false positives and increases localisation accuracy. Our patch generation is similar to EC-Net (Yu et al., 2018) . A patch contains a fixed number M of points, chosen by picking a patch centroid and finding the closest (M -1) points in terms of geodesic distance on the underlying surface. The geodesic distance avoids "shortcuts" in regions with thin structures or high curvature. To approximate the geodesic distance, the entire point cloud is linked into a k-nearest neighbour graph, with edge weights proportional to Euclidean distance. Point-to-point distances are then found as shortest paths on that graph. During training we randomly draw positive samples that contain a corner and negative ones that do not. To increase the diversity of positive samples, the same corner can be included in multiple patches with different centroids. For the negative samples, we ensure to draw not only patches on flat surfaces, but also several patches near edges, by doing this negative samples with strong curvature are well represented. For inference, patch seeds are drawn using farthest point sampling. The vertex detector takes as input the (indexed) features of the M points in the patch, max-pooled per dimension to achieve permutation invariance, and outputs a scalar vertex/no_vertex probability. Its loss function L pat is binary cross-entropy. Vertex Localiser. The vertex detector passes on a set of P patches predicted to contain a corner. The M points p i of such a patch form the input for the localisation block. The backbone features f i ∈ R C and the 3D coordinates p i ∈ R 3 of all M points are concatenated and fed into a multi-layer perceptron with fully connected layers, ReLU activations and batch normalisation. The MLP outputs a vector of M scalar weights w i , where i w i = 1, such that the predicted vertex location is the weighted mean of the input points, v = w i p i . The prediction v j is supervised by a regression loss L vert = 1 P P i v i -v GT i 2 2 , summed over all P corner patches. Predicting weights w i rather than directly the coordinates x safeguards against extrapolation errors on unusual patches. We note that the localiser not only improves the corner point accuracy but also improves efficiency, by making it possible to use farthest point sampling rather than testing every input point. Edge Detector. The edge detection block serves to determine which pairs of (predicted) vertices are connected by an edge. Candidate edges e ij are found by linking two vertices v i and v j , see below. To obtain a descriptor for an edge, we sample the N s equally spaced query locations Q = {q k } s-1 k=0 along the line segment between the two vertices, including the endpoints: q 0 = ṽi , q s-1 = ṽj . For each q k we find the nearest 3D input point p N N (k) (by Euclidean distance) and retrieve its backbone encoding f N N (k) . This yields an (N s × C) edge descriptor, which is max-pooled along its first dimension, inspired by the ROI pooling in Faster-RCNN (Ren et al., 2015) and LOI pooling in LCNN (Zhou et al., 2019a) . The resulting ( 1 stride N s × C) descriptor is flattened and fed through an MLP with two fully connected layers, which returns a scalar edge/no_edge score. During training we select a number of vertex pairs from both the ground truth vertex list {v i } G i=1 and the predicted vertex list {ṽ i } K i=1 , and draw positive and negative sets of edge samples, E + , E -. We empirically found that evaluating all vertex pairs contained in E + , E -during training leads to a large imbalance between positive and negative samples. Similar to Zhou et al. (2019a) , we thus draw input edges for the edge detector from the following sets: (a) Positive edges between ground truth vertices: This set comprises all true edges in the ground truth wireframe E gt+ = E (b) Negative edges between ground truth vertices: Two situations are relevant: "spurious edges", i.e., connections between two ground truth vertices that do not share an edge (but lie on the same surface, such that there are nearby points to collect features). And "inaccurate edges" where one of the endpoints is not a ground truth vertex, but not far from one (to cover difficult cases not far from a correct edge). Formally: (c) Positive edges between predicted vertices: Connections between two "correct" predicted vertices that have both been verified to coincide with a ground truth wireframe vertex up to a small threshold + . Formally: (d) Negative edges between predicted vertices: These are (i) "wrong links" as in E gt-, between "correctly" predicted vertices; and (ii) "hard negatives" where exactly one of the two vertices is close to a ground truth wireframe vertex, to cover "near misses". E gt-={e vi,vj = (v i , v j ) | v i , v j ∈ V, e ij / ∈ E, ∀p on e vi,vj , ∃p ∈ P, p -p 2 < } ∪ {e i,p k = (v i , p k ) | (v i , v j ) ∈ E, p k ∈ P, 1 ≤ v j -p k 2 ≤ 2 } E pred+ = {e ṽi,ṽj = (ṽ i , ṽj ) | min ṽi -v k 2 < + , min ṽj -v l 2 < + , (v k , v l ) ∈ E gt+ } E pred-={e ṽi,ṽj = (ṽ i , ṽj )  | min ṽi -v k 2 < -, min ṽj -v l 2 < -, (v k , v l ) ∈ E gt-} ∪ {e ṽi,ṽj = (ṽ i , ṽj ) | 1 ≤ ṽi -v k 2 ≤ 2 , ṽj -v l 2 ≤ 2 , v k , v l ∈ V} J l U p j T d 2 K 7 A X 9 x w = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R B K E k I u i y K I L L C v Y B T S y T 6 a Q d O n k w c y M t I X 6 N W / 0 A / 8 S d O 3 H r F z h p s j C t B y 4 c z r m X e + 9 x Q s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D C J B a J s E P B A 9 B 0 v K m U / b w I D T X i g o 9 h x O u 8 7 k J v W 7 T 1 R I F v g P M A u p 7 e G R z 1 x G M C h p U D 2 0 P A x j g n l 8 m z z G F t A p x C M 4 S 5 J B t W 4 0 j D n 0 Z W L m p I 5 y t A Y 1 r W w N A x J 5 1 A f C s Z R 9 0 w j B j r E A R j h N K l Y k a Y j J B I 9 o X 1 E f e 1 T a 8 f y F R D 9 R y l B 3 A 6 H K B 3 2 u / p 2 I s S f l z H N U Z 3 q w X P R S 8 T + v H 4 F 7 Z c f M D y O g P s k W u R H X I d D T P P Q h E 5 Q A n y m C i W D q V p 2 M s c A E V G q F L Y 5 X + C G e Z q c X t E x S 2 Z m L S S 2 T z n n D N B r m / U W 9 e Z 2 n W E Z H 6 B i d I h N d o i a 6 Q y 3 U R g Q 9 o x f 0 i t 6 0 d + 1 T + 9 K + s 9 a S l s 8 c o A K 0 n 1 / g J K 2 u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + c b p E S l i 3 9 h 1 J l U p j T d 2 K 7 A X 9 x w = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R B K E k I u i y K I L L C v Y B T S y T 6 a Q d O n k w c y M t I X 6 N W / 0 A / 8 S d O 3 H r F z h p s j C t B y 4 c z r m X e + 9 x Q s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D C J B a J s E P B A 9 B 0 v K m U / b w I D T X i g o 9 h x O u 8 7 k J v W 7 T 1 R I F v g P M A u p 7 e G R z 1 x G M C h p U D 2 0 P A x j g n l 8 m z z G F t A p x C M 4 S 5 J B t W 4 0 j D n 0 Z W L m p I 5 y t A Y 1 r W w N A x J 5 1 A f C s Z R 9 0 w j B j r E A R j h N K l Y k a Y j J B I 9 o X 1 E f e 1 T a 8 f y F R D 9 R y l B 3 A 6 H K B 3 2 u / p 2 I s S f l z H N U Z 3 q w X P R S 8 T + v H 4 F 7 Z c f M D y O g P s k W u R H X I d D T P P Q h E 5 Q A n y m C i W D q V p 2 M s c A E V G q F L Y 5 X + C G e Z q c X t E x S 2 Z m L S S 2 T z n n D N B r m / U W 9 e Z 2 n W E Z H 6 B i d I h N d o i a 6 Q y 3 U R g Q 9 o x f 0 i t 6 0 d + 1 T + 9 K + s 9 a S l s 8 c o A K 0 n 1 / g J K 2 u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + c b p E S l i 3 9 h 1 J l U p j T d 2 K 7 A X 9 x w = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R B K E k I u i y K I L L C v Y B T S y T 6 a Q d O n k w c y M t I X 6 N W / 0 A / 8 S d O 3 H r F z h p s j C t B y 4 c z r m X e + 9 x Q s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D C J B a J s E P B A 9 B 0 v K m U / b w I D T X i g o 9 h x O u 8 7 k J v W 7 T 1 R I F v g P M A u p 7 e G R z 1 x G M C h p U D 2 0 P A x j g n l 8 m z z G F t A p x C M 4 S 5 J B t W 4 0 j D n 0 Z W L m p I 5 y t A Y 1 r W w N A x J 5 1 A f C s Z R 9 0 w j B j r E A R j h N K l Y k a Y j J B I 9 o X 1 E f e 1 T a 8 f y F R D 9 R y l B 3 A 6 H K B 3 2 u / p 2 I s S f l z H N U Z 3 q w X P R S 8 T + v H 4 F 7 Z c f M D y O g P s k W u R H X I d D T P P Q h E 5 Q A n y m C i W D q V p 2 M s c A E V G q F L Y 5 X + C G e Z q c X t E x S 2 Z m L S S 2 T z n n D N B r m / U W 9 e Z 2 n W E Z H 6 B i d I h N d o i a 6 Q y 3 U R g Q 9 o x f 0 i t 6 0 d + 1 T + 9 K + s 9 a S l s 8 c o A K 0 n 1 / g J K 2 u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + c b p E S l i 3 9 h 1 J l U p j T d 2 K 7 A X 9 x w = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R B K E k I u i y K I L L C v Y B T S y T 6 a Q d O n k w c y M t I X 6 N W / 0 A / 8 S d O 3 H r F z h p s j C t B y 4 c z r m X e + 9 x Q s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D C J B a J s E P B A 9 B 0 v K m U / b w I D T X i g o 9 h x O u 8 7 k J v W 7 T 1 R I F v g P M A u p 7 e G R z 1 x G M C h p U D 2 0 P A x j g n l 8 m z z G F t A p x C M 4 S 5 J B t W 4 0 j D n 0 Z W L m p I 5 y t A Y 1 r W w N A x J 5 1 A f C s Z R 9 0 w j B j r E A R j h N K l Y k a Y j J B I 9 o X 1 E f e 1 T a 8 f y F R D 9 R y l B 3 A 6 H K B 3 2 u / p 2 I s S f l z H N U Z 3 q w X P R S 8 T + v H 4 F 7 Z c f M D y O g P s k W u R H X I d D T P P Q h E 5 Q A n y m C i W D q V p 2 M s c A E V G q F L Y 5 X + C G e Z q c X t E x S 2 Z m L S S 2 T z n n D N B r m / U W 9 e Z 2 n W E Z H 6 B i d I h N d o i a 6 Q y 3 U R g Q 9 o x f 0 i t 6 0 d + 1 T + 9 K + s 9 a S l s 8 c o A K 0 n 1 / g J K 2 u < / l a t e x i t > (b) Examples in E gt- < l a t e x i t s h a 1 _ b a s e 6 4 = " s o x m G 4 A x s n f K l O 0 j + i s 8 0 E T T R F 8 = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R 3 F g S E X R Z F M F l B f u A J p b J d N I O n T y Y u Z G W E L / G r X 6 A f + L O n b j 1 C 5 w 0 W Z j W A x c O 5 9 z L v f c 4 I W c S D O N D K 6 2 s r q 1 v l D c r W 9 s 7 u 3 v V 2 n 5 H B p E g t E 0 C H o i e g y X l z K d t Y M B p L x Q U e w 6 n X W d y k / r d J y o k C / w H m I X U 9 v D I Z y 4 j G J Q 0 q B 5 a H o Y x w T y + T R 5 j C + g U 4 h G c J c m g W j c a x h z 6 M j F z U k c 5 W o O a V r a G A Y k 8 6 g P h W M q + a Y R g x 1 g A I 5 w m F S u S N M R k g k e 0 r 6 i P P S r t e P 5 C o p 8 o Z a i 7 g V D l g z 5 X / 0 7 E 2 J N y 5 j m q M z 1 Y L n q p + J / X j 8 C 9 s m P m h x F Q n 2 S L 3 I j r E O h p H v q Q C U q A z x T B R D B 1 q 0 7 G W G A C K r X C F s c r / B B P s 9 M L W i a p 7 M z F p J Z J 5 7 x h G g 3 z / q L e v M 5 T L K M j d I x O k Y k u U R P d o R Z q I 4 K e 0 Q t 6 R W / a u / a p f W n f W W t J y 2 c O U A H a z y / j n q 2 w < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s o x m G 4 A x s n f K l O 0 j + i s 8 0 E T T R F 8 = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R 3 F g S E X R Z F M F l B f u A J p b J d N I O n T y Y u Z G W E L / G r X 6 A f + L O n b j 1 C 5 w 0 W Z j W A x c O 5 9 z L v f c 4 I W c S D O N D K 6 2 s r q 1 v l D c r W 9 s 7 u 3 v V 2 n 5 H B p E g t E 0 C H o i e g y X l z K d t Y M B p L x Q U e w 6 n X W d y k / r d J y o k C / w H m I X U 9 v D I Z y 4 j G J Q 0 q B 5 a H o Y x w T y + T R 5 j C + g U 4 h G c J c m g W j c a x h z 6 M j F z U k c 5 W o O a V r a G A Y k 8 6 g P h W M q + a Y R g x 1 g A I 5 w m F S u S N M R k g k e 0 r 6 i P P S r t e P 5 C o p 8 o Z a i 7 g V D l g z 5 X / 0 7 E 2 J N y 5 j m q M z 1 Y L n q p + J / X j 8 C 9 s m P m h x F Q n 2 S L 3 I j r E O h p H v q Q C U q A z x T B R D B 1 q 0 7 G W G A C K r X C F s c r / B B P s 9 M L W i a p 7 M z F p J Z J 5 7 x h G g 3 z / q L e v M 5 T L K M j d I x O k Y k u U R P d o R Z q I 4 K e 0 Q t 6 R W / a u / a p f W n f W W t J y 2 c O U A H a z y / j n q 2 w < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s o x m G 4 A x s n f K l O 0 j + i s 8 0 E T T R F 8 = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R 3 F g S E X R Z F M F l B f u A J p b J d N I O n T y Y u Z G W E L / G r X 6 A f + L O n b j 1 C 5 w 0 W Z j W A x c O 5 9 z L v f c 4 I W c S D O N D K 6 2 s r q 1 v l D c r W 9 s 7 u 3 v V 2 n 5 H B p E g t E 0 C H o i e g y X l z K d t Y M B p L x Q U e w 6 n X W d y k / r d J y o k C / w H m I X U 9 v D I Z y 4 j G J Q 0 q B 5 a H o Y x w T y + T R 5 j C + g U 4 h G c J c m g W j c a x h z 6 M j F z U k c 5 W o O a V r a G A Y k 8 6 g P h W M q + a Y R g x 1 g A I 5 w m F S u S N M R k g k e 0 r 6 i P P S r t e P 5 C o p 8 o Z a i 7 g V D l g z 5 X / 0 7 E 2 J N y 5 j m q M z 1 Y L n q p + J / X j 8 C 9 s m P m h x F Q n 2 S L 3 I j r E O h p H v q Q C U q A z x T B R D B 1 q 0 7 G W G A C K r X C F s c r / B B P s 9 M L W i a p 7 M z F p J Z J 5 7 x h G g 3 z / q L e v M 5 T L K M j d I x O k Y k u U R P d o R Z q I 4 K e 0 Q t 6 R W / a u / a p f W n f W W t J y 2 c O U A H a z y / j n q 2 w < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s o x m G 4 A x s n f K l O 0 j + i s 8 0 E T T R F 8 = " > A A A C O H i c b V D L S s N A F J 3 U V 6 2 v V j e C m 2 A R 3 F g S E X R Z F M F l B f u A J p b J d N I O n T y Y u Z G W E L / G r X 6 A f + L O n b j 1 C 5 w 0 W Z j W A x c O 5 9 z L v f c 4 I W c S D O N D K 6 2 s r q 1 v l D c r W 9 s 7 u 3 v V 2 n 5 H B p E g t E 0 C H o i e g y X l z K d t Y M B p L x Q U e w 6 n X W d y k / r d J y o k C / w H m I X U 9 v D I Z y 4 j G J Q 0 q B 5 a H o Y x w T y + T R 5 j C + g U 4 h G c J c m g W j c a x h z 6 M j F z U k c 5 W o O a V r a G A Y k 8 6 g P h W M q + a Y R g x 1 g A I 5 w m F S u S N M R k g k e 0 r 6 i P P S r t e P 5 C o p 8 o Z a i 7 g V D l g z 5 X / 0 7 E 2 J N y 5 j m q M z 1 Y L n q p + J / X j 8 C 9 s m P m h x F Q n 2 S L 3 Non-Maximum Suppression (NMS) is a standard post-processing step for detection tasks, to suppress redundant, overlapping predictions. We use it (i) to prevent duplicate predictions of vertices that are very close to each other, and (ii) to prune redundant edges. I j r E O h p H v q Q C U q A z x T B R D B 1 q 0 7 G W G A C K r X C F s c r / B B P s 9 M L W i a p 7 M z F p J Z J 5 7 x h G g 3 z / q L e v M 5 T L K M j d I x O k Y k u U R P d o R Z q I 4 K e 0 Q t 6 R W / a u / a p f W n f W W t J y 2 c O U A H a z y / j n q 2 w < / l a t e x i t > (c) Examples in E pred+ V Y = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R Z B E E o i g i 6 L I r i s Y B / Q x D K Z T N q h k w c z N 9 I S A n 6 N W / 0 A f 8 S t O 3 H r B z h p u j C t B w Y O 5 9 w 7 9 9 7 j R J x J M I w P r b S y u r a + U d 6 s b G 3 v 7 O 5 V a / s d G c a C 0 D Y J e S h 6 D p a U s 4 C 2 g Q G n v U h Q 7 D u c d p 3 x T e Z 3 n 6 i Q L A w e Y B p R 2 8 f D g H m M Y F D S o H p o + R h G B P P k N n 1 M L K A T S N Q H 7 l m a D q p 1 o 2 H M o C 8 T c 0 7 q a I 7 W o K a V L T c k s U 8 D I B x L 2 T e N C O w E C 2 C E 0 7 R i x Z J G m I z x k P Y V D b B P p Z 3 M j k j 1 E 6 W 4 u h c K 9 Q L Q Z + r f j g T 7 U k 5 9 R 1 V m K 8 t F L x P / 8 / o x e F d 2 w o I o B h q Q f J A X c x 1 C P U t E d 5 m g B P h U E U w E U 7 v q Z I Q F J q B y K 0 x x / M I N y S R f v a D l k s r O X E x q m X T O G 6 b R M O 8 v 6 s 3 r e Y p l d I S O 0 S k y 0 S V q o j v U Q m 1 E 0 D N 6 Q a / o T X v X P r U v 7 T s v L W n V Y = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R Z B E E o i g i 6 L I r i s Y B / Q x D K Z T N q h k w c z N 9 I S A n 6 N W / 0 A f 8 S t O 3 H r B z h p u j C t B w Y O 5 9 w 7 9 9 7 j R J x J M I w P r b S y u r a + U d 6 s b G 3 v 7 O 5 V a / s d G c a C 0 D Y J e S h 6 D p a U s 4 C 2 g Q G n v U h Q 7 D u c d p 3 x T e Z 3 n 6 i Q L A w e Y B p R 2 8 f D g H m M Y F D S o H p o + R h G B P P k N n 1 M L K A T S N Q H 7 l m a D q p 1 o 2 H M o C 8 T c 0 7 q a I 7 W o K a V L T c k s U 8 D I B x L 2 T e N C O w E C 2 C E 0 7 R i x Z J G m I z x k P Y V D b B P p Z 3 M j k j 1 E 6 W 4 u h c K 9 Q L Q Z + r f j g T 7 U k 5 9 R 1 V m K 8 t F L x P / 8 / o x e F d 2 w o I o B h q Q f J A X c x 1 C P U t E d 5 m g B P h U E U w E U 7 v q Z I Q F J q B y K 0 x x / M I N y S R f v a D l k s r O X E x q m X T O G 6 b R M O 8 v 6 s 3 r e Y p l d I S O 0 S k y 0 S V q o j v U Q m 1 E 0 D N 6 Q a / o T X v X P r U v 7 T s v L W n V Y = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R Z B E E o i g i 6 L I r i s Y B / Q x D K Z T N q h k w c z N 9 I S A n 6 N W / 0 A f 8 S t O 3 H r B z h p u j C t B w Y O 5 9 w 7 9 9 7 j R J x J M I w P r b S y u r a + U d 6 s b G 3 v 7 O 5 V a / s d G c a C 0 D Y J e S h 6 D p a U s 4 C 2 g Q G n v U h Q 7 D u c d p 3 x T e Z 3 n 6 i Q L A w e Y B p R 2 8 f D g H m M Y F D S o H p o + R h G B P P k N n 1 M L K A T S N Q H 7 l m a D q p 1 o 2 H M o C 8 T c 0 7 q a I 7 W o K a V L T c k s U 8 D I B x L 2 T e N C O w E C 2 C E 0 7 R i x Z J G m I z x k P Y V D b B P p Z 3 M j k j 1 E 6 W 4 u h c K 9 Q L Q Z + r f j g T 7 U k 5 9 R 1 V m K 8 t F L x P / 8 / o x e F d 2 w o I o B h q Q f J A X c x 1 C P U t E d 5 m g B P h U E U w E U 7 v q Z I Q F J q B y K 0 x x / M I N y S R f v a D l k s r O X E x q m X T O G s i L u Q 6 h n i W i u 0 x Q A n y q C C a C q V 1 1 M s I C E 1 C 5 F a Y 4 f u G G Z J K v X t B y S W V n L i s i L u Q 6 h n i W i u 0 x Q A n y q C C a C q V 1 1 M s I C E 1 C 5 F a Y 4 f u G G Z J K v X t B y S W V n L i s i L u Q 6 h n i W i u 0 x Q A n y q C C a C q V 1 1 M s I C E 1 C 5 F a Y 4 f u G G Z J K v X t B y S W V n L i s i L u Q 6 h n i W i u 0 x Q A n y q C C a C q V 1 1 M s I C E 1 C 5 F a Y 4 f u G G Z J K v X t B y S W V n L i

4. EXPERIMENTS

We could not find any publicly available dataset of point clouds with polyhedral wireframe annotations. Thus, we collect our own synthetic datasets: a subset from ABC (Koch et al., 2019) and a Furniture dataset. ABC is a collection of one million high-quality CAD models of mechanical parts with explicitly parameterised curves, surfaces and various geometric features, from which ground truth wireframes can be generated. We select a subset of models with straight edges. It consists of 3,000 samples, which we randomly split into 2,000 for training, 500 for validation, and 500 for testing. For the furniture dataset we have collected 250 models from Google 3DWarehouse for the categories bed, table, chair/sofa, stairs, monitor and render ground truth wireframes. Examples are shown in the supplementary material. To generate point clouds, we use the virtual scanner of Wang et al. (2017) . We place 14 virtual cameras on the face centres of the truncated bounding cubes of each object, uniformly shoot 16,000 parallel rays towards the object from each direction, and compute their intersections with the surfaces. Finally, we jitter all points by adding Gaussian noise N (0, 0.01), truncated to [-0.02, 0.02]. The number N of points varies between ≈ 100k -200k for different virtually scanned point clouds. For details, see the supplementary material.

4.1. EVALUATION MEASURES

We first separately evaluate the vertex detection and edge detection performance of our architecture, then assess the reconstructed wireframes in terms of structural average precision and of a newly designed wireframe edit distance. Mean Average precision mAP v for vertices. To generate high quality wireframes, the predicted vertex positions must be accurate. This metric evaluates the geometric quality of vertices, regardless of edge links. A predicted vertex is considered a true positive if the distance to the nearest ground truth vertex is below a threshold η v . For a given η v we generate precision-recall curves by varying the detection threshold. Average precision (AP v ) is defined as the area under this curve, mAP v is averaged over different thresholds η v . Point-wise precision and recall for edges. We do not have a baseline that directly extracts 3D wireframe edges. Rather, existing methods label individual input points as "lying on an edge", and produce corresponding precision-recall curves (Hackel et al., 2016; Yu et al., 2018) . To use this metric, we label a 3D point as "edge point" if its distance to the nearest wireframe edge is below the average point-to-point distance. As for AP v , we can now check whether a predicted "edge point" is correct up to η e , and compute AP e and mean average precision (mAP e ) over different η e . Structural average precision for wireframes. The above two evaluation metrics looks at vertices and edges separately, in terms of geometry. In order to comprehensively evaluate the generated wireframe, we also need to take into account the connectivity. The structural average precision (sAP, Zhou et al., 2019a ) metric, inspired by mean average precision, can be used to assess the overall prediction. This metric checks if the two end points (ṽ i , ṽj ) of a detected edge both match the end points (v k , v l ) of a ground truth edge up to a threshold η w , i.e., min v k ∈V ṽi -v k 2 + min v l ∈V ṽj -v l 2 < η w , (v k , v l ) ∈ E. We report sAP for different η w and mean sAP (msAP). Wireframe edit distance (WED). In order to topologically compare wireframes, we have designed a quality measure based on edit distance. We consider wireframes as geometric graphs drawn in 3D space. Distance measures commonly used in computational geometry and graph theory, such as Hausdorff distance and Frechet distance, resistance distance, or geodesic distance, do not seem to apply to our situation. Inspired by Cheong et al. (2009) , we propose a wireframe edit distance (WED). It is a variant of the graph edit distance (GED), which measures how many elementary graph edit operators (vertex insertion/deletion, edge insertion/deletion, etc.) are needed to transform one graph to another. Instead of allowing arbitrary sequences of operations, which makes computing the GED NP-hard, we fix their order, see Fig. 3: (1) vertex matching; (2) vertex translation; (3) vertex insertion; (4) edge deletion; (5) edge insertion. Let the ground truth wireframe W gt = (V gt , E gt ) and the predicted wireframe W o = (V o , E o ). First, we match each vertex v o i in W o to the nearest ground truth vertex v gt u in W gt , then translate them at a cost of c v • v o i v gt u . New vertices are inserted where a ground truth vertex has no match (at no cost, as cost will accrue during edge insertion). After all vertex operations, we perform edge deletion and insertion based on ground truth edges at a cost of c e • e , where e is the length of deleted/inserted edges. The final WED is the sum of all cost terms.

4.2. RESULTS AND COMPARISONS

Vertex detection. We validate the PC2WF vertex detection module and compare to a Harris3D baseline (with subsequent NMS to guarantee a fair comparison) on the ABC dataset, Tab. 1. Exemplary visual results and precision-recall curves are shown in Fig. 4 and Fig. 5 . Harris3D often fails to accurately localise a corner point, and on certain thin structures it returns many false positives. The patch size M used in our vertex detector has an impact on performance. Patch sizes between 20 and 50 achieve the best performance, where the precision stays at a high level over a large recall range. Patch size 1 leads to low precision (although still better than Harris3D), confirming that per-point features, even when computed from a larger receptive field, are affected by noise and insufficient to find corners with acceptable false positive ratio. We refer the interested reader to the supplementary material for a more exhaustive ablation study. Edge prediction. Due to the limited number of methods able to extract wireframes from 3D point clouds, we compare PC2WF to recent algorithms that detect edge points, such as FREE (Bazazian et al., 2015) , ECD (Ahmed et al., 2018) , FEEM (Xia & Wang, 2017) , Contour Detector (Hackel et al., 2016) and EC-Net (Yu et al., 2018) . We also compare with Polyfit (Nan & Wonka, 2017) , which is the only method able to reconstruct a vectorial representation similar to ours from a point cloud. Since Polyfit extracts planar surfaces of the object, we must convert that representation into a wireframe, which amounts to identifying the plane intersections. Precision-recall curves for all methods are shown in Although all predicted vertices need some editing, the WED v caused by those edits is small (0.2427), meaning that the predictions are accurate. On the contrary, the 2.6 edges that need to be edited on average increase WED e a lot more (1.4142) and dominate the WED, i.e., the inserted/deleted edges have significant length.

5. CONCLUSION

We have proposed PC2WF, an end-to-end trainable deep architecture to extract a vectorised wireframe from a raw 3D point cloud. The method achieves very promising results for man-made, polyhedral objects, going one step further than low-level corner or edge detectors, while at the same time outperforming them on the isolated vertex and edge detection tasks. We see our method as one further step from robust, but redundant low-level vision to compact, editable 3D geometry. As future work we aim to address the biggest limitation of our current method which is the inability to handle wireframes with strongly curved edges.



3 8 b p S u 8 t S L M A Z n M M l u H A D N b i H B j S B g o Q 3 e I c P 9 I m + 0 D e a p 6 0 b K J s 5 h R z Q z y 9 m H K Z C < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 B h c e f w i h 3 r i d U c G w v s T b X F j W 7 0 = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i y e J Y B 6 Y L G F 2 M k m G z M 4 u M 7 N i W P I X X v U D / B p v k q t f 4 i S 7 B z e x o K G o 6 q a 7 y 4 8 E 1 8 Z x 5 m h j c 2 t 7 Z 7 e w V 9 w / O D w 6 L p

3 8 b p S u 8 t S L M A Z n M M l u H A D N b i H B j S B g o Q 3 e I c P 9 I m + 0 D e a p 6 0 b K J s 5 h R z Q z y 9 m H K Z C < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 B h c e f w i h 3 r i d U c G w v s T b X F j W 7 0 = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i y e J Y B 6 Y L G F 2 M k m G z M 4 u M 7 N i W P I X X v U D / B p v k q t f 4 i S 7 B z e x o K G o 6 q a 7 y 4 8 E 1 8 Z x 5 m h j c 2 t 7 Z 7 e w V 9 w / O D w 6 L p

3 8 b p S u 8 t S L M A Z n M M l u H A D N b i H B j S B g o Q 3 e I c P 9 I m + 0 D e a p 6 0 b K J s 5 h R z Q z y 9 m H K Z C < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 B h c e f w i h 3 r i d U c G w v s T b X F j W 7 0 = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i y e J Y B 6 Y L G F 2 M k m G z M 4 u M 7 N i W P I X X v U D / B p v k q t f 4 i S 7 B z e x o K G o 6 q a 7 y 4 8 E 1 8 Z x 5 m h j c 2 t 7 Z 7 e w V 9 w / O D w 6 L p

3 8 b p S u 8 t S L M A Z n M M l u H A D N b i H B j S B g o Q 3 e I c P 9 I m + 0 D e a p 6 0 b K J s 5 h R z Q z y 9 m H K Z C < / l a t e x i t > M ⇥ C < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5

a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > PatchNet PatchNet PatchNet ... < l a t e x i t s h a 1 _ b a s e 6 4 = " t m u b b y

5 O 6 p C j O a i h c n 8 Y 0 S R k 0 l B B t O 5 5 b m z 8 l C j D q W D z S j / R L C Z 0 S s a s Z 6 k k I d N + u r x 1 j k + t M s S j S N m S B i / V 3 x M p C b W e h Y H t D I m Z 6 F V v I f 7 n 9 R I z u v J T L u P E M E m z R a N E Y B P h x e N 4 y B W j R s w s I V R x e y u m E 6 I I N T a e w p Y g L P y Q P m W n F 7 R M s t l 5 q 0 n 9 J e 1 z x 3 M d 7 + 6 i 3 r j O U y z D M Z z A G X h w C Q 2 4 h S a 0 g M I Y n u E F X t E b e k c f 6 D N r L a F 8 5 g g K Q N 8 / r K O j T Q = = < / l a t e x i t > t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o

x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o

x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o

x + t K 7 S 5 L s Q B n c A 6 X 4 M I N 1 O A e G t A E C h L e 4 B 0 + 0 C f 6 Q t 9 o n r Z u o G z m F H J A P 7 9 k W K Z B < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " i I H I T s B c 7 s L G t 3 G 3 6 y B q l N 4 7 r C A = " > A A A C K H i c b V D L S g N B E O z x G e M r 0 a O X w S B 4 C r s i 6 D G Y i x c h g n l g s o T Z y S Q Z M j u 7 z M y K Y c l f e N U P 8 G u 8 S a 5 + i Z P s H t z E g o a i q p v u L j 8 S X B v H m a O N z a 3 t n d 3 C X n H / 4 P D o

Figure 1: Illustration of our wireframe modelling architecture.

Fig. 2 visually depicts the different edge sets. To train the edge detector we randomly sample positive examples from E + = E gt+ ∪ E pred+ and negative examples from E -= E gt-∪ E pred-. As loss function L edge we use the balanced binary cross entropy. During inference, we go through the fully connected graph of edge candidates connecting any two (predicted) vertices. Candidates with a too high average distance to the input points (i.e., not lying on any object surface) are discarded. All others are fed into the edge detector for verification.

(a) Examples in E gt+ < l a t e x i t s h a 1 _ b a s e 6 4 = " + c b p E S l i 3 9 h 1

< l a t e x i t s h a 1 _ b a s e 6 4 = " B b l 7 m a 4 2 b f s g 9 / s 9 0 l k P l B H P 7

z n g N U g P b z C 6 d 2 r p I = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " B b l 7 m a 4 2 b f s g 9 / s 9 0 l k P l B H P 7

z n g N U g P b z C 6 d 2 r p I = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " B b l 7 m a 4 2 b f s g 9 / s 9 0 l k P l B H P 7

6 b R M O 8 v 6 s 3 r e Y p l d I S O 0 S k y 0 S V q o j v U Q m 1 E 0 D N 6 Q a / o T X v X P r U v 7 T s v L W n z n g N U g P b z C 6 d 2 r p I = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " B b l 7 m a 4 2 b f s g 9 / s 9 0 l k P l B H P 7V Y = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R Z B E E o i g i 6 L I r i s Y B / Q x D K Z T N q h k w c z N 9 I S A n 6 N W / 0 A f 8 S t O 3 H r B z h p u j C t B w Y O 5 9 w 7 9 9 7 j R J x J M I w P r b S y u r a + U d 6 s b G 3 v 7 O 5 V a / s d G c a C 0 D Y J e S h 6 D p a U s 4 C 2 g Q G n v U h Q 7 D u c d p 3 x T e Z 3 n 6 i Q L A w e Y B p R 2 8 f D g H m M Y F D S o H p o + R h G B P P k N n 1 M L K A T S N Q H 7 l m a D q p 1 o 2 H M o C 8 T c 0 7 q a I 7 W o K a V L T c k s U 8 D I B x L 2 T e N C O w E C 2 C E 0 7 R i x Z J G m I z x k P Y V D b B P p Z 3 M j k j 1 E 6 W 4 u h c K 9 Q L Q Z + r f j g T 7 U k 5 9 R 1 V m K 8 t F L x P / 8 / o x e F d 2 w o I o B h q Q f J A X c x 1 C P U t E d 5 m g B P h U E U w E U 7 v q Z I Q F J q B y K 0 x x / M I N y S R f v a D l k s r O X E x q m X T O G 6 b R M O 8 v 6 s 3 r e Y p l d I S O 0 S k y 0 S V q o j v U Q m 1 E 0 D N 6 Q a / o T X v X P r U v 7 T s v L W n z n g N U g P b z C 6 d 2 r p I = < / l a t e x i t > (d) Examples in E pred- < l a t e x i t s h a 1 _ b a s e 6 4 = " f a x m A k O 5 D Q U z 2 2 6 d q c W a X 9 L H n n E = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R b B j S U R Q Z d F E V x W s A 9 o Y p l M J u 3 Q y Y O Z G 2 k J A b / G r X 6 A P + L W n b j 1 A 5 w 0 X Z j W A w O H c + 6 d e + 9 x I s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D G N B a J u E P B Q 9 B 0 v K W U D b w I D T X i Q o 9 h 1 O u 8 7 4 J v O7 T 1 R I F g Y P M I 2 o 7 e N h w D x G M C h p U D 2 0 f A w j g n l y m z 4 m F t A J J O o D 9 y x N B 9 W 6 0 T B m 0 J e J O S d 1 N E d r U N P K l h u S 2 K c B E I 6 l 7 J t G B H a C B T D C a V q x Y k k j T M Z 4 S P u K B t i n 0 k 5 m R 6 T 6 i V J c 3 Q u F e g H o M / V v R 4 J 9 K a e + o y q z l e W i l 4 n / e f 0 Y v C s 7 Y U E U A w 1 I P

a 1 T D r n D d N o m P c X 9 e b 1 P M U y O k L H 6 B S Z 6 B I 1 0 R 1 q o T Y i 6 B m 9 o F f 0 p r 1 r n 9 q X 9 p 2 X l r R 5 z w E q Q P v 5 B a r w r p Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f ax m A k O 5 D Q U z 2 2 6 d q c W a X 9 L H n n E = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R b B j S U R Q Z d F E V x W s A 9 o Y p l M J u 3 Q y Y O Z G 2 k J A b / G r X 6 A P + L W n b j 1 A 5 w 0 X Z j W A w O H c + 6 d e + 9 x I s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D G N B a J u E P B Q 9 B 0 v K W U D b w I D T X i Q o 9 h 1 O u 8 7 4 J v O 7 T 1 R I F g Y P M I 2 o 7 e N h w D x G M C h p U D 2 0 f A w j g n l y m z 4 m F t A J J O o D 9 y x N B 9 W 6 0 T B m 0 J e J O S d 1 N E d r U N P K l h u S 2 K c B E I 6 l 7 J t G B H a C B T D C a V q x Y k k j T M Z 4 S P u K B t i n 0 k 5 m R 6 T 6 i V J c 3 Q u F e g H o M / V vR 4 J 9 K a e + o y q z l e W i l 4 n / e f 0 Y v C s 7 Y U E U A w 1 I P

a 1 T D r n D d N o m P c X 9 e b 1 P M U y O k L H 6 B S Z 6 B I 1 0 R 1 q o T Y i 6 B m 9 o F f 0 p r 1 r n 9 q X 9 p 2 X l r R 5 z w E q Q P v 5 B a r w r p Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f ax m A k O 5 D Q U z 2 2 6 d q c W a X 9 L H n n E = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R b B j S U R Q Z d F E V x W s A 9 o Y p l M J u 3 Q y Y O Z G 2 k J A b / G r X 6 A P + L W n b j 1 A 5 w 0 X Z j W A w O H c + 6 d e + 9 x I s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D G N B a J u E P B Q 9 B 0 v K W U D b w I D T X i Q o 9 h 1 O u 8 7 4 J v O 7 T 1 R I F g Y P M I 2 o 7 e N h w D x G M C h p U D 2 0 f A w j g n l y m z 4 m F t A J J O o D 9 y x N B 9 W 6 0 T B m 0 J e J O S d 1 N E d r U N P K l h u S 2 K c B E I 6 l 7 J t G B H a C B T D C a V q x Y k k j T M Z 4 S P u K B t i n 0 k 5 m R 6 T 6 i V J c 3 Q u F e g H o M / V vR 4 J 9 K a e + o y q z l e W i l 4 n / e f 0 Y v C s 7 Y U E U A w 1 I P

a 1 T D r n D d N o m P c X 9 e b 1 P M U y O k L H 6 B S Z 6 B I 1 0 R 1 q o T Y i 6 B m 9 o F f 0 p r 1 r n 9 q X 9 p 2 X l r R 5 z w E q Q P v 5 B a r w r p Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f ax m A k O 5 D Q U z 2 2 6 d q c W a X 9 L H n n E = " > A A A C O n i c b V D L S s N A F J 3 U V 6 2 v V n e 6 C R b B j S U R Q Z d F E V x W s A 9 o Y p l M J u 3 Q y Y O Z G 2 k J A b / G r X 6 A P + L W n b j 1 A 5 w 0 X Z j W A w O H c + 6 d e + 9 x I s 4 k G M a H V l p Z X V v f K G 9 W t r Z 3 d v e q t f 2 O D G N B a J u E P B Q 9 B 0 v K W U D b w I D T X i Q o 9 h 1 O u 8 7 4 J v O 7 T 1 R I F g Y P M I 2 o 7 e N h w D x G M C h p U D 2 0 f A w j g n l y m z 4 m F t A J J O o D 9 y x N B 9 W 6 0 T B m 0 J e J O S d 1 N E d r U N P K l h u S 2 K c B E I 6 l 7 J t G B H a C B T D C a V q x Y k k j T M Z 4 S P u K B t i n 0 k 5 m R 6 T 6 i V J c 3 Q u F e g H o M / V vR 4 J 9 K a e + o y q z l e W i l 4 n / e f 0 Y v C s 7 Y U E U A w 1 I P

Figure 2: Positive and negative samples for edge verification during training. (a) Positive edge samples (green lines) between ground truth vertices (black dots). (b) "spurious" (left) and "inaccurate" (right) connections between ground truth vertices. (c) positive samples (blue lines) between predicted vertices (blue dots). (d) "wrong links" (left) and "near misses" (right) between predicted vertices.

vertex translation/insertion (c) edited vertices (d) edge deletion/ insertion (e) ground truth

Figure 3: Illustration of WED. (a) First match vertices to (e) ground truth; (b) move matched vertices, insert missing ones and compute the cost; (d) delete and insert edges and compute the cost.

Figure 4: Vertex detection results of Harris3D (left) and ours (right) on the ABC dataset.

Visual results are shown in Fig.8. PC2WF outperforms all baselines based on hand-crafted features like FREE, ECD, FEEM and Contour Detector, as well as the deep learning method EC-Net, across all quality measures (Tab. 2). A visual comparison between the ground truth, PC2WF, EC-NetYu et al. (2018), and Polyfit is shown in Fig.9. One can clearly see the advantage of predicting vector edges rather than "edge points", as they lead to a much sharper result. The inevitable blur of edge points across the actual contour line, and their irregular density, make vectorisation a non-trivial problem. Polyfit and PC2WF show comparable performance for this specific point cloud. PC2WF failed to localise some corners and missed some small edges, whereas Polyfit failed to reconstruct the lower part of the armchair.Wireframe Reconstruction. We visualise PC2WF's output wireframes in Fig.10. Structural precision-recall curves with various patch sizes are shown in Fig.7. For additional examples of output predictions, including cases with curved edges, please refer to the supplementary material. The results are aligned with the ones of the vertex module. Moderate patch sizes between 20 and 50 achieve the best performance. Tab. 3 shows the structural average precision with patch size 50, which exceeds the one of Polyfit. The results in terms of the wireframe edit distance are shown in Tab. 4 for PC2WF and Polyfit. The average number of predicted vertices (21.7) and ground truth vertices (20.1) per object are close, such that only few vertex insertions are needed in most cases.

Figure 5: Vertex prediction accuracy on ABC dataset.

Figure 8: Comparison with edge point detection algorithms.

Figure 9: Visual comparison of graph structures. Vertices are orange and edges are blue.

Figure 10: Wireframe reconstruction results on ABC (a)(b)(c) and furniture (d)(e)(f) datasets. Top: input raw point clouds. Middle: predicted wireframe. Bottom: ground truth.

Vertex detection results.

Edge prediction results on the ABC subset and furniture dataset.

sAP evaluation on two datasets. Method sAP 0.03 sAP 0.05 sAP 0.07 msAP sAP 0.03 sAP 0.05 sAP 0.07 msAP

Wireframe edit distance (WED) on ABC dataset.

