We're currently using density functions to classify and as a mechanism to eliminate noise and also trend functions to separate out the ground. These are primarily useful as a pre-process to automatic vectorisation, as they allow you to cull the input depending on what you're trying to extract. There is a ton of academic material out there on this is you do a search,
Point cloud classification
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Point cloud classification
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Point cloud classification
Kind of agree, but what I think you'll find longer term is ML applied to point cloud data too rather than just images. Image decomposition is basically searching for features (e.g. corners) in an image and matching sets of features to known objects after correcting for pose. In many ways this is even easier with point clouds but just needs a different starting point, and no doubt the best results will be achieved combining imaging and scan data. AFAIK, the tree recognition stuff is more voxel based at present, but again, this can be combined with ML.jamesworrell wrote: ↑Thu May 17, 2018 2:02 amMy 2c .. best systems will use images (far easier to train ML systems!) and _optionally_ laser for recognition. Classified mesh is probably the winner longer term. Once you start playing with mesh .. you don't want to go back!
I reckon the ideal output will include meshes for general surfaces, strings or alignments with smart templates for linear features (also possibly meshed as well), and meshed BIM objects for discrete objects. Reading the tea leaves a bit, but I reckon point clouds as we currently use them will be a largely ignored interim product within the next ten years.
-
- V.I.P Member
- Posts: 537
- Joined: Mon Jun 16, 2014 1:45 pm
- 9
- Full Name: James Worrell
- Company Details: Bennett and Francis
- Company Position Title: Director
- Country: Australia
- Linkedin Profile: Yes
- Location: Brisbane, Queensland, Australia
- Has thanked: 14 times
- Been thanked: 87 times
- Contact:
Re: Point cloud classification
Teaching an algorithm to recognise a chair (or more likely part of chair) from an image is a "piece of cake" and has been done.
If you have a dozen images of the chair's surrounds, each part of the chair from each coordinated picture can create an outline. Intersect the outlines projected from the image camera positions and you have your chair.
You can "cut" up the point cloud this way if you have one - basically a region grow chair. Or create a point cloud from the images.
Machine learning is a little different - you aren't looking for geometry - it is far more esoteric. Machine learning for images can recognise a cat for example after being trained - even to the point of ignoring dogs.
The extra "texture" .. colour .. etc from an image is a much richer data set for a machine learning algorithm to work with, well beyond simple geometric matching. I suspect it would be orders of magnitude faster as well.
First hit on google .. was google .. ironically - search was "machine learning images" .. heads to TensorFlow.
Images would also provide serial numbers .. model numbers .. brands - you can read it is a Fisher and Paykel dishwasher off the thing.
You can also get "condition" .. So SiteSee are doing all of this already in the telecom / mobile tower domain - the images give them rust detection, antenna types etc.
If you have a dozen images of the chair's surrounds, each part of the chair from each coordinated picture can create an outline. Intersect the outlines projected from the image camera positions and you have your chair.
You can "cut" up the point cloud this way if you have one - basically a region grow chair. Or create a point cloud from the images.
Machine learning is a little different - you aren't looking for geometry - it is far more esoteric. Machine learning for images can recognise a cat for example after being trained - even to the point of ignoring dogs.
The extra "texture" .. colour .. etc from an image is a much richer data set for a machine learning algorithm to work with, well beyond simple geometric matching. I suspect it would be orders of magnitude faster as well.
First hit on google .. was google .. ironically - search was "machine learning images" .. heads to TensorFlow.
Images would also provide serial numbers .. model numbers .. brands - you can read it is a Fisher and Paykel dishwasher off the thing.
You can also get "condition" .. So SiteSee are doing all of this already in the telecom / mobile tower domain - the images give them rust detection, antenna types etc.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Point cloud classification
Still very new to machine learning myself, first google hit for me takes me to sites on ML I've already visited, such as this. Bit of learning going on by google in the background there too From what I've looked at, image based ML does start with feature vectors, in much the same way as cloud to cloud registration. I reckon that it has potential to play well with point cloud classification going forward but wonder whether dealing with huge numbers of potential objects and huge datasets will become a stumbling point in terms of performance. Given that much point cloud data also has matching high resolution imagery, it seems like a natural progression.jamesworrell wrote: ↑Thu May 17, 2018 8:09 amFirst hit on google .. was google .. ironically - search was "machine learning images" .. heads to TensorFlow.
- infrared_sight
- I have made 20-30 posts
- Posts: 28
- Joined: Thu Aug 17, 2017 12:18 pm
- 6
- Full Name: Charles Thomson
- Company Details: Correvate - Developers of Vercator
- Company Position Title: Chief Product Manager
- Country: UK
- Linkedin Profile: Yes
- Location: London
- Has thanked: 7 times
- Been thanked: 6 times
- Contact:
Re: Point cloud classification
Machine Learning can be as much an art as a science. We get about 90% success with recovery of certain objects: lighting units, chairs, etc with our approach.
If interested you can see a small result of this approach here:
youtu.be/hQ2RbLq9W10
If interested you can see a small result of this approach here:
youtu.be/hQ2RbLq9W10
Last edited by infrared_sight on Wed Jun 05, 2019 3:17 pm, edited 1 time in total.
Charles Thomson, Chief Product Manager
Vercator - fast, automatic point cloud processing:
Registration, Classification, Conversion
Free trial at vercator.com
Vercator - fast, automatic point cloud processing:
Registration, Classification, Conversion
Free trial at vercator.com
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Point cloud classification
Nice work! I see you're at Geo-business next week as are we, I'll drop over to say Hi and have a look at Vercator at some point,infrared_sight wrote: ↑Thu May 17, 2018 12:30 pm Machine Learning can be as much an art as a science. We get about 90% success with recovery of certain objects: lighting units, chairs, etc with our approach.
If interested you can see a small result of this approach here:
youtu.be/hQ2RbLq9W10
Shane
- infrared_sight
- I have made 20-30 posts
- Posts: 28
- Joined: Thu Aug 17, 2017 12:18 pm
- 6
- Full Name: Charles Thomson
- Company Details: Correvate - Developers of Vercator
- Company Position Title: Chief Product Manager
- Country: UK
- Linkedin Profile: Yes
- Location: London
- Has thanked: 7 times
- Been thanked: 6 times
- Contact:
Re: Point cloud classification
Thanks Shane, look forward to it.
Charles Thomson, Chief Product Manager
Vercator - fast, automatic point cloud processing:
Registration, Classification, Conversion
Free trial at vercator.com
Vercator - fast, automatic point cloud processing:
Registration, Classification, Conversion
Free trial at vercator.com
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
-
- V.I.P Member
- Posts: 1037
- Joined: Tue Mar 29, 2011 7:39 pm
- 13
- Full Name: Scott Page
- Company Details: Scott Page Design- Architectural service
- Company Position Title: Owner
- Country: USA
- Linkedin Profile: No
- Location: Berkeley, CA USA
- Has thanked: 206 times
- Been thanked: 78 times
- Contact:
Re: Point cloud classification
I stumbled upon a couple of interesting academic papers from the same authors:
Contour detection in unstructured 3D point clouds
https://www.ethz.ch/content/dam/ethz/sp ... pr2016.pdf
Excerpt:
"Another problem comes from the machine learning backbone of our method. Regions that have a large amount of geometric structure, but have not been seen in the (neg- ative) training data, tend to get high contour scores and induce false positives."
Conclusion
"We have proposed a novel approach to detect contours in unorganized 3D point clouds, which is able to handle data from complex outdoor environments. In our experi- ments the proposed detector work robustly across a range of laser-scanning datasets, whereas a standard Canny-like baseline badly fails. Our method is also computationally efficient enough for practical application: processing ten million points only takes a couple of minutes an a single desktop PC."
...
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016
FAST SEMANTIC SEGMENTATION OF 3D POINT CLOUDS WITH STRONGLY VARYING DENSITY
https://www.isprs-ann-photogramm-remote ... 7-2016.pdf
"We describe an effective and efficient method for point-wise semantic classification of 3D point clouds. The method can handle unstructured and inhomogeneous point clouds such as those derived from static terrestrial LiDAR or photogammetric reconstruction; and it is computationally efficient, making it possible to process point clouds with many millions of points in a matter of minutes. The key issue, both to cope with strong variations in point density and to bring down computation time, turns out to be careful handling of neighborhood relations. By choosing appropriate definitions of a point’s (multi-scale) neighborhood, we obtain a feature set that is both expressive and fast to compute. We evaluate our classification method both on benchmark data from a mobile mapping platform and on a variety of large, terrestrial laser scans with greatly varying point density. The proposed feature set outperforms the state of the art with respect to per-point classification accuracy, while at the same time being much faster to compute."
Large-Scale Point Cloud Classification Benchmark -related website (worth a look)
http://www.semantic3d.net
Contour detection in unstructured 3D point clouds
https://www.ethz.ch/content/dam/ethz/sp ... pr2016.pdf
Excerpt:
"Another problem comes from the machine learning backbone of our method. Regions that have a large amount of geometric structure, but have not been seen in the (neg- ative) training data, tend to get high contour scores and induce false positives."
Conclusion
"We have proposed a novel approach to detect contours in unorganized 3D point clouds, which is able to handle data from complex outdoor environments. In our experi- ments the proposed detector work robustly across a range of laser-scanning datasets, whereas a standard Canny-like baseline badly fails. Our method is also computationally efficient enough for practical application: processing ten million points only takes a couple of minutes an a single desktop PC."
...
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016
FAST SEMANTIC SEGMENTATION OF 3D POINT CLOUDS WITH STRONGLY VARYING DENSITY
https://www.isprs-ann-photogramm-remote ... 7-2016.pdf
"We describe an effective and efficient method for point-wise semantic classification of 3D point clouds. The method can handle unstructured and inhomogeneous point clouds such as those derived from static terrestrial LiDAR or photogammetric reconstruction; and it is computationally efficient, making it possible to process point clouds with many millions of points in a matter of minutes. The key issue, both to cope with strong variations in point density and to bring down computation time, turns out to be careful handling of neighborhood relations. By choosing appropriate definitions of a point’s (multi-scale) neighborhood, we obtain a feature set that is both expressive and fast to compute. We evaluate our classification method both on benchmark data from a mobile mapping platform and on a variety of large, terrestrial laser scans with greatly varying point density. The proposed feature set outperforms the state of the art with respect to per-point classification accuracy, while at the same time being much faster to compute."
Large-Scale Point Cloud Classification Benchmark -related website (worth a look)
http://www.semantic3d.net
-
- V.I.P Member
- Posts: 136
- Joined: Thu Mar 16, 2017 3:29 pm
- 7
- Full Name: Neerav Mehta
- Company Details: Indoor Intelligence
- Company Position Title: CTO
- Country: USA
- Linkedin Profile: No
- Has thanked: 1 time
- Been thanked: 6 times
Re: Point cloud classification
True, but in most cases, the algorithm returns only the bounding box of the chair. Teaching an algorithm to identify the exact pixels which belong to a chair is much more difficult and as of now, won't produce as good a result.Teaching an algorithm to recognise a chair (or more likely part of chair) from an image is a "piece of cake" and has been done.
I agree. By just using images and not point cloud, we are withholding depth information from the machine learning system and as a result, the output will be less accurate. Why not provide depth information if we already have it and let machine learning system decide how to use it?Kind of agree, but what I think you'll find longer term is ML applied to point cloud data too rather than just images.
The reason image based machine learning system are more widespread now is because one, there are way more images to train on than point clouds and two, because of the extra depth dimension, running machine learning on point cloud is much slower than running it on images.
BTW if you are interested, we have developed a Revit add-in for point cloud classification within point cloud. Here's the link if you want to give it a shot: http://rep3d.com. In addition to classifying the point that belong to the object of interest, the add-in places the Revit family in the correct orientation.