Software for finding repeating items/patterns in point cloud

3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
neeravbm
V.I.P Member
V.I.P Member
Posts: 135
Joined: Thu Mar 16, 2017 3:29 pm
Full Name: Neerav Mehta
Company Details: Indoor Intelligence
Company Position Title: CTO
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 3 times

Software for finding repeating items/patterns in point cloud

Post by neeravbm »

Hi,

During SPAR3D last week, I heard a requirement for a software that would allow users to select one instance of an object or pattern in a point cloud and then ask the software to find all the instances of that object. In a plant, this could be used to find all the pumps. In a building, this could be used to find all the instances of water sprinklers, lamps or even patterns in building facade. According the user who wished for this software, it is useful because:

1) It is tedious and time-consuming to find all the instances of the object, especially if it's small.
2) There is a high chance user will miss out on some objects if he is finding them manually. This software will reduce this possibility.

From what I know, ClearEdge does this for finding doors and windows in building facade but is not general enough to be used for any object or pattern.

Do you think such a software will be useful to you? If yes, for finding which objects/patterns will you typically use it for?

Thank you.

Regards,
Neerav.
Neerav Mehta
CTO, Indoor Intelligence
Creators of http://scantobim.xyz and http://rep3d.com

jamesworrell
V.I.P Member
V.I.P Member
Posts: 344
Joined: Mon Jun 16, 2014 1:45 pm
Full Name: James Worrell
Company Details: Bennett and Francis
Company Position Title: Director
Country: Australia
Linkedin Profile: Yes
Location: Brisbane, Queensland, Australia
Been thanked: 16 times
Contact:

Re: Software for finding repeating items/patterns in point c

Post by jamesworrell »

Would want to take this one step and apply machine learning to teach the algorithm what a window is, say as opposed to matching the exact window - ie I would want it to work for all windows.

Stamford are doing this .. should be able to goggle it. They taught it to recognise chairs, filing cabinets, walls etc. Google's cars do this, just need to train it for the built environment. The cars classify objects as bicycles, pedestrians etc based on the scan data.

Once trained, you could either "classify" the recognised mesh and move on, or model the geometry out.

neeravbm
V.I.P Member
V.I.P Member
Posts: 135
Joined: Thu Mar 16, 2017 3:29 pm
Full Name: Neerav Mehta
Company Details: Indoor Intelligence
Company Position Title: CTO
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 3 times

Re: Software for finding repeating items/patterns in point c

Post by neeravbm »

James, thanks for the reply! Other than windows, which elements would you want the software to recognize automatically?
Neerav Mehta
CTO, Indoor Intelligence
Creators of http://scantobim.xyz and http://rep3d.com

User avatar
sshemro
I have made 30-40 posts
I have made 30-40 posts
Posts: 38
Joined: Fri Mar 24, 2017 1:24 pm
Full Name: Skyler Shemro
Company Details: Rand Construction
Company Position Title: BIM Specialist
Country: United States
Linkedin Profile: Yes
Location: Alexandria, VA
Contact:

Re: Software for finding repeating items/patterns in point c

Post by sshemro »

neeravbm wrote:James, thanks for the reply! Other than windows, which elements would you want the software to recognize automatically?
Seems to me that it would be ideal if the software could be instructed what to recognize. In other words, if you could select a group of points or maybe three different groups, and call it "Sprinkler" and then have the software recognize all existences of points in that pattern within a certain tolerance. I don't know anything about the algorithm or coding that would drive a software like this, but it seems like it could be done.

For my personal purposes, I would like to be able to find all repeating instances in a cloud, sprinklers, ducts, pumps, valves, etc.

jamesworrell
V.I.P Member
V.I.P Member
Posts: 344
Joined: Mon Jun 16, 2014 1:45 pm
Full Name: James Worrell
Company Details: Bennett and Francis
Company Position Title: Director
Country: Australia
Linkedin Profile: Yes
Location: Brisbane, Queensland, Australia
Been thanked: 16 times
Contact:

Re: Software for finding repeating items/patterns in point c

Post by jamesworrell »

Machine learning needs you to think outside the box. You train the system - much like you would train a human of sorts.

Take a look at the Ted talk on How an autonomous car sees .. Google give the cars (last I heard) 3 million miles of training data per day to work through.

https://www.ted.com/talks/chris_urmson_ ... s_the_road

They can recognise a policeman waving .. a bicycle .. they have extensive 3D point cloud libraries they have used to train the system.

Built environment is no different - although I suspect the algorithm would do better with the photography - there are already well proven systems to find objects in images. Because you know where the images were taken - you can then use the images to classify the points - presto - an object is identified. This can be augmented with pure cloud recognition.

Getting enough points on a sprinkler is currently a tough ask - although a human modeller may recognise it as a sprinkler - so one might assume the machine learning algorithm can be trained to do the same - but where there is doubt, the machine could check the photography - much the same as a human might.

Brave new world and well on the way.
http://buildingparser.stanford.edu/method.html

jedfrechette
V.I.P Member
V.I.P Member
Posts: 917
Joined: Mon Jan 04, 2010 7:51 pm
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 6 times
Been thanked: 30 times
Contact:

Re: Software for finding repeating items/patterns in point c

Post by jedfrechette »

neeravbm wrote:Other than windows, which elements would you want the software to recognize automatically?
I would like it to be able to recognize any object that I can find a photo of via Google's Image Search.

I know how big of an ask that is. In the long term, I think there is great potential for point cloud classification via the sort of machine learning approaches commonly used in production for many image classification tasks. In the short-term though, I'm not sure about the general practicality outside of research labs and some other very narrow applications.

One of the reasons production image classification systems like Google Image Search are feasible is that they are based on a huge amount of training data and prior research on image classification. That baseline data and research is, in turn, the result of end users having access to countless tools for building and testing their own image classification processes using everything from simple manual identification to sophisticated multi-step statistical processes.

Similar tools simply do not exist in our field. It seems a bit like we're trying to jump from MS Photoviewer (Yeah I can view a point cloud!) to Google Image Search, but I'm not sure we can skip over all the hard work in-between.

Seeing companies like ClearEdge and Trimble at least thinking about classification is great, but how many different types of objects can their classifiers identify? How quickly can they add new identifiable objects and at that rate how long will take until we get to something like Google Image Search where nearly any object imaginable can be identified? I'm sure there is a limit to the number of different objects people might like to identify in point clouds but that number is almost certainly very large and the more people have access to point clouds the larger it is going to get.

Google Point Cloud Search would be great, but right now what I want are tools that make it easier to start building my own point cloud classification processes. That would be much more useful than another pretrained blackbox classifier that can only identify a few dozen or even a few hundred object types that I may or may not be interested in. Give users the tools we need to innovate and this industry will start progressing much more rapidly than it will be able to if development continues to be driven from the top down.
Jed

Scott
V.I.P Member
V.I.P Member
Posts: 862
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Owner
Country: USA
Linkedin Profile: No
Location: Berkeley, CA USA
Has thanked: 44 times
Been thanked: 16 times
Contact:

Re: Software for finding repeating items/patterns in point c

Post by Scott »

neeravbm wrote:James, thanks for the reply! Other than windows, which elements would you want the software to recognize automatically?
Dimensions are the hidden elements that I'd like to see automated -all of the dimensioning tasks that are done in CAD/BIM platforms and converted into smart chains of relationships between walls, floors, ceilings, door/window openings, center-to-center, etc. If point clouds are measurable, shouldn't we have better/smarter tools to access those important dimensions without having to model or trace them first? I do understand that this is a difficult thing to do, but it's not impossible.

neeravbm
V.I.P Member
V.I.P Member
Posts: 135
Joined: Thu Mar 16, 2017 3:29 pm
Full Name: Neerav Mehta
Company Details: Indoor Intelligence
Company Position Title: CTO
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 3 times

Re: Software for finding repeating items/patterns in point c

Post by neeravbm »

Skyler, thanks for the feedback! Ducts might be difficult to recognize since they are essentially planar without a lot of complex geometry, but recognizing pumps and valves should be possible. Recognizing sprinklers should be possible as long as it has enough points.

If you had this software which could be instructed what to recognize, how often will you use it? Do you think it will save you enough time in a month to pay for this software?
Neerav Mehta
CTO, Indoor Intelligence
Creators of http://scantobim.xyz and http://rep3d.com

neeravbm
V.I.P Member
V.I.P Member
Posts: 135
Joined: Thu Mar 16, 2017 3:29 pm
Full Name: Neerav Mehta
Company Details: Indoor Intelligence
Company Position Title: CTO
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 3 times

Re: Software for finding repeating items/patterns in point c

Post by neeravbm »

James and Jed, it was good and refreshing to hear your views on using machine learning in point cloud processing and classification!

James, you mentioned that we could use photography in addition to point clouds for classification. From what I understood, laser scanners give a colored point cloud as output. Can they provide separate photographs as output as well? If yes, are these 360 photographs or regular ones? With each photograph, do we also get the scanner position from where it was captured? Also I am very interested in knowing what you think of Jed's reply and my understanding of it below.

Jed, you suggested building a tool that will make it easier to start building your own point cloud classification processes. How do you visualize this will work? Here's what I am thinking. Please correct me if I am wrong. When you first start using the software, it will have no intelligence. You will have to start teaching it. So you'll select an object a few times in the point cloud and classify it as Object 1. This will be stored in your personal library now. You'll create many such objects from multiple point cloud files. Once the tool has been taught, any time you open a new point cloud file, it will find those objects for you automatically. Any machine learning system is going to have a few errors. So you'll need to validate the object that this tool finds. But in effect, you are creating a library of objects that you personally are interested in and that you personally come across in many of your different projects. Am I correct?
Neerav Mehta
CTO, Indoor Intelligence
Creators of http://scantobim.xyz and http://rep3d.com

neeravbm
V.I.P Member
V.I.P Member
Posts: 135
Joined: Thu Mar 16, 2017 3:29 pm
Full Name: Neerav Mehta
Company Details: Indoor Intelligence
Company Position Title: CTO
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 3 times

Re: Software for finding repeating items/patterns in point c

Post by neeravbm »

Scott, from what I understood, you want to measure dimensions within point cloud. For e.g. wall thickness, floor to ceiling height, etc. Am I correct? As I see it, the problem is that point cloud by itself doesn't know what a wall vs floor vs ceiling is. So there has to be a higher layer which segments and classifies point clouds and says that this group of points is a wall, this group of points is a floor, etc. Then you can measure distances between these higher order entities. As of now, I can't think of any way to directly measure distances between these higher order entities without actually creating these higher order entities. Please let me know if your reply was something totally different and I misunderstood it. Thanks!
Neerav Mehta
CTO, Indoor Intelligence
Creators of http://scantobim.xyz and http://rep3d.com

Post Reply

Return to “Modeling Software”