We dream a magic button for 3-D point cloud processing

3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 183
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon » Mon Feb 04, 2019 4:11 pm

Hi Shane,

It’s my pleasure. Thanks a lot!

The line-segments of curves or the triangle-segments of surfaces are the necessary intermediate information for visualizing the analytic curves or surfaces, accepting a bounded loss of concavity/convexity.

What I like to say is that the meshes from measurement points are arbitrary and dispensable. What about if we could process the measurement points directly without meshing. It would be online, speedy and accurate.

Measurement points are not sampling points of analytic (i.e., perfect and errorless) curves and surfaces. They are sampling points of real unknown object curves and surfaces, subject to noises and errors.

The meshes are locally linearizing the measurement points, irreversibly destroying the more or the less the geometric, statistical and probabilistic information contained in the measurement points.

Please imagine 4 vertex points building 2 triangles. There are 2 sets of 2 triangles resulting in totally different normal and positions of triangles. Meshes using tetrahedral rather than triangular will not solve the problem but only increase the complexity of job.

Meshing the measurement points is a process of local linearization. Any linearization destroys valuable information collected during the measuring process.

We are tamed with conventions, convenient and tools available (OpenCV, MATLAB, PCL, etc.). The DL/ML/AI might be the next drying oasis. It’s the time to rethink the way of 3-D data processing.

Joon

User avatar
smacl
V.I.P Member
V.I.P Member
Posts: 175
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Been thanked: 1 time
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by smacl » Mon Feb 04, 2019 7:16 pm

Hi Joon,

While I agree to some extent, I think meshes are a very pragmatic way to solve quite a wide range of computational problems related to surface modelling. As the sensors we're using collect data at an ever increasing rate, e.g. the RTC360 is capturing 2 million points per second and HDR imagery to match, we need to reduce the raw data to what it accurately represents in a compact and usable format. So for example if I'm using a good laser scanner in a well controlled environment, I might get a point cloud with points that are accurate to ~3mm at 95% confidence. If I mesh those points and find that all points within a given area in the data set are co-planar to 3mm I can store the 3d polygon represented the concave hull of that planar area and delete all of the points in that polygon with no loss off accuracy. The same holds true for other geometric primitives such as spheres, cones, cylinders, etc... but doesn't work at all for other scenarios, e.g. an area of long grass or foliage. In this case we can mesh the foliage, look for low spikes which represent the ground, re-mesh just those points to get the low ground surface, and then re-mesh the remainder to get a top of foliage surface. As a tool, meshes are very useful beyond visualising data that allows us to group similar points that describe irregular features.

From an idealistic point of view, I agree that we should be using point clouds directly more often in place of meshes, but the problem as I see it is that they've become so large in recent years they're clogging up hard drives and network bandwidth for very many of the users. As such it is important that they're reduced to a representative subset which meets the accuracy requirements for the task in hand. For example, if I see a scan job from a fast terrestrial laser scanner that hasn't been cleaned up, I'd expect to see about 60% of the points within 5 metres of an instrument setup. Most of this data is redundant and adaptive meshes are a great way to clean it up.

Another area where meshes are computationally important is in combining HDR images with 3d measurement as they allow us to tessellate the images such that we can get an accurate 3d coordinate for every pixel. The applications of this go beyond visualization and allow us to combine technologies such as deep learning with laser scanning.

I think we really should be using all the tools at our disposal in combination, where meshing is just one more tool,

Shane

jamesworrell
V.I.P Member
V.I.P Member
Posts: 285
Joined: Mon Jun 16, 2014 1:45 pm
Full Name: James Worrell
Company Details: Bennett and Francis
Company Position Title: Director
Country: Australia
Linkedin Profile: Yes
Location: Brisbane, Queensland, Australia
Been thanked: 1 time
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by jamesworrell » Mon Feb 04, 2019 9:56 pm

Not that I have tried, I am not a researcher in machine learning / AI, but I suspect that spherical projections of point clouds from the scanner positions (eg an image) is perhaps the easiest data to throw at a machine learning algorithm for object recognition.

There are plenty of image recognition algorithms out there, so we train the ML to recognise windows, doors, columns, video cameras, fire sprinklers etc and this is done against an image. The projection would be a pixel per point, could be intensity and/or RGB.

This object outline - even partial - is projected back on to each cloud and the object can be followed / extracted / segmented from there with the additional depth information.

I can look at a snippet of point cloud and recognise an object from the 2D projection - ie the monitor - so it doesn't seem much of a leap to train the image recognition algorithm similar to how the human might recognise an object.

Totally unrelated - but the Blackmore FM Lidar sounds awesome for noise removal - if it is moving - nuke it.

Scott
V.I.P Member
V.I.P Member
Posts: 761
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Owner
Country: USA
Linkedin Profile: No
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Scott » Mon Feb 04, 2019 10:00 pm

This is an excellent discussion, but I had to refresh my understanding of some of the concepts with 'deep caffeine' and this article:
https://skymind.ai/wiki/ai-vs-machine-l ... p-learning

What I want to see is a semi-automated way to select the x,y,z points where room walls, floors and ceilings intersect (3 plane intersections), to dimension the space(s). We don't need the millions of points in a point cloud if we are to measure key building relationships. Yes, it's more complicated than that -we have wall/floor thickness, doors, windows, and simple openings to locate and dimension, but it's a start. Add the important associations: centerlines, datums, incline planes -it's complicated, yet doable, IMHO. What dimensions do we really need for others to build stuff from? Give AEC businesses what they really need.

john-newbegin
V.I.P Member
V.I.P Member
Posts: 116
Joined: Wed Apr 29, 2015 4:06 pm
Full Name: John Newbegin
Company Details: Travis County Sheriff
Company Position Title: Engineering Specialist
Country: USA
Linkedin Profile: Yes

Re: We dream a magic button for 3-D point cloud processing

Post by john-newbegin » Tue Feb 05, 2019 3:00 pm

Good point Scott, there are so many millions and billions of things we can measure but what about back to basics like you said. I think the minimalist approach is the way to tackle this problem. Are there any software that do what you say measure the intersection of 3 planes and dimension for you automatically? I Like the approach this thread is taking also seems like they are starting basic and then building upon that. It will be very interesting to see how this develops in the coming months/years.

Hope to see more discussion about this and other peoples point of view etc.

John

User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 183
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon » Fri Mar 22, 2019 10:42 am

3-D Augmented Reality - ARKit point cloud

Extraction of object geometries from ARKit point cloud under device motion.

Tools used:
- Apple iPad Pro (A1980)
- Apple ARKit 2.0
- CurvSurf FindSurface.


youtu.be/h4YVh2-3p9s

User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 183
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon » Tue Mar 26, 2019 10:29 am

3-D Measurement of Tree Branches - Apple ARKit

Extraction of cylinders of tree branches from ARKit point cloud under device motion.

Tools used:
- Apple iPad Pro (A1980)
- Apple ARKit 2.0
- CurvSurf FindSurface.


youtu.be/h9s8_lt1I64

yankoch
V.I.P Member
V.I.P Member
Posts: 392
Joined: Wed Oct 05, 2011 10:14 am
Full Name: Yan Koch
Company Details: TagLabs
Company Position Title: President
Country: France
Linkedin Profile: Yes

Re: We dream a magic button for 3-D point cloud processing

Post by yankoch » Tue Mar 26, 2019 10:44 am

You are doing a great work!
Kind regards,

Yan Koch
[email protected]

User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 183
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon » Tue Mar 26, 2019 1:57 pm

Tgl_Nan wrote:
Tue Mar 26, 2019 10:44 am
You are doing a great work!
Hi Yan,

Thank you for your praise. :like:

Joon

User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 183
Joined: Wed Aug 21, 2013 8:01 pm
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: CEO_CTO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon » Sat Mar 30, 2019 7:52 am

3-D Measurement of Cone Sculpture - Apple ARKit

Extraction of cone from ARKit point cloud under device motion.

Tools used:
- Apple iPad Pro (A1980)
- Apple ARKit 2.0
- CurvSurf FindSurface.


youtu.be/9Ka9j9cnIiw

Post Reply

Return to “Modeling Software”