We dream a magic button for 3-D point cloud processing

aurivus|REVIT, 3D Studio MAX, 3D Coat, Lightwave, ZBrush, VRMesh, TopoGun, Blender, Rhino, Trimble Sketchup Etc.
Post Reply
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Carbix wrote: Fri Jun 12, 2020 9:06 am what if you use created a model of the space to help train the system. I'm thinking of how this could be used to AR in layers of a BIM. You would be able to see where conduit is without the need to GPR. (if it was scanned before poor)

This could also use an Ai training element like with RTX cards and super sampling for games.

The math that goes into this stuff makes my eyes hurt.
Hi Daniel,

CurvSurf is a small team specialized in 3-D point cloud processing.

Our product is FindSurface with the potential of a variety of applications. We are not developing a commercial application software, but providing FindSurface for a wide range of applications.

Currently, ML is out of the range we can touch. Of course, we or You will eventually combine FindSurface with ML.

Joon
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Visualizing Point Cloud - Apple iPad Pro LiDAR

The rear 3-D camera of Apple iPad Pro LiDAR projects about 500 laser pulses in predefined directions onto object surfaces.

The single-photon CMOS image sensor (CIS) is measuring the time elapses between the projection and the bounce-back of the laser pulses. The distances of object surface points are calculated by the Time-of-Flight of light.

The point distances in predefined directions (polar coordinate system) determine the xyz-coordinates (rectangular coordinate system) of the object surface points.

CurvSurf FindSurface will be extracting more than 1,000 ( or 2,000 ) times per second the ball sphere from the about 500 measurement points. It will be fast enough for a real-time full-frame (i.e. 60 FPS) object tracking and measurement.

The source codes of the App are available.


youtu.be/OkxHRRv6Joo
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Visualizing confidenceMap - Apple iPad Pro LiDAR


youtu.be/NgNXAICnk5c

Apple's advanced ML algorithm makes the 60 Hz ARDepthData including the depthMap and the confidenseMap, by processing the LiDAR output together with the frame images; Depth API for ARKit 4.

ARMeshAnchor is updated by processing constantly the last frames of the ARDepthData; Scene Geometry API for ARKit 3.5.

The confidenceMap reveals low confidences (reflectances) in larger distances than 4.5 m, or in shadow regions, or with inclined/dark/specular object surfaces.

ARConfidenceLevel: low - red, medium - dark, high - fogged.

The source codes of the App are available:
"Creating a Fog Effect Using Scene Depth"
https://developer.apple.com/documentati ... cene_depth

// Set to `true` to visualize confidence.
bool confidenceDebugVisualizationEnabled = false;
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Visualizing depthMap - Apple iPad Pro LiDAR

Tim Field's Twitter Video
ARKit 4 depth maps (256x192 px)

The 60 Hz ARDepthData including the depthMap and the confidenceMap has the resolution of 256x192 (aspect ration of 4:3).

The number of the laser pulses emitted from VCSEL (vertical cavity surface emitting laser) cells are 64 (8x8). A diffractive optical element (DOE) above the VCSELs multiplies the number with 3x3 yielding total 576 (8x8x3x3 = 24x24) pulses, projecting a dot pattern of equilateral triangles onto object surfaces viewed from the frame camera.

Because the aspect ratio of frame image is 4:3, only 24x18 of 24x24 depth points may be used for building ARDepthData of 256x192 resolution. The additional information of the color frame image interpolates the 24x18 to the 256x192 depth points, a magnification ratio of about 113 (10.7x10.7 = 49,152/432).

The max 49,152 depth points (point cloud) with individual confidence level might be the utmost 3-D measurement information available for App development.

How to process efficiently the original 24 x 24 or 24 x 18 depth points together with frame images is untouchable and unnecessary to know.
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Looking into depthMap - Apple iPad Pro LiDAR


Apple's advanced ML algorithm generates the 60 Hz ARDepthData including the depthMap and the confidenseMap; Depth API for ARKit 4.

CurvSurf has investigated the quality of the point cloud measured by Apple iPad Pro LiDAR. The depthMap of 256x192 pixel are unprojected to max 49,152 3-D points.

As we investigated, the depthMap represents relatively well planar object surfaces, but not curved object surfaces.

Apple's ML algorithm may work based on the linear color variations of the RGB frame image, interpolating the depth values between the max 576 (24x24) laser dots.

The raw point cloud of the max 576 laser dots should be investigated, if available.

Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 4 with Depth API
- Apple Metal API
- CurvSurf FindSurface for Web Demo.


youtu.be/h3PBy0psayY
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Outdoors depthMap - Apple iPad Pro LiDAR


60 Hz ARDepthData of Apple iPad Pro LiDAR contains the depthMap and the confidenceMap.

Apple's LiDAR 3-D camera is constantly updating the depthMap of 256x192 points even from object surfaces more than 10 meter away.

At the same time, the confidenceMap records the point reliabilities; low, medium or high.

As announced by Apple, the maximum measuring distance of Apple iPad Pro LiDAR is practically 5 meter.


Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 4
- Apple Metal API
- CurvSurf FindSurface for Web Demo.


youtu.be/4BUMhnt4Qx8
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by smacl »

Hi Joon,

I went to register at https://developers.curvsurf.com/ to download a demo of the SDK but it does not appear to be available yet. Any idea when the trial version will be available and what cost and licensing implications will be to developers for a commercial license for Windows based applications? It says free until early 2020 on the web site but I'm guessing that date has slipped with the Covid pandemic.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

smacl wrote: Wed Jul 08, 2020 8:20 am Hi Joon,

I went to register at https://developers.curvsurf.com/ to download a demo of the SDK but it does not appear to be available yet. Any idea when the trial version will be available and what cost and licensing implications will be to developers for a commercial license for Windows based applications? It says free until early 2020 on the web site but I'm guessing that date has slipped with the Covid pandemic.
Hi Shane,

Thank you for your interest in FindSurface.

We do not provide the FindSurface SDK more but web or cloud service. The current FindSurface web service is beta.

In the light of 5G connection, a web or cloud service is advantageous. First, it is platform independent. Second, you do not need to install any SDK before we will be having figured out the licensing and pricing model. Till then, the FindSurface web service is for free.

You will only pay for the services you use, e.g., per successful extraction of a geometric feature from your point cloud data.

We will not save your point cloud data on our server without your explicit permission.

The sample codes for requesting the FindSurface web service are available:
https://github.com/CurvSurf

Before you develop your own application requesting the FindSurface web service, you can try the FindSurface WebDemo:
https://developers.curvsurf.com/WebDemo/
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

The RGB-image of Apple iPad Pro LiDAR interpolates the raw max 576 (3x3x8x8) laser dot points, showing distortions in the resultant depthMap of max 256x192 points.

The access API to the max 576 raw laser dot points of the 3-D camera must be provided/opened.

Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 4
- Apple Metal API
- CurvSurf FindSurface for Web Demo.


youtu.be/nM4cPwUEdYw
User avatar
Joon
V.I.P Member
V.I.P Member
Posts: 335
Joined: Wed Aug 21, 2013 8:01 pm
10
Full Name: Sung Joon Ahn
Company Details: CurvSurf
Company Position Title: Founder + CEO
Country: Republic of Korea
Linkedin Profile: Yes
Location: Seongnam-si, Korea
Has thanked: 7 times
Been thanked: 35 times
Contact:

Re: We dream a magic button for 3-D point cloud processing

Post by Joon »

Real Time Ball Tracking in depthMap - Apple iPad Pro LiDAR

The max 256x192 depthMap points with the confidence threshold of medium are sampled randomly with the ratio of 1/128.

Tools used:
- Apple iPad Pro LiDAR
- Apple ARKit 4
- Apple Metal API
- CurvSurf FindSurface.


youtu.be/zc6GQOtgS7M
Post Reply

Return to “Modeling Software”