Open Point Clouds Format | Interview with DR. Daniel Wujanz

An open petition for unrestricted point cloud exchange.
Post Reply
User avatar
Jason Warren
Administrator
Administrator
Posts: 4224
Joined: Thu Aug 16, 2007 9:21 am
16
Full Name: Jason Warren
Company Details: Laser Scanning Forum Ltd
Company Position Title: Co-Founder
Country: UK
Skype Name: jason_warren
Linkedin Profile: No
Location: Retford, UK
Has thanked: 443 times
Been thanked: 246 times
Contact:

Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by Jason Warren »


youtu.be/WXbjqKhDUN0

In this video we have an interview with Dr. Daniel Wujanz, who is a co-founder of the Open Point Clouds Format petition. Due to the ever faster scanners, data processing is currently the bottleneck. The scanner manufacturers create their own data formats and other users can only get SDKs to read the data if the scanner manufacturer is interested. Otherwise there are free exchange formats, such as e57, which are not the best way to transfer data.

The idea of the petetion is that all market participants support each other in the best possible way, so that the user gets an optimal support of point clouds in his IT environment.

The petition in the original:

With this petition all subscribers "driven by the inspiration of a free and fair market" kindly ask all hardware manufacturers and software vendors to either provide Software Development Kits (SDK), format descriptions or executable code that consequently allow to have efficient read and possible write ability of raw and preprocessed (filtered and combined) point cloud formats, and all ancillary data1, in the best case free of charge and easy to license.

more info here:
https://openpointcloudformats.org/
Jason Warren
Co_Founder

Dedicated to 3D Laser Scanning
LaserScanningForum
jedfrechette
V.I.P Member
V.I.P Member
Posts: 1237
Joined: Mon Jan 04, 2010 7:51 pm
14
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 62 times
Been thanked: 220 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by jedfrechette »

At the end of the interview Daniel mentions that there were about 9 companies who contributed to authoring the petition. Since I couldn't find any specific info about the authors on openpointcloudformats.org I'm curious who those companies are?

Given what the petition is asking for I also assume that they're primarily software vendors, which leads to my second question: Are those founding companies already providing SDKs for their software products like they're asking other companies to provide?
Jed
User avatar
landmeterbeuckx
V.I.P Member
V.I.P Member
Posts: 1615
Joined: Tue May 01, 2012 5:19 pm
11
Full Name: Lieven Beuckx
Company Details: Studiebureau Beuckx
Company Position Title: Owner
Country: Belgium
Linkedin Profile: Yes
Has thanked: 183 times
Been thanked: 548 times

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by landmeterbeuckx »

jedfrechette wrote: Tue Oct 05, 2021 6:58 pm At the end of the interview Daniel mentions that there were about 9 companies who contributed to authoring the petition. Since I couldn't find any specific info about the authors on openpointcloudformats.org I'm curious who those companies are?

Given what the petition is asking for I also assume that they're primarily software vendors, which leads to my second question: Are those founding companies already providing SDKs for their software products like they're asking other companies to provide?
I'm one of the 9. I'm not a vendor, only a provider of scandata. I'm participating in this role as an independent guy. I shared some knowledge and obstructions when using pointcloud data crossplatform.

I also find difficulties in accessing certain industries because of these proprierty formats some deliver. Makes me weaker in some negotiating discussions.
LSBbvba
Surveying services - 3D Laserscanning
Tel : +32477753126
www.lsbbvba.be
[email protected]
pmalatzky
I have made 30-40 posts
I have made 30-40 posts
Posts: 35
Joined: Wed Mar 29, 2017 6:19 am
7
Full Name: Paul Malatzky
Company Details: Point Share Plus
Company Position Title: CEO
Country: Australia
Linkedin Profile: Yes
Has thanked: 7 times
Been thanked: 11 times

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by pmalatzky »

At the end of the interview Daniel mentions that there were about 9 companies who contributed to authoring the petition. Since I couldn't find any specific info about the authors on openpointcloudformats.org I'm curious who those companies are?
You will find them listed in the Imprint section of the website here...OPCF Imprint
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by smacl »

jedfrechette wrote: Tue Oct 05, 2021 6:58 pm At the end of the interview Daniel mentions that there were about 9 companies who contributed to authoring the petition. Since I couldn't find any specific info about the authors on openpointcloudformats.org I'm curious who those companies are?

Given what the petition is asking for I also assume that they're primarily software vendors, which leads to my second question: Are those founding companies already providing SDKs for their software products like they're asking other companies to provide?
Also one of the nine mentioned and owner of the small software house that develops SCC. I'm in the process of developing an interface and matching SDK that will allow full access to SCC point clouds. I'm confident that all other software houses that are involved in this will also be providing a similar SDK. This is absolutely to the benefit of both the end users and the developers as it enables the efficient combination of multiple software packages across different workflows, which is of course the goal of this petition. Most of my clients already use an array of software in addition to my own, all this does is make this more efficient and robust. I'll be posting updates on the forum in relation to this.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
VXGrid
V.I.P Member
V.I.P Member
Posts: 544
Joined: Fri Feb 24, 2017 10:47 am
7
Full Name: Martin Graner
Company Details: PointCab GmbH
Company Position Title: Research and Development
Country: Germany
Linkedin Profile: No
Has thanked: 160 times
Been thanked: 175 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by VXGrid »

jedfrechette wrote: Tue Oct 05, 2021 6:58 pm At the end of the interview Daniel mentions that there were about 9 companies who contributed to authoring the petition. Since I couldn't find any specific info about the authors on openpointcloudformats.org I'm curious who those companies are?

Given what the petition is asking for I also assume that they're primarily software vendors, which leads to my second question: Are those founding companies already providing SDKs for their software products like they're asking other companies to provide?
This is a good point Jed!

Since we are the first company in the imprint, let me confess:
We have a SDK, which is called LSDSDK (since our data has the ending lsd), with which you can read and write our point cloud format.
As the website indicates this is already in use by technet and Lupos3d:
https://pointcab-software.com/en/lsdsdk-2/

The cooperation with technet was based on customer needs (perhaps you know that Scantra is not saving any point cloud data, they have only a database to store plane information to do a high quality registration).
So the workflow for the customers were like:
  1. Import raw scan files into our software
  2. Export E57 from our software
  3. Read E57 in Scantra
  4. Apply Registration parameters on E57 in Scantra
  5. Import the E57 in our software again
We are talking here about projects with more than 500 scan stations.
With the integration the time expensive part of exporting and reimporting the E57 files is now gone.
The ones really benefitting from the openpointclouds petition are the customers who save a lot of time and drive space.
In addition in this process a lot less information is lost, because with the interface you can make sure to read/write all the data available/in use by the other software.
jedfrechette
V.I.P Member
V.I.P Member
Posts: 1237
Joined: Mon Jan 04, 2010 7:51 pm
14
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 62 times
Been thanked: 220 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by jedfrechette »

pmalatzky wrote: Wed Oct 06, 2021 7:05 am You will find them listed in the Imprint section of the website here...OPCF Imprint
Ahh thanks! I missed that page. It looks like roughly half are software vendors and the rest are service providers or universities.

I guess I'm struggling to see how the proposed solution, where everybody provides SDKs for their own internal data structures, can scale to solve the problem the authors are trying to solve. If you have 5 software vendors and want all of them to talk to each that's already 20 sets of interface code that need to be maintained, not including the 5 stable APIs that the vendors need to provide and maintain so that the others can access their data. Double the number of vendors to 10 and you've already got 90 interfaces plus the APIs and it just gets exponentially worse from there.

In practice what probably happens is that each of the vendors can only afford to support a few of the most popular APIs. That benefits the biggest players, i.e. Leica, Faro, Autodesk etc., much more than the smaller players. I'm also skeptical of the idea that providing and using native APIs will inherently improve reliability and performance compared to using well defined open file formats. Just look at how many incompatibilities have historically plagued interoperability between those big 3 (Leica, Faro, and Autodesk) even though they're already using each other's native APIs.

Furthermore, the idea of internal formats being better than ones explicitly designed for exchange always makes me chuckle and think of the infamous source code comment about Photoshop files. I'm sure all the internal point cloud formats in use are much better. ;-)

I'm coming at this from the perspective of a service provider who also spends a lot of time developing our own internal tools. All of the main software applications that we've adopted already provide an SDK or API in one form or another. In most cases we make heavy use of these APIs. Although failure to provide an API doesn't immediately preclude adoption of your software in our pipeline it is a major strike against it.

Even though we prioritize using software that provides good APIs, wherever possible we're actively replacing data exchange via APIs with exchange via open file formats because of the instability and pain that proprietary APIs have led to in our experience. The extra minutes (or hours) "lost" to publishing and transferring data via well defined and immutable open file formats pales in comparison to the amount of time we've lost because undocumented change X to proprietary API Y in release foo.0.1 completely changed the way that API delivered data and broke everything that depended on that API. If your software doesn't provide good quality exports in open file formats, it is almost guaranteed that we won't adopt your software, regardless of what else it might offer.

I have zero interest in maintaining half a dozen different sets of completely different interface code so we can read native data using as many different APIs. Although it would certainly be nice, for example, to have e57 dialects that are more rigidly defined, I would much rather deal with the variations of that format than the infinitely variable set of native API interfaces that seem to be the alternative being proposed
Jed
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by smacl »

Hi Jed,

Some excellent points there, which I've also raised in discussion with the group.
jedfrechette wrote: Thu Oct 07, 2021 3:36 amI guess I'm struggling to see how the proposed solution, where everybody provides SDKs for their own internal data structures, can scale to solve the problem the authors are trying to solve. If you have 5 software vendors and want all of them to talk to each that's already 20 sets of interface code that need to be maintained, not including the 5 stable APIs that the vendors need to provide and maintain so that the others can access their data. Double the number of vendors to 10 and you've already got 90 interfaces plus the APIs and it just gets exponentially worse from there.
I have zero interest in maintaining half a dozen different sets of completely different interface code so we can read native data using as many different APIs. Although it would certainly be nice, for example, to have e57 dialects that are more rigidly defined, I would much rather deal with the variations of that format than the infinitely variable set of native API interfaces that seem to be the alternative being proposed
My preferred approach to this would be to develop and publish a standardised interface to which these SDKs must adhere, including a test kit which both verifies the data provider SDK is working and benchmarks it to demonstrate that it is working efficiently, and also one or more sample implementations which wrap existing SDKs such as LAStools and E57lib. Any given provider implementation can then be independently tested with results published to show users what formats are supported. This would allow those using the interface to consume point cloud information to work with a single, well defined and robust interface across all supported SDKs. It would also allow enhancements by adding to (not changing) the interface and enable development of wrappers in other languages. It would also allow for combination of proprietary and open source provider implementations with differing licenses. I think this is a far superior approach to standardised formats, some more discussion here.

Needless to say, as someone deeply involved in working with point clouds from a coding perspective, your thoughts and input on this would be greatly appreciated. It is very much intended to be a community effort which is there first and foremost to serve the needs of the community. Note the above is simply my own preferred approach and very much at discussion stage.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
jedfrechette
V.I.P Member
V.I.P Member
Posts: 1237
Joined: Mon Jan 04, 2010 7:51 pm
14
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 62 times
Been thanked: 220 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by jedfrechette »

smacl wrote: Thu Oct 07, 2021 8:07 am
My preferred approach to this would be to develop and publish a standardised interface to which these SDKs must adhere, including a test kit which both verifies the data provider SDK is working and benchmarks it to demonstrate that it is working efficiently, and also one or more sample implementations which wrap existing SDKs such as LAStools and E57lib. Any given provider implementation can then be independently tested with results published to show users what formats are supported. This would allow those using the interface to consume point cloud information to work with a single, well defined and robust interface across all supported SDKs. It would also allow enhancements by adding to (not changing) the interface and enable development of wrappers in other languages. It would also allow for combination of proprietary and open source provider implementations with differing licenses.
Isn't this just a less flexible variation of what PDAL is already doing? Asking SDK providers to make their output adhere to a specific interface specification makes it hard for them to push changes when they want to add features. They either need to get the spec changed to include what they want to add or, more likely, just publish the new feature as an unsupported extension outside the spec. At which point we're right back to where we started with a lowest common denominator interface and a bunch of vendor specific special cases outside the spec.

The PDAL approach of building a library that defines one interface consumers can code against, while also housing the code that transforms from arbitrary data sources to that interface seems much more scalable. Vendors are free to write their own SDKs to provide whatever data they feel is most relevant for their particular product and it is up to PDAL to make that data compatible with its interface so that consumers coding against PDAL's interface don't need to worry about where the data came from. Most importantly, the overall amount of code that needs to be maintained by everybody involved goes way down.

As additional prior art just look at GDAL and OGR, whose design PDAL was inspired by. All three are open source libraries that know how to read and write data from a bunch of different data sources. In the GIS world the GDAL and OGR libraries have been widely adopted by both open and closed source applications and have largely solved the types of compatibility issues we're still dealing with. I don't see why our part or the industry couldn't follow a similar model especially when a lot of the very hard groundwork has already been done over the last several years by the folks contributing to PDAL.

Why start a new initiative from scratch rather than building on what has already been done?
Jed
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: Open Point Clouds Format | Interview with DR. Daniel Wujanz

Post by smacl »

jedfrechette wrote: Thu Oct 07, 2021 4:18 pmWhy start a new initiative from scratch rather than building on what has already been done?
With respect, I think the goals of this initiative and those of PDAL are different while entirely compatible. For example, I have a client at the moment that is regularly taking large LGS files into SCC. The current workflow involves translation from LGS into E57 and then from E57 into native SCC formats. This occupies a Leica Publisher license for the export process, a SCC license for the import process, wastes a ton of time for the user and also loses important data en-route. This initiative seeks to get rid of those intermediate import and export processes involving the E57 and maintaining data integrity. PDAL can't currently help here, it neither reads LGS nor writes SCC point clouds. If Leica were to provide an SDK to allow reading of LGS and that was implemented within SCC, the problem is solved. Similarly if SCC were to provide a similar interface, the same would be true in the other direction. If these and similar SDKs are implemented using a common interface, it reduces the development workload you raised in your previous post. Similarly, should someone then want to use any of these data formats in a PDAL pipeline, they have the means to do so. While PDAL has a decent array of readers and writers they largely deal with existing open source formats sitting on existing open source SDKs for those formats. As such, it inherits all of the performance and data loss issues associated with those formats and SDKs. This is not a problem with PDAL, the problem is a lack of efficient and lossless data exchange SDKs to the most commonly used native formats within the industry. This is the problem we are trying to address here.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
Post Reply

Return to “openpointcloudformats.org”