What Point Cloud format do you use and why?

Please post all your polls you want others opinion on.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: What Point Cloud format do you use and why?

Post by smacl »

RJGEOMATICS wrote: Sat Jul 31, 2021 10:28 pmShane from SCC Atlas Computers can give a far more technical explanation as to e57.
E57 is a great format and very flexible, but unfortunately the support for it in different software packages can be patchy, so you may well lose data in the process of moving data between packages. There is a reference implementation in C++ for reading and writing e57 files, but it hasn't seen any active development since 2014 and its performance characteristics on large modern point clouds is not good, see http://www.libe57.org/ There is a more recent unofficial fork here https://github.com/asmaloney/libE57Form ... ANGELOG.md which has seen more updates but does not implement all E57 functionality. E57 can be a good option for transferring terrestrial laser scanning data where you want to preserve setup information with a view to re-registering or want to transfer photography.

For archiving point cloud data, I'd recommend storing all the raw data in its native format as well as the finished unified data in an open format such as LAS, LAZ, E57 or PTS. The reason for storing the raw data in its native format is that transfer into any other open format is likely to be lossy. The reason for the open format data is that you can supply it to anyone who does not have a license to your processing software. Personally, I'm of the opinion that if you're archiving potentially huge quantities of data on behalf of your clients, you should be charging them an annual service fee for doing so as maintaining this type of data, either in the cloud or on-site, has an associated ongoing cost.

While I haven't used it to any great degree as yet, for storage of unified point clouds in the cloud or on big servers, Potree 2.1 looks like a very promising medium to long term option. The reason here is that the data is already stored in an octree in a format that can be viewed and analysed directly online with no extra import/export overhead. I really need to look at this myself in some more depth before commenting further.

As for the closed proprietary formats, many have 3rd party SDKs to allow transfer, which many software developers implement. In SCC, these currently include Recap, Leica MS60, Z+F and Faro. So while they're closed formats, they're not entirely closed.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
jedfrechette
V.I.P Member
V.I.P Member
Posts: 1237
Joined: Mon Jan 04, 2010 7:51 pm
14
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 62 times
Been thanked: 219 times
Contact:

Re: What Point Cloud format do you use and why?

Post by jedfrechette »

smacl wrote: Mon Aug 02, 2021 7:01 pm For archiving point cloud data, I'd recommend storing all the raw data in its native format as well as the finished unified data in an open format such as LAS, LAZ, E57 or PTS. The reason for storing the raw data in its native format is that transfer into any other open format is likely to be lossy. The reason for the open format data is that you can supply it to anyone who does not have a license to your processing software.
Agreed, but licensing aside, how confident are you that in 10 years [VENDOR]'s software will still read the proprietary format's that they are writing today? 20?

Using open formats, or constant data maintenance and migrations, are about the only ways to assure you'll actually be able to read the data if you do need to access it after an extended period of time.
Jed
Augusto 3D
V.I.P Member
V.I.P Member
Posts: 436
Joined: Sat Dec 06, 2014 5:26 am
9
Full Name: C-Augusto
Company Details: 3d scanning
Company Position Title: USA
Country: US
Has thanked: 115 times
Been thanked: 30 times

Re: What Point Cloud format do you use and why?

Post by Augusto 3D »

^ this is one of those thing where I think, that just like upgrade our Revit models every few releases, you should do the same with our data ?
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: What Point Cloud format do you use and why?

Post by smacl »

jedfrechette wrote: Mon Aug 02, 2021 9:32 pm
smacl wrote: Mon Aug 02, 2021 7:01 pm For archiving point cloud data, I'd recommend storing all the raw data in its native format as well as the finished unified data in an open format such as LAS, LAZ, E57 or PTS. The reason for storing the raw data in its native format is that transfer into any other open format is likely to be lossy. The reason for the open format data is that you can supply it to anyone who does not have a license to your processing software.
Agreed, but licensing aside, how confident are you that in 10 years [VENDOR]'s software will still read the proprietary format's that they are writing today? 20?

Using open formats, or constant data maintenance and migrations, are about the only ways to assure you'll actually be able to read the data if you do need to access it after an extended period of time.
Agreed Jed, that's another reason why I'd recommend archiving both the original native formats as well as an open source format. If you just archive native format, you run the risk of not being able to access the data at some point in the future. If you just archive in an open source format you may lose some of your original data and the ability to process it, e.g. trajectory information in mobile mapping, mapping projection and grid system details, certain classes of photography and lens correction (internal parameters), etc... Coming from a native unified coordinated native data format such as SCC point cloud, Leica Jetstream etc... you're still liable to lose a lot of information exporting to an open format, e.g. any associated vector data and annotation, any generated extra per point attributes such as computed height difference, design surface separation etc... While some of the open formats are extensible and could theoretically include this information, most implementations don't support such extensions for either read or write. We really lack any robust and widely used standards in this area at present and in some regards I'd personally regard E57 as the least worst option.

There is a real need for a better open source format for point clouds, for data transfer, visualization and analysis and long term storage. It needs to have the following characteristics;
  • Ability to store all data related to the point cloud data capture, registration and processing. This includes scanned or measured points from TLS, MLS and photogrammetry, all related registration and control information, all images and image control, all per point derived information, and all additional information derived through extraction.
  • High speed (multi-threaded and/or GPU compute based) import, export from other formats and via SDK.
  • High speed direct access via SDK for tool developers using an open source, non-restrictive license. i.e. suitable as a fully performant native format for point cloud processing packages without need for any import/export for storage or transfer to related packages. By performant, I'm referring to being speed and space efficient for multi-billion point clouds and associated data on mid-range workstation, scalable to larger clouds with more powerful computer resources.
  • Fully backward and forward compatible across versions
  • End user tools for fast online and local visualisation and annotation.
The best prospect that I'm seeing as a starting point for this at present is the POTREE 2.1 implementation but it is just a starting point. I'd be interested in hearing your thoughts on this, and anyone else's. My feeling is that at this point in time that the industry has become heavily constrained by too many closed formats, some of which are being intentionally used to lock users to specific vendors.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: What Point Cloud format do you use and why?

Post by smacl »

Just to add an additional thought to the above following on from some work I've been doing with a client this morning. If your point cloud is classified, LAS or LAZ are better options for archiving your data long term than E57 as LAS can include both standardised and extended point class info. This is also true of POTREE. Not sure if this can be done in E57, but I don't think so and have certainly never seen it. We use point classification heavily in SCC as do many other providers such as Leica. It is also typically used as the basis for point cloud segmentation, which in my opinion will become increasingly important for automated point cloud analysis and streamlined processing. The basic LAS classification is limited (32 predefined classes) but still extremely useful, e.g. hide all trees, show only buildings, mark points as noise (deleted) etc...
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
badam
V.I.P Member
V.I.P Member
Posts: 916
Joined: Tue May 11, 2021 5:36 pm
2
Full Name: Adam Berta
Company Details: InnoScan 3D Hungary Kft
Company Position Title: unknown
Country: Hungary
Linkedin Profile: No
Has thanked: 51 times
Been thanked: 297 times
Contact:

Re: What Point Cloud format do you use and why?

Post by badam »

Potree is a one way conversion (currently), So you need a different version of that cloud. We just keep that version because usualy thats our deliverable where we can add panorama photos, tags,etc.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: What Point Cloud format do you use and why?

Post by smacl »

badam wrote: Thu Aug 05, 2021 6:30 am Potree is a one way conversion (currently), So you need a different version of that cloud. We just keep that version because usualy thats our deliverable where we can add panorama photos, tags,etc.
Agreed, though Potree is also open source so adding exports to other open formats in future should be relatively straightforward. My main interest in it is that it is a good candidate as the starting point for a much more attractive open format going forward.
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
RJGEOMATICS
V.I.P Member
V.I.P Member
Posts: 133
Joined: Tue Apr 23, 2019 7:20 pm
4
Full Name: Bart Man
Company Details: Laser Scanning
Company Position Title: Passionate about making amazing PCs
Country: Canada
Linkedin Profile: No
Has thanked: 38 times
Been thanked: 68 times

Re: What Point Cloud format do you use and why?

Post by RJGEOMATICS »

smacl wrote: Thu Aug 05, 2021 10:59 am
badam wrote: Thu Aug 05, 2021 6:30 am Potree is a one way conversion (currently), So you need a different version of that cloud. We just keep that version because usualy thats our deliverable where we can add panorama photos, tags,etc.
Agreed, though Potree is also open source so adding exports to other open formats in future should be relatively straightforward. My main interest in it is that it is a good candidate as the starting point for a much more attractive open format going forward.
Like your thoughts on PoTree Shane. Would be nice to find a more universally stable format.

unfortunately there are just too many issues with sharing e57 across everyone. And the Photo Quality for Mirror Balls from e57 files is awful. With the great increase of Photogrammetry, it is very important that the new format include HDR values and everything.

Often been issues with e57 formats. There are many programs that can ingest e57s. But not always correctly. And often they can't be written back out in a suitable format for ingestion to other programs... ReCAP for instance can Ingest almost everyone's e57 files but what it writes out as e57 is not always able to be ingested by other software, and unfortunately the same with CloudCompare. Now that being said, PTS can be utilized by almost everyone. But all it is really, is a fancy Cogo Point File with no vector data and other important aspects. That being said, PTS and LAS seem to be the most uniform formats that everyone can read, export and utilize. They are just really simple, and do not contain nearly enough information for todays industry...
mschuetz
I have made 40-50 posts
I have made 40-50 posts
Posts: 40
Joined: Fri Nov 01, 2013 7:03 am
10
Full Name: Markus Schuetz
Company Details: potree.org
Company Position Title: developer
Country: Austria
Linkedin Profile: No
Been thanked: 5 times

Re: What Point Cloud format do you use and why?

Post by mschuetz »

Hi, nice to hear that the Potree format is a serious contender. However, I'd recommend against using it for long-term or storage since I've mainly developed it for visualization, and not necessarily reliability and reversibility. You should always keep a copy of the original las/laz files.

For persistent storage with the ability to stream and real-time render, this early-stage project here might be interesting: https://github.com/copcio/copcio.github.io
Their goal would essentially be streamable laz, and since laz nowadays supports arbitrary extra attributes, it seems like a good choice to me. It's still WIP, though.
User avatar
smacl
Global Moderator
Global Moderator
Posts: 1409
Joined: Tue Jan 25, 2011 5:12 pm
13
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 627 times
Been thanked: 657 times
Contact:

Re: What Point Cloud format do you use and why?

Post by smacl »

mschuetz wrote: Thu Aug 05, 2021 9:07 pm Hi, nice to hear that the Potree format is a serious contender. However, I'd recommend against using it for long-term or storage since I've mainly developed it for visualization, and not necessarily reliability and reversibility. You should always keep a copy of the original las/laz files.

For persistent storage with the ability to stream and real-time render, this early-stage project here might be interesting: https://github.com/copcio/copcio.github.io
Their goal would essentially be streamable laz, and since laz nowadays supports arbitrary extra attributes, it seems like a good choice to me. It's still WIP, though.
Hi Markus,

Firstly, thanks for the heads up and all the great work on potree to date. As things stand, all of the current open source formats that I'm aware of are lossy in one form or another, hence the need to also archive all the native data. e.g. as noted above for E57 in weak image handling by RJ and myself in not storing class information. Similarly, while LAZ supports extra attributes most implementations out there don't do so, so currently using LAZ is also lossy. As the readme on copcio notes, we absolutely need a mechanism to relate points back to their original setup number for purposes of image linking, QA and potential re-registering. We also need support for an arbitrary number of extra per point attributes which can have different types (e.g. byte, word, float, double) to allow persistent storage of attributes such as class, temperature for thermal data, design separation, and all the other future applications yet to be imagined.

COPC – Cloud Optimized Point Cloud looks like it could be a real contender here. Ideally it should work well for desktop as well as cloud but this should be reasonably trivial. The thinking here is that if it was performant to the extent that it could be used as a native format for desktop (& laptop) for point cloud processing, we could avoid the need for many time and resource consuming (and potentially lossy) import export operations. WIP is good, means it is an ideal time to get involved ;)

Shane
Shane MacLaughlin
Atlas Computers Ltd
www.atlascomputers.ie

SCC Point Cloud module
Post Reply

Return to “Opinion Polls”