Threadripper

Discuss all Cyclone related issues here.
Post Reply
User avatar
smacl
V.I.P Member
V.I.P Member
Posts: 224
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 19 times
Been thanked: 20 times
Contact:

Re: Threadripper

Post by smacl » Wed Apr 10, 2019 9:33 am

lastools wrote:
Fri Apr 05, 2019 9:22 pm
Hello from @LAStools,

Yep. LASzip decompression on a single core is typically CPU bound (unless the data comes straight across the Internet). But LASzip was designed with parallelism for decompression in mind such that the use of multi-cores when decompressing a single LAZ file is theoretically possible. It's just a matter of writing (and then maintaining) the code. Each "chunk" of 50000 points is compressed independently from all other chucks, so that 4, 8, 16, 48, or 128 chunks could be decompressed simultaneously if you have fast file read access and many cores. However, someone (you?) would need to implement a multi-threaded decompressor. All the code is open source so go right ahead ... (-:

Regards,

Martin @rapidlasso
Hi Martin,

First off, big fan of the LAZ format and the advantages in brings in moving big point clouds around the place. In terms of an optimal format however, I think there is more needed than a faster read/writer. The issue as I see it with LAS, E57 and other point cloud transfer formats is that they invariably have to be translated to a faster spatially indexed format (e.g. octree, MNO, hierarchy of voxels, etc...) in order to be used efficiently in interactive point cloud analysis. There is a clear requirement to quickly extract subsets of points based on various spatial queries or rendering culls, and to load or unload tiles or similar larger groups from storage. I would be great to have an open point cloud exchange format that also worked as an efficient native format.

That said, if I get a bit of free time, I'll have a look at multi-threading the LAZ I/O,

Shane

Scott
V.I.P Member
V.I.P Member
Posts: 792
Joined: Tue Mar 29, 2011 7:39 pm
Full Name: Scott Page
Company Details: Scott Page Design- Architectural service
Company Position Title: Owner
Country: USA
Linkedin Profile: No
Location: Berkeley, CA USA
Has thanked: 6 times
Been thanked: 8 times
Contact:

Re: Threadripper

Post by Scott » Wed Apr 10, 2019 3:29 pm

smacl wrote:
Wed Apr 10, 2019 9:33 am
lastools wrote:
Fri Apr 05, 2019 9:22 pm
Hello from @LAStools,
Yep. LASzip decompression on a single core is typically CPU bound (unless the data comes straight across the Internet). But LASzip was designed with parallelism for decompression in mind such that the use of multi-cores when decompressing a single LAZ file is theoretically possible. It's just a matter of writing (and then maintaining) the code. Each "chunk" of 50000 points is compressed independently from all other chucks, so that 4, 8, 16, 48, or 128 chunks could be decompressed simultaneously if you have fast file read access and many cores. However, someone (you?) would need to implement a multi-threaded decompressor. All the code is open source so go right ahead ... (-:
Regards,

Martin @rapidlasso
Hi Martin,
First off, big fan of the LAZ format and the advantages in brings in moving big point clouds around the place. In terms of an optimal format however, I think there is more needed than a faster read/writer. The issue as I see it with LAS, E57 and other point cloud transfer formats is that they invariably have to be translated to a faster spatially indexed format (e.g. octree, MNO, hierarchy of voxels, etc...) in order to be used efficiently in interactive point cloud analysis. There is a clear requirement to quickly extract subsets of points based on various spatial queries or rendering culls, and to load or unload tiles or similar larger groups from storage. I would be great to have an open point cloud exchange format that also worked as an efficient native format.
That said, if I get a bit of free time, I'll have a look at multi-threading the LAZ I/O,
Shane
An interesting topic (and analysis) -perhaps it is time to start a new tread topic on 'cloud transfer formats'?

User avatar
lastools
V.I.P Member
V.I.P Member
Posts: 144
Joined: Tue Mar 16, 2010 3:06 am
Full Name: Martin Isenburg
Company Details: rapidlasso - fast tools to catch reality
Company Position Title: creators of LAStools and LASzip
Country: Germany
Skype Name: isenburg
Linkedin Profile: Yes
Contact:

Re: Threadripper

Post by lastools » Wed Apr 10, 2019 11:13 pm

Hello from @LAStools,
First off, big fan of the LAZ format and the advantages in brings in moving big point clouds around the place. In terms of an optimal format however, I think there is more needed than a faster read/writer. The issue as I see it with LAS, E57 and other point cloud transfer formats is that they invariably have to be translated to a faster spatially indexed format (e.g. octree, MNO, hierarchy of voxels, etc...) in order to be used efficiently in interactive point cloud analysis. There is a clear requirement to quickly extract subsets of points based on various spatial queries or rendering culls, and to load or unload tiles or similar larger groups from storage. I would be great to have an open point cloud exchange format that also worked as an efficient native format.
Shane, there is an (open) extension to the LAZ format that does exactly that. It is called LASindex or LAX. It was optimized for 2D queries (aka aerial and mobile LiDAR collects) and is based on a 2D quadtree decomposition of the plane but there is nothing preventing us from extending this to a full 3D version using an octree. The nice thing is that it works on top of the existing LAZ or LAS file and even for other seek-able format that the LASlib has a driver for such as NASA's QFIT format or Terrasolid's old BIN format. This effectiveness of this spatial index depends in the order of points in the file. The free lasoptimize.exe - which arranges the points in a space-filling curve order - can improve this order to the near optimum. Below is a video of me explaining details about LASindex and the LAX format at the ELMF conference in Salzburg. And the really really nice thing: it's already implemented in the tools and in the LASzip DLL. Just add '-lax' to your laszip.exe command and you have spatially indexed files. Or use lasoptimize.exe instead of laszip.exe to compress the points into a "better order" for spatial queries. And you can readily enable spatially-accelerated "area-of-interest" queries for reading LAS or LAZ files when you use LASlib or the LASzip DLL in your own software.


youtu.be/FMcBywhPgdg

Regards,

Martin @rapidlasso

User avatar
smacl
V.I.P Member
V.I.P Member
Posts: 224
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 19 times
Been thanked: 20 times
Contact:

Re: Threadripper

Post by smacl » Mon Apr 15, 2019 6:31 pm

Hi Martin, thanks for the feedback and links, it is something I need to spend more time on and look into in depth. It would certainly need an octree for very many applications, e.g. structures of any kind. Out of interest, can you store additional arbitrary information in a LAS file. Looking at where we're going with deep learning, I'm of the opinion that the combination of images and point clouds is going to become increasingly important, and the instrument setup and orientation likewise to make use of this in terrestrial scanning.

dhirota
V.I.P Member
V.I.P Member
Posts: 549
Joined: Sun Nov 01, 2009 11:18 pm
Full Name: Dennis Hirota
Company Details: Sam O. Hirota Inc.
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Been thanked: 3 times

Re: Threadripper

Post by dhirota » Sun Oct 06, 2019 8:47 pm

Just in case you needed to price a system

From Quora, Kristoffer Johnsen

"Well the fastest, most powerful setup with parts currently on the market would probably be this:

Two AMD Epyc 7742 CPUs which each are 64 core 128 thread monsters for a total of 128 cores and 256 threads. Each costs $7,000 for a total of 14,000
A Supermicro MBD-H11DSI-NT-B dual socket motherboard for around $700
To cool the CPUs you wouldn't need anything more crazy than two Noctua NH-U14S tower coolers for $90 each so $180 total.
Two Nvidia Titan RTX graphics cards running with Nvlink. $5,000 in total.
2 TB of 2666MT/s quad rank ECC memory which would mean sixteen 128GB Crucial CT128G4YFE426S memory dimms each costing around $3,500 totalling $56,000 in memory.
3x SYNOLOGY M2D18 PCIe x8 to 2x M.2 adapters for $150 each and $450 total
7x Samsung 970 EVO Plus 2TB M.2 NVME SSDs each $500 running in RAID0 like a total madman for a total of 14TB and $3,500
3x Seagate IronWolf Pro ST16000NE000 16TB hard drives. 48TB of storage and $1,500 total
An EVGA Supernova T2 1600W power supply for around $500
A nice Corsair Crystal Series 680X case to put it all in. With enough fans, nice RGB lighting and tempered glass panels all over for around $300

This would probably be THE BEST and fastest combo of parts ever put together. You could probably run a small nation with this thing and play Fortnite at the same time, but it would cost you $82,130"
Dennis Hirota, PhD, PE, LPLS
[email protected]

Oatfedgoat
Global Moderator
Global Moderator
Posts: 1045
Joined: Mon Oct 06, 2008 8:31 pm
Full Name: Matthew McCarter
Company Details: Costain
Company Position Title: BIM Manager
Country: England
Linkedin Profile: Yes
Has thanked: 5 times
Been thanked: 2 times

Re: Threadripper

Post by Oatfedgoat » Mon Oct 07, 2019 8:10 am

My son has become obsessed with upgrading his already pretty decent gaming computer at home. As a result I've been doing some research on top end home user CPU's and release dates.

Intel are due to release their 10th generation i9 CPU's some time in October/November.
These have a significant price drop of around 50% compared to the current crop of i9's with no drop in performance/cores/clock speed.

So for anyone not looking at Xeon's from the house of intel or at AMD, the CPU's in the table below look like they will be very good value.

Intel 10th Gen Core i9 'Cascade Lake' X-Series CPU Lineup:

Intel Core i9-10980XE 18/36 3.0 GHz 4.8 GHz 3.8 GHz 24.75 MB 165W $979
Intel Core i9-10940X 14/28 3.3 GHz 4.8 GHz 4.1 GHz 19.25 MB 165W $784
Intel Core i9-10920X 12/24 3.5 GHz 4.8 GHz 4.3 GHz 16.50 MB 165W $689
Intel Core i9-10900X 10/20 3.7 GHz 4.7 GHz 3.8 GHz 13.75 MB 165W $590

There's a good table here showing the comparisons:
https://videocardz.com/82105/intel-10th ... ing-leaked

On a personal note, we will probably upgrde his i7 8086k (OC to 5.0ghz) to the i9 10829X above..... plus a new motherboard to accomodate :roll:

User avatar
smacl
V.I.P Member
V.I.P Member
Posts: 224
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 19 times
Been thanked: 20 times
Contact:

Re: Threadripper

Post by smacl » Mon Oct 07, 2019 9:16 am

While Intel i9 will have better single core performance, if you consider the TR 1950x 16 core 32 thread 4ghz CPU comes in at $489 the i9 still seems expensive even at reduced prices. Given threadripper 3 CPUs are due out in November I'd tend to put off the upgrade for a month as it is likely to drive down prices of existing versions and competitive chips that bit more. I don't think Intel will really be competitive again until they get their 7nm process sorted.

User avatar
smacl
V.I.P Member
V.I.P Member
Posts: 224
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 19 times
Been thanked: 20 times
Contact:

Re: Threadripper

Post by smacl » Thu Nov 07, 2019 3:50 pm

Threadripper 3 release date is set to 25th November with 24 core and 32 core versions, more here. I'll be very interested to see the benchmarks following the release.

Post Reply

Return to “Leica Cyclone”