Threadripper
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: Threadripper
We are running Ubuntu 18.04 and 14.04 on three platforms (Dell 7710 laptop, i7x8 thread PC, i9x36 thread PC) for either displaying NavVIS M6 scans+imaging or post-processing registration. I have interfaced with Riegl support checking our VZ400i, and they were using linux to test the scanner. Riegl's software folders contain many linux libraries, but I am not aware of any RiScan Pro or RiSolve programs available in linux to the public.
We are evaluating other software applications for BIM using Ubuntu as well as server systems.
We are evaluating other software applications for BIM using Ubuntu as well as server systems.
- gabrio
- V.I.P Member
- Posts: 138
- Joined: Thu Mar 03, 2016 6:32 pm
- 8
- Full Name: Gabrio Rossi
- Company Details: Intellegere srl
- Company Position Title: technical director
- Country: Italy
- Linkedin Profile: Yes
- Location: Bergamo - Italy
- Been thanked: 9 times
- Contact:
Re: Threadripper
Thanks Dennis!
arch. Gabrio Rossi, MRICS
[email protected]
Passaggio dei Canonici Lateranensi n. 4
24121 Bergamo - Italy
+39.035.601.1492
[email protected]
Passaggio dei Canonici Lateranensi n. 4
24121 Bergamo - Italy
+39.035.601.1492
-
- V.I.P Member
- Posts: 205
- Joined: Mon Feb 25, 2019 4:42 am
- 5
- Full Name: Major Domo
- Company Details: VeritasVfx
- Company Position Title: ceo
- Country: Portugal
- Linkedin Profile: No
- Has thanked: 17 times
- Been thanked: 53 times
Re: Threadripper
I have used Cyclone running in a Windows 7 VM under a Linux host (kubuntu 16.04 64 bits) Leica will have to enable virtualization for your EID license.
We had to do this due to security/clearance reasons.
I did not see any significant difference in performance, except for some lag introduced by the low-end virtual graphics card in the region of 20ms, as it was on a laptop I could not passtrough a specific card to the guest OS, i am curious as to how it would perform on a workstation with multiple discreet graphics cards.
We had to do this due to security/clearance reasons.
I did not see any significant difference in performance, except for some lag introduced by the low-end virtual graphics card in the region of 20ms, as it was on a laptop I could not passtrough a specific card to the guest OS, i am curious as to how it would perform on a workstation with multiple discreet graphics cards.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Threadripper
Techradar are speculating up to 64 cores which would mean 128 threads but it will be a while longer before we know. As a software dev this makes things very interesting and increases the push to make all the performance critical code heavily multi-threaded. It would be interesting to run a performance monitoring tool on various processes to see how many slow processes out there are still single threaded. Most of the I/O for reading and writing to disk using 3rd party libraries (e.g. LAS,LAZ, E57, RCP/RCS, FLS) are currently bottlenecks in my experience.
-
- V.I.P Member
- Posts: 136
- Joined: Thu Mar 16, 2017 3:29 pm
- 7
- Full Name: Neerav Mehta
- Company Details: Indoor Intelligence
- Company Position Title: CTO
- Country: USA
- Linkedin Profile: No
- Has thanked: 1 time
- Been thanked: 6 times
Re: Threadripper
With multi-threaded applications, I/O will remain the bottleneck. In fact, if try to use multiple threads by breaking down the reading/writing of a huge file, it could turn out to be even slower since now the HDD's mechanical head needs to move more when switching between threads. Even SSDs, that don't have mechanical head, may be slower when read this way.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Threadripper
I think this would only be true if the time to read a point cloud from a disk was as fast as copying that file from a fast SSD, but typically it isn't. To check this, compare how long it takes to copy a large E57/LAS/PTS/FLS between fast SSDs with how long it takes to import it into your application and get it displayed on screen. If the import is much slower than the copy, there is room for CPU based optimization to close that gap.neeravbm wrote: ↑Sat Mar 23, 2019 6:35 pm With multi-threaded applications, I/O will remain the bottleneck. In fact, if try to use multiple threads by breaking down the reading/writing of a huge file, it could turn out to be even slower since now the HDD's mechanical head needs to move more when switching between threads. Even SSDs, that don't have mechanical head, may be slower when read this way.
-
- V.I.P Member
- Posts: 1237
- Joined: Mon Jan 04, 2010 7:51 pm
- 14
- Full Name: Jed Frechette
- Company Details: Lidar Guys
- Company Position Title: CEO and Lidar Supervisor
- Country: USA
- Linkedin Profile: Yes
- Location: Albuquerque, NM
- Has thanked: 62 times
- Been thanked: 220 times
- Contact:
Re: Threadripper
I wouldn't underestimate the value of throwing lots of cores at a problem, even ones that require significant IO. This discussion reminded me of some simple test results I shared on this forum a few years ago.
viewtopic.php?f=88&t=10474&p=55862#p55810
The short version was that a multithreaded application accessing a very inefficient data store (text csv) was much faster than a single threaded application accessing an efficient data store (laz)
viewtopic.php?f=88&t=10474&p=55862#p55810
The short version was that a multithreaded application accessing a very inefficient data store (text csv) was much faster than a single threaded application accessing an efficient data store (laz)
Jed
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: Threadripper
Jed and Shanejedfrechette wrote: ↑Sun Mar 24, 2019 4:50 pm I wouldn't underestimate the value of throwing lots of cores at a problem, even ones that require significant IO. This discussion reminded me of some simple test results I shared on this forum a few years ago.
viewtopic.php?f=88&t=10474&p=55862#p55810
The short version was that a multithreaded application accessing a very inefficient data store (text csv) was much faster than a single threaded application accessing an efficient data store (laz)
I did a quick test with a recent 400 acre scanned Hawaiian jungle area using our VZ400i, stop and go, generating a 76GB LAS file (2.3B pts). I took one of the three scans (516M pts, 17GB LAS) and loaded it into our dual Xeon (E5-2637v4) workstation (256GB RAM, 16 threads @3.5GHz) and the same file into our i9-9980XE (128GB RAM, 36 [email protected]) workstation. I used Sequoia 1.163 in both cases to import and create the SPRT cache file and view the imported file on the screen at 4K. Both systems were running all threads at 100% most of the time(sorting and writing) except at the beginning during the read phase.
dual Xeon workstation took 6min:09sec to finish loading, creating the cache and displaying view.
i9-9980XE workstation took 4min:09sec to finish loading, creating the cache and displaying view.
I would expect more threads to be significantly faster, but it might be dividing the file into more smaller pieces which may be increasing the write time. As mentioned earlier, this would be dependent on the application, function, programmer approach.
I processed the same file to load and display it with another program and the application used 5% CPU of the dual Xeon and 100% of the GTX 1080. I quit at 175M pts displayed after 14min:19sec.
- Carbix
- V.I.P Member
- Posts: 236
- Joined: Sat Mar 16, 2019 1:36 am
- 5
- Full Name: Daniel Loney
- Company Details: Excelsior Measuring
- Company Position Title: Owner
- Country: Canada
- Linkedin Profile: Yes
- Location: Canada
- Has thanked: 33 times
- Been thanked: 50 times
- Contact:
Re: Threadripper
If your still using HDD’s you got bigger issues. SSD raid 0 striped or NVMe is the only way to go. Leave the HDD for your NAS or server backups.neeravbm wrote: ↑Sat Mar 23, 2019 6:35 pm With multi-threaded applications, I/O will remain the bottleneck. In fact, if try to use multiple threads by breaking down the reading/writing of a huge file, it could turn out to be even slower since now the HDD's mechanical head needs to move more when switching between threads. Even SSDs, that don't have mechanical head, may be slower when read this way.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: Threadripper
Hi Dennis,
Great work, those speeds are impressive. It would be interesting taking the same data into a range of different software and seeing the comparative performance. It is also worth considering that some software, e.g. potree and anything else building out of core data structures, compromise initial import speed to improve performance scalability across huge data
I've on a much older AMD 8350 box we use for automated testing here (8 threads, 4ghz, pre-Ryzen tech, 16GB ram, 720rpm HDD) currently sitting idle and tried something similar. I used a 680 million point LAZ job we received from a client a couple of weeks ago which was split over 60 files. It took 33 minutes to come into SCC resulting in a 7gb file that takes about 20 seconds to open subsequently. Same data on recap took 105 minutes, resulted in 19gb of storage on disk and opens again in about 20 seconds. It came into Cloudcompare in 16 minutes but crashed on the last file, which I'm guessing was probably due to running out of memory. Taking the first 100 million points in was also quick relatively speaking, but I couldn't figure out how to save it in a native format for faster re-opening. (Which probably says more about my lack of knowledge on CC than the software). When I get a chance I'll repeat the same with scene viewer, point tools and potree, unfortunately I don't have licenses to much else. I'd also like to repeat the process with E57 and possibly PTS at some point.
What the above tells me is that point cloud I/O is considerably more CPU bound than disk bound when translating from a point cloud data transfer format, yet opening native formats for any given program typically is not.
Great work, those speeds are impressive. It would be interesting taking the same data into a range of different software and seeing the comparative performance. It is also worth considering that some software, e.g. potree and anything else building out of core data structures, compromise initial import speed to improve performance scalability across huge data
I've on a much older AMD 8350 box we use for automated testing here (8 threads, 4ghz, pre-Ryzen tech, 16GB ram, 720rpm HDD) currently sitting idle and tried something similar. I used a 680 million point LAZ job we received from a client a couple of weeks ago which was split over 60 files. It took 33 minutes to come into SCC resulting in a 7gb file that takes about 20 seconds to open subsequently. Same data on recap took 105 minutes, resulted in 19gb of storage on disk and opens again in about 20 seconds. It came into Cloudcompare in 16 minutes but crashed on the last file, which I'm guessing was probably due to running out of memory. Taking the first 100 million points in was also quick relatively speaking, but I couldn't figure out how to save it in a native format for faster re-opening. (Which probably says more about my lack of knowledge on CC than the software). When I get a chance I'll repeat the same with scene viewer, point tools and potree, unfortunately I don't have licenses to much else. I'd also like to repeat the process with E57 and possibly PTS at some point.
What the above tells me is that point cloud I/O is considerably more CPU bound than disk bound when translating from a point cloud data transfer format, yet opening native formats for any given program typically is not.