System Benchmarking

To chat about anything else.
Dirkie2710
I have made 60-70 posts
I have made 60-70 posts
Posts: 61
Joined: Sun Jan 14, 2018 2:06 pm
6
Full Name: Dirkie Uys
Company Details: Landpartners
Company Position Title: Surveyor
Country: Australia
Skype Name: duaus
Linkedin Profile: No
Has thanked: 3 times
Been thanked: 10 times

Re: System Benchmarking

Post by Dirkie2710 »

Carbix wrote: Sat Feb 29, 2020 12:38 am This great data. My goal is to see what systems run this stuff the best. I wanted the file size to be something large enough to tax a system but small enough that we could run multiple runs and not take forever (we all have real work to do)

I'm very interested on how fast your system did the import. I have been using Cyclone Register 360. It looks like your using Cyclone "Original" (needs a better naming system).

One thing thats coming up to be very true is that the number of cores does not change the import time. Core speed is the big one.
Yeah I just used 10 cause it's easy to deal with :). I think also we might need to remember the firmware version. Pre-April 2019 data is much larger, so guessing less compressed so needs less de-compressing? Not sure if that'll make a difference at all.

We just got these machines actually. So nice and clean installs (we actually got them in December, but been in storeroom for 2 months while IT does their thing). The machines are i9-9900X with the Turbo Boost 3.0 function that is supposed to enable automatic higher clocks speeds of around 4.4Ghz but although enabled in the BIOS, Intel utility shows its disabled, and we are only getting 3.6Ghz.

And yes I have Cyclone 360 so I will get it installed and run a few tests when I have a gap.

Agreed on CPU clock speed. Seems to be the major limiting factor so far.
User avatar
Carbix
V.I.P Member
V.I.P Member
Posts: 236
Joined: Sat Mar 16, 2019 1:36 am
5
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 33 times
Been thanked: 50 times
Contact:

Re: System Benchmarking

Post by Carbix »

I'm going to run the tests again but with one of the guy's spreadsheets and with my system switched into "Gamer Mode". This mode turns of 3 of the 4 NUMA groups and runs on only 8 cores over 32. With this its able to overlock the 8 cores to 4.2ghz. This should give me a good idea of the change speed makes over cores.

I'm planning out my next benchmark file. Its a Frisbee park that loops through a forest. The idea is to scan it in low res with over 32 setups, no images. The goal is to see how well multi core systems deal with lots of setups and lots of noise for cleaning.

If you guys think this is a good idea I will go ahead with it.
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com
dhirota
V.I.P Member
V.I.P Member
Posts: 958
Joined: Sun Nov 01, 2009 11:18 pm
14
Full Name: Dennis Hirota
Company Details: Sam O Hirota Inc
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 87 times
Been thanked: 379 times

Re: System Benchmarking

Post by dhirota »

Carbix wrote: Sat Feb 22, 2020 1:02 am Wow that’s way faster. Which makes sense as my 2990wx does not get over 3.8ghz
As a test I downloaded Daniel Loney's data set and imported it into Cyclone Register 360 v2020.0.0 without any parameter setup from our imaging server (maximum disk I/O speed 261 MB/second; on a AVAGO raid 6 12Gbps SAS, 50TB system). The software is on our Ryzen 3990X + 1TB Inland PCIe4 M.2 rated at 5/4GBps, NO OVERCLOCKING. If you need details of the setup, go to the ThreadRipper thread

https://www.laserscanningforum.com/foru ... 9&start=60

6=3-26-2020 12-58-16 PM.jpg

1=R360-IMPORT-3-26-2020 11-06-57 AM.jpg
2=TM-3-26-2020 11-05-32 AM.jpg
3=CORE-W-3-26-2020 11-08-10 AM.jpg
4-CPU-5=3-26-2020 11-10-50 AM.jpg
5=BUNDLE-3-26-2020 11-28-04 AM.jpg

Import time= 7m:46s
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
www.samhirota.com
[email protected]
User avatar
Carbix
V.I.P Member
V.I.P Member
Posts: 236
Joined: Sat Mar 16, 2019 1:36 am
5
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 33 times
Been thanked: 50 times
Contact:

Re: System Benchmarking

Post by Carbix »

That’s sweet!!
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com
dhirota
V.I.P Member
V.I.P Member
Posts: 958
Joined: Sun Nov 01, 2009 11:18 pm
14
Full Name: Dennis Hirota
Company Details: Sam O Hirota Inc
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 87 times
Been thanked: 379 times

Re: System Benchmarking

Post by dhirota »

Carbix wrote: Tue Mar 03, 2020 6:48 am .....
I'm planning out my next benchmark file. Its a Frisbee park that loops through a forest. The idea is to scan it in low res with over 32 setups, no images. The goal is to see how well multi core systems deal with lots of setups and lots of noise for cleaning.

If you guys think this is a good idea I will go ahead with it.
It is an interesting concept of scanning at lower resolution, but the difference on our Riegl VZ400i between our lower resolution and what we normally scan at is 30 seconds vs 45 seconds (including images) for a scan position.

I think it would be better to test the best quality scans with images to see how good the system really is, with time to import, process and export. Is LGS improving their software to enable users to use available local multiple core processing to save resources from cloud based processing? I have not checked on the maximum number of cores available for processing using Register 360 2020.0.0, but it looks like 32 cores based on Daniel Loney's data. Someone else needs to check this out.
Dennis Hirota, PhD, PE, LPLS
www.samhirota.com
[email protected]
User avatar
Carbix
V.I.P Member
V.I.P Member
Posts: 236
Joined: Sat Mar 16, 2019 1:36 am
5
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 33 times
Been thanked: 50 times
Contact:

Re: System Benchmarking

Post by Carbix »

dhirota wrote: Fri Mar 27, 2020 8:00 pm
Carbix wrote: Tue Mar 03, 2020 6:48 am .....
I'm planning out my next benchmark file. Its a Frisbee park that loops through a forest. The idea is to scan it in low res with over 32 setups, no images. The goal is to see how well multi core systems deal with lots of setups and lots of noise for cleaning.

If you guys think this is a good idea I will go ahead with it.
It is an interesting concept of scanning at lower resolution, but the difference on our Riegl VZ400i between our lower resolution and what we normally scan at is 30 seconds vs 45 seconds (including images) for a scan position.

I think it would be better to test the best quality scans with images to see how good the system really is, with time to import, process and export. Is LGS improving their software to enable users to use available local multiple core processing to save resources from cloud based processing? I have not checked on the maximum number of cores available for processing using Register 360 2020.0.0, but it looks like 32 cores based on Daniel Loney's data. Someone else needs to check this out.
The idea for the park scan at low rez is to keep the file sample size small and to see how much having many cores (lower clock speed) vs fewer cores (high clock speed) will change things. Trying to keep the sample data under 50gb. In the end I want to see the difference 8-12 cores at 5ghz has on 32 at 4.1-4.5ghz.

Some of the steps Register 360 uses will assign multiple cores. For instance it uses all cores when doing setup point cloud cleaning. Its hard to see the value when we are looking at only 5-12 setups and I need the sample set to not be 500gb.

I also want the test to be less than an hour. target is 5-20min
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com
dhirota
V.I.P Member
V.I.P Member
Posts: 958
Joined: Sun Nov 01, 2009 11:18 pm
14
Full Name: Dennis Hirota
Company Details: Sam O Hirota Inc
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 87 times
Been thanked: 379 times

Re: System Benchmarking

Post by dhirota »

Carbix wrote: Sat Mar 28, 2020 7:58 am .....
The idea for the park scan at low rez is to keep the file sample size small and to see how much having many cores (lower clock speed) vs fewer cores (high clock speed) will change things. Trying to keep the sample data under 50gb. In the end I want to see the difference 8-12 cores at 5ghz has on 32 at 4.1-4.5ghz.

Some of the steps Register 360 uses will assign multiple cores. For instance it uses all cores when doing setup point cloud cleaning. Its hard to see the value when we are looking at only 5-12 setups and I need the sample set to not be 500gb.

I also want the test to be less than an hour. target is 5-20min
I decided to try my one-and-only RTC360 scan in Register360 v2020.0.0, using a "borrowed" unit on December 3, 2018 with very little instruction and no help on that day to set anything up. We had only 30 minutes in the building to scan using our NavVIS M3 and the RTC360, so the NavVIS scanned the entire building in 33 minutes (1656 images, 275 panoramas, 561.9 Mpts) and the RTC360 covered as much of the building in 15 scan positions at 2 minutes each for 525.7 Mpts. The comparison of these two sensors for another time if anyone is interested.

The RTCstore file was 38.5GB, probably larger than a RTCstore file today. I stored the RTCstore file on the Inland NVMe stick (5/4+ GBps) to get the best I/O on our Ryzen 3990X + 256GB RAM + RTX2080Ti GPU + 10GbE (not used) + DeepCool 360X cooler, NO OVERCLOCKING.

1=3-27-2020 4-14-43 PM.jpg
2=3-27-2020 4-15-32 PM.jpg
4=3-27-2020 4-27-21 PM.jpg
5=3-27-2020 4-28-58 PM.jpg
6=3-27-2020 4-35-18 PM.jpg
7=3-27-2020 4-42-49 PM.jpg
8=3-27-2020 4-47-01 PM.jpg
9=3-27-2020 4-50-20 PM.jpg

The color in the last attachment fitting is not brown, but red. I am sure that has been corrected in the last year?


Import time = 21m:57s
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
www.samhirota.com
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 958
Joined: Sun Nov 01, 2009 11:18 pm
14
Full Name: Dennis Hirota
Company Details: Sam O Hirota Inc
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 87 times
Been thanked: 379 times

Re: System Benchmarking

Post by dhirota »

Phil's comment
viewtopic.php?f=88&t=12689&start=70
Phil Marsh wrote: Sun Mar 29, 2020 9:12 am If that’s RTC360 data you shouldn’t need to have AutoCloud ticked on import :geek:
Phil

Since I do not pretend to be a Register 360 expert, I just installed the 2020.0.0 version and dropped in Daniel Loney's KingsWay and my only RTC360 B137 data sets and clicked import. Since you mentioned it, I decided to run Daniel's data set with and without the autocloud tick on import on our PC147.

PC147
i9-9980XE (18-cores/36-threads, OVERCLOCKED all Cores to 4.2GHz)) + 128GB RAM + RTX2080Ti

Kings Way with Autocloud tick = 8m:29s
Kings Way w/out Autocloud tick = 8m:29s
B137 with Autocloud tick = 30m:46s

PC166
Ryzen 3990X (64-cores/128-threads, NOT OVERCLOCKED) + 256GB RAM + RTX 2080Ti

Kings Way with Autocloud tick = 7m:46s
B137 with Autocloud tick = 21m:57s

There are probably parameters in the setup that I have not checked on this latest version of Register 360, so I have been consistent on running it for these benchmarks.
Dennis Hirota, PhD, PE, LPLS
www.samhirota.com
[email protected]
dhirota
V.I.P Member
V.I.P Member
Posts: 958
Joined: Sun Nov 01, 2009 11:18 pm
14
Full Name: Dennis Hirota
Company Details: Sam O Hirota Inc
Company Position Title: President
Country: USA
Linkedin Profile: Yes
Location: Hawaii, USA
Has thanked: 87 times
Been thanked: 379 times

Re: System Benchmarking

Post by dhirota »

Happy Easter for those that are celebrating. The Easter Bunny has brought me a "borrowed" BLK2GO since I was unable to test it several months ago. Because of the CORONAVirus and the local politicians, everything is shutdown, so we can not scan some of the more interesting locations in HNL. The only two locations are Hawaii Loa Ridge (HLR, where I live) and my office in downtown HNL.

For those folks that have not visited me, we are at an elevation of 751 feet above the Pacific Ocean, 4 doors from the HLR park, shown below.

1=4-12-2020 10-53-16 AM.jpg


I will be posting this information twice, since there are some that are interested in the system benchmarking with the details, whereas it does not seem that anyone on the BLK2GO thread is that interested in posting any detail.

Here are the basics: 3 scans. I did 5, but since this is the first significant try (I actually did a self-test on Friday), which needed improvement, 2 scans messed up. I used the most recent version of Register360 (2020.0.0), BLK2GO without VIS and no paper target recognition.

LESSON LEARNED: Multiple scans are better to get maximum coverage, since if you have maximum overlap, you hopefully will not miss anything if scans mess up.


Way Point spacing (they say default is 10m), I used 20m for the outdoors and trees, maybe 10 is better for outside as well.

LESSON LEARNED: The Register360 import processing totally sucks for BLK2GO. The import uses virtually no CPU or RAM If you are trying to figure out how long this process will take. It took 21min:30sec on a 64-core, 256GB RAM processor, with the CPU at 4% and RAM at 4 to 6GB. The processing output looked significantly better than earlier point clouds on planes and walls that I have seen which looked like vertical spaghetti. There are other filtering tools in Register360 that I did not use because of time constraints (I can do it later).

LESSON LEARNED: Be sure to save raw data sets, so if you need to process it with other datasets, so you will not get stuck having your BLK2GO glued by a USB-C cable to your processor.

2=4-12-2020 9-38-29 AM.jpg
3=4-12-2020 9-28-02 AM.jpg
4=4-12-2020 10-43-34 AM.jpg
5=4-12-2020 9-39-57 AM.jpg
6=4-12-2020 10-47-04 AM.jpg
7=04-12-2020 03-56-22 PM.jpg

Just finished the exterior of our office property and the properties on either side
of us with a Cloud Registration of 5.4mm Absolute Mean, whatever that means. We will be comparing
the BLK2GO with our other sensors (VZ400i, Z+F5010X, BLK360, NavVIS M6) to see how they can used in parallel using conventional paper targets as well as unorthodox scanning approaches.
You do not have the required permissions to view the files attached to this post.
Dennis Hirota, PhD, PE, LPLS
www.samhirota.com
[email protected]
Dirkie2710
I have made 60-70 posts
I have made 60-70 posts
Posts: 61
Joined: Sun Jan 14, 2018 2:06 pm
6
Full Name: Dirkie Uys
Company Details: Landpartners
Company Position Title: Surveyor
Country: Australia
Skype Name: duaus
Linkedin Profile: No
Has thanked: 3 times
Been thanked: 10 times

Re: System Benchmarking

Post by Dirkie2710 »

Carbix wrote: Fri Feb 21, 2020 4:47 am Ok here we go

https://www.dropbox.com/transfer/AAAAAI ... K-2uS0qkBQ

The Reg store file is in the folder.
Hi Daniel,

any chance you still have this dataset on hand?

I have a feeling my system has slowed down considerably the last 14 months and would like to test using the same dataset and see if time are the same. I imported scans yesterday that was done on MED res, images importing 4096 cubes, and took nearly 4 hours to import 45 scans. That's 5m30s per scan, while back when I ran these tests on your data I was getting around 1m40s per scan. Only change has been from cyclone 9.4.2 to cyclone 2021.0.0.

Thanks
Post Reply

Return to “General Chat”