System Benchmarking

To chat about anything else.
User avatar
James Hall
V.I.P Member
V.I.P Member
Posts: 198
Joined: Tue Feb 02, 2010 5:13 pm
Full Name: James E Hall
Company Details: Dewberry Engineering Inc
Company Position Title: Survey Technician - Cyclone Modeler
Country: USA
Location: Frederick, MD
Has thanked: 4 times
Been thanked: 29 times

Re: System Benchmarking

Post by James Hall »

Carbix wrote:
Wed Feb 19, 2020 2:45 am
I think we need to MVP this thing (minimum viable product). See how we all like it and move from there.

Should we start with raw hardware scans or say an e57 (or other format) that is not cleaned in any way. I personal think we should come up with a library of hardware scans that we all agree on. 3-5 scans with images if applicable. Should be of something interesting. I have about 30tb of raw RTC scans that I can pick from.

Lets make a list of the most commonly used scanners that put out a unique file. for instance an p30, p40, p50 all put out roughly (don't hold me to this) the same files. I would assume this is the same for most of the FARO line. The BLK360 might be a little different. I don't know what the C10 puts out.

I think each scan environment should be relative to what the hardware was designed for. No small room scans with a Pxx series. BLK360 should be small and close. RTC360 could be a mix of mid range to inside scans (I'm curious to see how foliage will affect this kind of stuff).

I have BLK360 and RTC360 Data. Who has C10, Pxx and FARO?
Sounds good.
Working up a test set 3-5 scans as a raw RTC and the same scans as an .e57 is a good place to start.
We would need reporting criteria.
Time to import
Time to copy to back up
ect.
Also What bus is being used for storage, CPU, RAM, OS general config data in a standardized format.
An Excel work sheet would do for now. Automation and additional Data sets can come later.

The BLK360 out puts a .BLK file and the Raw file for the C-10 is a folder you path the import to. I'll look into it tomorrow.

How to we want to share data?

James,

User avatar
smacl
Global Moderator
Global Moderator
Posts: 361
Joined: Tue Jan 25, 2011 5:12 pm
Full Name: Shane MacLaughlin
Company Details: Atlas Computers Ltd
Company Position Title: Managing Director
Country: Ireland
Linkedin Profile: Yes
Location: Ireland
Has thanked: 91 times
Been thanked: 110 times
Contact:

Re: System Benchmarking

Post by smacl »

Carbix wrote:
Wed Feb 19, 2020 2:45 am
I think we need to MVP this thing (minimum viable product). See how we all like it and move from there.

Should we start with raw hardware scans or say an e57 (or other format) that is not cleaned in any way. I personal think we should come up with a library of hardware scans that we all agree on. 3-5 scans with images if applicable. Should be of something interesting. I have about 30tb of raw RTC scans that I can pick from.

Lets make a list of the most commonly used scanners that put out a unique file. for instance an p30, p40, p50 all put out roughly (don't hold me to this) the same files. I would assume this is the same for most of the FARO line. The BLK360 might be a little different. I don't know what the C10 puts out.

I think each scan environment should be relative to what the hardware was designed for. No small room scans with a Pxx series. BLK360 should be small and close. RTC360 could be a mix of mid range to inside scans (I'm curious to see how foliage will affect this kind of stuff).

I have BLK360 and RTC360 Data. Who has C10, Pxx and FARO?
This sounds really good Daniel. If the data is in one or more of the open formats I can put together some benchmark software based on open source reference implementations for E57, LAS, and LAZ to read and translate. I should be able to include FLS and RCP as well thinking about it. This should cover relative disk read/write performance for PCs reasonably well and potentially relative processing speeds for the same data in different formats. I have a number of users with mobile LIDAR so I'll ask around if they've any big data that can be placed in the public domain. Dealing with some very large jobs at the moment but the data is confidential as I guess most jobs are.

Lieven Beuckx has been posting some great looking terrestrial Riegl data recently and may have some sample data there. I've some Topcon samples but they're not big enough to stress any half way decent PC.

MikeDailey
I have made 20-30 posts
I have made 20-30 posts
Posts: 22
Joined: Wed Dec 12, 2018 5:56 pm
Full Name: Mike Dailey
Company Details: Survey work
Company Position Title: Office Tech
Country: United States
Been thanked: 3 times

Re: System Benchmarking

Post by MikeDailey »

Carbix wrote:
Wed Feb 19, 2020 2:45 am
I think each scan environment should be relative to what the hardware was designed for. No small room scans with a Pxx series. BLK360 should be small and close. RTC360 could be a mix of mid range to inside scans (I'm curious to see how foliage will affect this kind of stuff).

I have BLK360 and RTC360 Data. Who has C10, Pxx and FARO?
I think you may be over thinking this a bit. I would suggest we look at file size for the created project from the scanner and the number of points contained in said project. Then we would also learn what manufactures are doing the best to condense the data with their own file type.

We will learn from the test, but I feel safe in assuming that the only difference that interior or exterior scans would have for bench-marking purposes is file size. Full dome scans outside will not get a return for the sky while interior scans would get returns on the majority of measurements, making these setups have larger file sizes.

User avatar
Carbix
I have made 90-100 posts
I have made 90-100 posts
Posts: 91
Joined: Sat Mar 16, 2019 1:36 am
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 8 times
Been thanked: 9 times
Contact:

Re: System Benchmarking

Post by Carbix »

MikeDailey wrote:
Wed Feb 19, 2020 12:20 pm
Carbix wrote:
Wed Feb 19, 2020 2:45 am
I think each scan environment should be relative to what the hardware was designed for. No small room scans with a Pxx series. BLK360 should be small and close. RTC360 could be a mix of mid range to inside scans (I'm curious to see how foliage will affect this kind of stuff).

I have BLK360 and RTC360 Data. Who has C10, Pxx and FARO?
I think you may be over thinking this a bit. I would suggest we look at file size for the created project from the scanner and the number of points contained in said project. Then we would also learn what manufactures are doing the best to condense the data with their own file type.

We will learn from the test, but I feel safe in assuming that the only difference that interior or exterior scans would have for bench-marking purposes is file size. Full dome scans outside will not get a return for the sky while interior scans would get returns on the majority of measurements, making these setups have larger file sizes.
I would agree with you. But we might as well use data the way it was intended for.
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com

User avatar
Carbix
I have made 90-100 posts
I have made 90-100 posts
Posts: 91
Joined: Sat Mar 16, 2019 1:36 am
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 8 times
Been thanked: 9 times
Contact:

Re: System Benchmarking

Post by Carbix »

Ok so I think I have a good first try at a test file. Its a roof top we did. So its kind of fun.

It was done with an RTC360, HighRes, with images.

It took me 25:05min to import and 4:41 sec to export.

Im currently uploading the files I used to my Dropbox High speed transfer.

I used an AMD 2990wx
SystemSum.png
This system needs some fine tuning as I'm sure the ram is working correctly. The ram should be posting at 2993mhz. Also the temps are fine. AMD just looks hot. its not. True temp is 53c.

Imported Setting: No features used clean import. Images imported at full res. I believe that the RTC and Pxx cleaning options are applied at export.
I imported the files from my C Drive (NVMe 970 Evo 1TB) to the Z Drive (2x NVMe 970 Evo Plus 2tb in windows raid 0 stripped run from an ASUS HyperX PCi Card).

Export Settings: Full RTC clean with e57
I Exported from Z to D: (NVMe 970 Evo 500gb)

Most of the time only 2-3 cores where being taxed. For the export I think we will next to come up with a test that has over 64 setups as Register 360 uses all the cores to clean the export.

I will run an import to Recap next.
You do not have the required permissions to view the files attached to this post.
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com

JClay15
I have made <0 posts
I have made <0 posts
Posts: 5
Joined: Wed Aug 22, 2018 12:30 am
Full Name: Jeffrey Clay
Company Details: TWM
Company Position Title: Reality Capture Specialist
Country: United States
Linkedin Profile: Yes

Re: System Benchmarking

Post by JClay15 »

Carbix wrote:
Fri Feb 21, 2020 2:27 am
Ok so I think I have a good first try at a test file. Its a roof top we did. So its kind of fun.

It was done with an RTC360, HighRes, with images.

It took me 25:05min to import and 4:41 sec to export.

Im currently uploading the files I used to my Dropbox High speed transfer.

I used an AMD 2990wx

SystemSum.png

This system needs some fine tuning as I'm sure the ram is working correctly. The ram should be posting at 2993mhz. Also the temps are fine. AMD just looks hot. its not. True temp is 53c.

Imported Setting: No features used clean import. Images imported at full res. I believe that the RTC and Pxx cleaning options are applied at export.
I imported the files from my C Drive (NVMe 970 Evo 1TB) to the Z Drive (2x NVMe 970 Evo Plus 2tb in windows raid 0 stripped run from an ASUS HyperX PCi Card).

Export Settings: Full RTC clean with e57
I Exported from Z to D: (NVMe 970 Evo 500gb)

Most of the time only 2-3 cores where being taxed. For the export I think we will next to come up with a test that has over 64 setups as Register 360 uses all the cores to clean the export.

I will run an import to Recap next.
RTC and Pxx filtering is done at import, not export, so let us know your settings on that.

Can you elaborate what you mean by full RTC clean with e57? You’re running publish with an e57 export as well?
Do you have a download link to your Dropbox?

User avatar
Carbix
I have made 90-100 posts
I have made 90-100 posts
Posts: 91
Joined: Sat Mar 16, 2019 1:36 am
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 8 times
Been thanked: 9 times
Contact:

Re: System Benchmarking

Post by Carbix »

RTC360 Filtering Options:
Remove Intensity Overloaded Points - On
Remove Ambiguous Points - On
Remove Mixed-Pixel Points - On
Mixed Pixel Filter - High
Link will be up shortly.
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com

User avatar
Carbix
I have made 90-100 posts
I have made 90-100 posts
Posts: 91
Joined: Sat Mar 16, 2019 1:36 am
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 8 times
Been thanked: 9 times
Contact:

Re: System Benchmarking

Post by Carbix »

Ok here we go

https://www.dropbox.com/transfer/AAAAAI ... K-2uS0qkBQ

The Reg store file is in the folder.
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com

MikeDailey
I have made 20-30 posts
I have made 20-30 posts
Posts: 22
Joined: Wed Dec 12, 2018 5:56 pm
Full Name: Mike Dailey
Company Details: Survey work
Company Position Title: Office Tech
Country: United States
Been thanked: 3 times

Re: System Benchmarking

Post by MikeDailey »

IMP BM2.JPG
Total Project time: 23min, 22sec
Import: 12min, 45sec
Process time(link setups and batch adjust and apply pano): 9min, 02sec
Export to E57: 1min, 35sec

Ran entire project from one PCIe NVMe drive. Reading and writing to same drive
IMP BM1.JPG
IMP BM3.JPG
IMP BM4.JPG
You do not have the required permissions to view the files attached to this post.

User avatar
Carbix
I have made 90-100 posts
I have made 90-100 posts
Posts: 91
Joined: Sat Mar 16, 2019 1:36 am
Full Name: Daniel Loney
Company Details: Excelsior Measuring
Company Position Title: Owner
Country: Canada
Linkedin Profile: Yes
Location: Canada
Has thanked: 8 times
Been thanked: 9 times
Contact:

Re: System Benchmarking

Post by Carbix »

Wow that’s way faster. Which makes sense as my 2990wx does not get over 3.8ghz
Daniel Loney - Owner
Excelsior Measuring inc.
Vancouver - Okanagan - Calgary
www.ExcelsiorLevel.com

Post Reply

Return to “General Chat”