Data cleaning...what is the best method?

Discuss all Leica Cyclone, Cyclone REGISTER 360 & Cyclone FIELD 360 software here.
User avatar
James Hall
V.I.P Member
V.I.P Member
Posts: 220
Joined: Tue Feb 02, 2010 5:13 pm
14
Full Name: James E Hall
Company Details: Dewberry Engineering Inc
Company Position Title: Survey Technician - Cyclone Modeler
Country: USA
Location: Frederick, MD
Has thanked: 5 times
Been thanked: 37 times

Re: Data cleaning...what is the best method?

Post by James Hall »

dirtysouthsurveyor wrote: Thu Oct 04, 2018 7:35 pm When you say "compact" are you referring to optimizing the database?

I had a 3 mile stretch of an extremely busy roadway that we scanned at night. It was a nightmare cleaning that thing up with region grow because of how quickly it would break the unification. I ended up sectioning off parts of the road with the limit box, flip to ortho profile few then fenced in everything approximately 0.1' above the surface and deleted inside fence.

In retrospect, i wouldn't have brought in every day of data into the same database. Definitely going to keep future projects in separate databases and then clean them as I go. Just not sure of how to register the clean scanworlds together...
A way and not the most elegant, is to select the the cleaned data and export them to an e57 file. Then re-import them (make sure to calculate normals) and registered the cleaned scans. This is also a way to fix a problem scan world.

James,
Origin.KR
I have made 10-20 posts
I have made 10-20 posts
Posts: 15
Joined: Wed Jan 17, 2018 8:15 pm
6
Full Name: Kyle Rothwell
Company Details: Origin 4n6
Company Position Title: Tech
Country: usa
Linkedin Profile: Yes
Been thanked: 1 time

Re: Data cleaning...what is the best method?

Post by Origin.KR »

I have found the Cloth Simulation Fliter used in CloudCompare to be very useful to isolate noise generated from passing traffic.

http://www.cloudcompare.org/doc/wiki/in ... F_(plugin)
Jonny5isAlive
I have made 50-60 posts
I have made 50-60 posts
Posts: 53
Joined: Thu Feb 18, 2010 5:01 pm
14
Full Name: Jon Adams
Company Details: Forensic and Architectural Consulting
Company Position Title: Director of 3D Reality Capture
Country: USA
Linkedin Profile: Yes
Location: PA

Re: Data cleaning...what is the best method?

Post by Jonny5isAlive »

Thank you all for your responses!

It sounds like there is still yet room for improvement inside of Cyclone (i.e. keep all the data there, no export/process/reimport) to address the extremely slow process of cleaning scans.

Region grow is nice and does a decent job, but I look forward to a time when selections are made instantly, and deletions as well.

The waiting for these two items is mind-numbingly painful.

I hope something comes soon.

Sent from my SM-N950U using Tapatalk

Yuelin
I have made 90-100 posts
I have made 90-100 posts
Posts: 99
Joined: Sat Mar 19, 2011 5:16 am
13
Full Name: Yuelin
Company Details: Leica Geosystems
Company Position Title: Lead software engineer
Country: USA
Linkedin Profile: No
Has thanked: 1 time
Been thanked: 11 times

Re: Data cleaning...what is the best method?

Post by Yuelin »

Hi Jon,

Regarding to data cleaning, please read my recent posting here: https://laserscanningforum.com/forum/vi ... 984#p73836
Jonny5isAlive
I have made 50-60 posts
I have made 50-60 posts
Posts: 53
Joined: Thu Feb 18, 2010 5:01 pm
14
Full Name: Jon Adams
Company Details: Forensic and Architectural Consulting
Company Position Title: Director of 3D Reality Capture
Country: USA
Linkedin Profile: Yes
Location: PA

Re: Data cleaning...what is the best method?

Post by Jonny5isAlive »

Yuelin wrote:Hi Jon,

Regarding to data cleaning, please read my recent posting here: https://laserscanningforum.com/forum/vi ... 984#p73836
Yes, but even with this process, cleaning is a slow affair.

Starts out relatively quick, but the more edits made the slower the process becomes.

Compact/optimize helps some when slow downs occur, but overall selections/deletions are far from instantaneous (which would be ideal).



Sent from my SM-N950U using Tapatalk

User avatar
Matt Young
Honorary Member
Honorary Member
Posts: 3929
Joined: Fri Nov 30, 2007 3:03 pm
16
Full Name: Matt Young
Company Details: Baker Hicks
Company Position Title: CAD-BIM Lead
Country: UK
Linkedin Profile: No
Has thanked: 45 times
Been thanked: 40 times

Re: Data cleaning...what is the best method?

Post by Matt Young »

When you are working with very large datasets, instant delete is unlikely to be a thing for a while yet. It's like someone telling you to open a thousand page novel and manually remove every occurrence of a single word. A human could do that but it would take a very long time. Now make the book a point cloud with hundreds of millions of points. The computer has to read all the points, decide which ones are to be hidden (Cyclone does not delete points) and then hide them. When you optimize and compress your database, the points are deleted and the database decreased in size to compensate for the new point count. This is like you re-writing the thousand page book, so that no spaces occur where you removed all the words.
If you don't see that there is nothing, then you are kidding yourself.
Post Reply

Return to “Leica Cyclone, Cyclone REGISTER 360 & Cyclone FIELD 360”