fast storage solution

CAD Workstations, Network Attached Storage [NAS], Servers, Monitors, SSD`s, Imaging Systems, Field Kit: Targets & Tripods etc.
Post Reply
User avatar
landmeterbeuckx
V.I.P Member
V.I.P Member
Posts: 1615
Joined: Tue May 01, 2012 5:19 pm
11
Full Name: Lieven Beuckx
Company Details: Studiebureau Beuckx
Company Position Title: Owner
Country: Belgium
Linkedin Profile: Yes
Has thanked: 183 times
Been thanked: 548 times

fast storage solution

Post by landmeterbeuckx »

Hello,
The amount of data on my system is going crazy. I'm searching for a system to work with for all my scanning needs.

Currently i'm working on the system drives C and D of my main pc but it gets full rather fast and i'm constantly moving files along.

Would this https://www.synology.com/nl-nl/products/FS1018#features system be something to work on with current projects and move to backup later on?
LSBbvba
Surveying services - 3D Laserscanning
Tel : +32477753126
www.lsbbvba.be
[email protected]
User avatar
stevenramsey
Global Moderator
Global Moderator
Posts: 1937
Joined: Sun Aug 12, 2007 9:22 pm
16
Full Name: Steven Ramsey
Company Details: 4DMax
Company Position Title: Technical Specialist Scanning
Country: UK
Skype Name: steven.ramsey
Linkedin Profile: Yes
Location: London
Has thanked: 30 times
Been thanked: 72 times
Contact:

Re: fast storage solution

Post by stevenramsey »

looks great add the 10gb card and one for the pc or that will be a big bottle neck without
Steven Ramsey

Home [email protected]
Work [email protected]
Mobile +44 7766 310 915
User avatar
landmeterbeuckx
V.I.P Member
V.I.P Member
Posts: 1615
Joined: Tue May 01, 2012 5:19 pm
11
Full Name: Lieven Beuckx
Company Details: Studiebureau Beuckx
Company Position Title: Owner
Country: Belgium
Linkedin Profile: Yes
Has thanked: 183 times
Been thanked: 548 times

Re: fast storage solution

Post by landmeterbeuckx »

the problem is also the price of ssd's. it would cost a fortune to fill it up with just 1TB ones
LSBbvba
Surveying services - 3D Laserscanning
Tel : +32477753126
www.lsbbvba.be
[email protected]
User avatar
blazaj
V.I.P Member
V.I.P Member
Posts: 510
Joined: Tue Jan 26, 2010 1:07 pm
14
Full Name: Jiri Blazek
Company Details: BW precision systems sro
Company Position Title: Managing Director
Country: Czech Republic
Linkedin Profile: No
Has thanked: 45 times
Been thanked: 18 times
Contact:

Re: fast storage solution

Post by blazaj »

I don't think it is recommended to put SSD's in a NAS server, but you better consult an IT expert. I have a 512 GB fast SSD for actual projects and 4Gig NAS on gigabyte LAN filled with 3 HDD's and an external one for backup. It is reasonably fast to move projects around. If I was to upgrade I would go for 1GB SSD for actual projects. I had a couple of projects where there was not sufficient space for registration in Faro Scene, a sole project on the 512GB SSD! It is crazy.
User avatar
JohnT2
I have made 40-50 posts
I have made 40-50 posts
Posts: 45
Joined: Mon Feb 17, 2014 6:33 pm
10
Full Name: John Tocci Jr
Company Details: PCL Construction
Company Position Title: VDC Manager
Country: United States
Linkedin Profile: Yes
Been thanked: 1 time

Re: fast storage solution

Post by JohnT2 »

SSDs in a synology are a supported solution, both as a RAID array, and solid state caching.

Depending upon your current IT deployment, look at both the rack mounted solution (RS series) and the desktop solution (DS series).

The way we run our Synologies is with a Fast IOPS active projects array, and much slower archive "cold storage" array.

To really get the most out of pure-SSD storage array, you'll need a 10 GBE network topology (NAS, switch, network adapters in client machines/servers/NAS, and compatible cables)

Today (July 2018), We've got our users working with local M.2 NVME drives with Microsoft OneDrive actively syncing the projects (Office 365 E5 has unlimited onedrive storage for users), and our Synology running our project drive and archive storage. We've got a gigabit internet pipe, courtesy Verizon FIOS and onedrive bursts very nicely.

We're holding off on a 10GBE network environment for at least the next 12-18 months, since our VDC department operates off laptops, and there isn't a good 10GBE solution there (yet).

If you're on desktops & workstations, that might change the cost/benefit study.
User avatar
gsisman
V.I.P Member
V.I.P Member
Posts: 898
Joined: Fri Oct 07, 2016 1:51 pm
7
Full Name: Steve Long
Company Details: Montgomery County DOT _ MD
Company Position Title: Land Survey Supervisor
Country: United States
Skype Name: gsisman1
Linkedin Profile: Yes
Has thanked: 767 times
Been thanked: 149 times

Re: fast storage solution

Post by gsisman »

JohnT2 wrote: Thu Jul 26, 2018 4:53 pm SSDs in a synology are a supported solution, both as a RAID array, and solid state caching.

Depending upon your current IT deployment, look at both the rack mounted solution (RS series) and the desktop solution (DS series).

The way we run our Synologies is with a Fast IOPS active projects array, and much slower archive "cold storage" array.

To really get the most out of pure-SSD storage array, you'll need a 10 GBE network topology (NAS, switch, network adapters in client machines/servers/NAS, and compatible cables)

Today (July 2018), We've got our users working with local M.2 NVME drives with Microsoft OneDrive actively syncing the projects (Office 365 E5 has unlimited onedrive storage for users), and our Synology running our project drive and archive storage. We've got a gigabit internet pipe, courtesy Verizon FIOS and onedrive bursts very nicely.

We're holding off on a 10GBE network environment for at least the next 12-18 months, since our VDC department operates off laptops, and there isn't a good 10GBE solution there (yet).

If you're on desktops & workstations, that might change the cost/benefit study.
We are getting ready to provision a second Synolgy NAS for Scan cloud operating, editing, and short term storage on a 1GB network
Our first one was 48TB (divided into 10GB iSCI and 38GB partitions) and is running a RAID5 configuration across 8 6GB HDD
We do a partial dailt backup using Synology Cloud Backup app to Azure.

Our new one will be a bit bigger 8 10TB (7200rpm HDD.)
We would really like to take advantage of the 10GB Network capability of the NAS to at least one of our Workstations, and I'm assuming that we need a 10TB Switch to handle that connection, even if we are only hooking up one station right now, possible 2-4 more in the future

Just wondering whether there is a better solution than RAID 5 for ultimate READ/WRITE Performance balance.
At the present time we are serving up our Jetstream from this NAS, working our project directory on it, and storing our RAW Data on it. The iSCI portion we are using for storing LGS portable files and RAF Register 360 Archive projects.
When we add the second NAS should we divide the purposes and use one NAS for operation/Editing Projects, and one for Jetstream service and RAW Storage?
Should we have different RAID on each system depending on how it will be most used
....Jetstream or LGS storage mostly all READ
....RAW storage and Project editing Read/Write
User avatar
Leandre Robitaille
V.I.P Member
V.I.P Member
Posts: 542
Joined: Sat Aug 03, 2019 1:53 am
4
Full Name: Leandre Robitaille
Company Details: Cima+
Company Position Title: Civil Technician - Surveyor
Country: Canada
Has thanked: 61 times
Been thanked: 246 times

Re: fast storage solution

Post by Leandre Robitaille »

This is the biggest issue with 3d scanning. We have filled our servers with over 100tb worth of data(we keep only 5 years worth, as per stated in all our quotes). I have several Scene projects that do by themselves over 1tb (usually get 1tb when 2000+scans)

The way we work internally is that everyone has a local ssd on their computer, our whole server setup is 7200rpm (40tb available, active project, raid10)

then we have 60tb in archives, those are slower 5200rpm.

Our workstations have a m.2 controller that plugs in 4x1tb m.2 ssd in raid 0. We use those for our Scene projects /recap etc. Then when finished we drop the files on our 7200rpm server and strongly suggest to anyone using the data to copy it onto their local ssd. In short, dont get a ssd server it is not worth the money. With hdd you can always do data recovery if many drive fails. SSD when it dies its over.

I'm sadly not a IT, simply a surveyor, but I know about this cause im the source for all the headaches my IT team has haha.
brian715
V.I.P Member
V.I.P Member
Posts: 199
Joined: Wed Apr 06, 2016 3:24 pm
7
Full Name: Brian Boehmer
Company Details: Meridian Eng.
Company Position Title: LiDAR Technician
Country: United States
Linkedin Profile: No
Has thanked: 14 times
Been thanked: 5 times

Re: fast storage solution

Post by brian715 »

We now have two machines with 4tb SSD's internal for our current projects. We found that if we had a large project, everything wouldn't fit on our 2TB SSD with PPC, Scene2Go, Raw Scans, Exported e57's, Recap Project and RCS files. This gets backed up to a NAS drive and the projects go onto 8tb backup drives for permanent storage. Drives are cheap compared to a client calling several years later with more extraction work for you and you having to tell them you deleted it because it was taking up too much space.
I occasionally work on projects located on the NAS without much problem. Just don't try to do any new registrations or anything. Its simply extraction from a loaded point cloud.
jedfrechette
V.I.P Member
V.I.P Member
Posts: 1237
Joined: Mon Jan 04, 2010 7:51 pm
14
Full Name: Jed Frechette
Company Details: Lidar Guys
Company Position Title: CEO and Lidar Supervisor
Country: USA
Linkedin Profile: Yes
Location: Albuquerque, NM
Has thanked: 62 times
Been thanked: 220 times
Contact:

Re: fast storage solution

Post by jedfrechette »

gsisman wrote: Wed Jan 29, 2020 6:10 pm We would really like to take advantage of the 10GB Network capability of the NAS to at least one of our Workstations, and I'm assuming that we need a 10TB Switch to handle that connection, even if we are only hooking up one station right now
If it's a just a small number of workstations you could always just cable them directly without a switch. This is what we do between our primary NAS and the backup NAS that it replicates to, so that we can keep the replication traffic off of the main network.
gsisman wrote: Wed Jan 29, 2020 6:10 pm Just wondering whether there is a better solution than RAID 5
After switching to ZFS based central storage I would never go back to simple RAID. ZFS is the closest thing to magic (in a good way) I've come across in a while.
Jed
User avatar
Matt Young
Honorary Member
Honorary Member
Posts: 3929
Joined: Fri Nov 30, 2007 3:03 pm
16
Full Name: Matt Young
Company Details: Baker Hicks
Company Position Title: CAD-BIM Lead
Country: UK
Linkedin Profile: No
Has thanked: 45 times
Been thanked: 40 times

Re: fast storage solution

Post by Matt Young »

Just got hold of a 'My Cloud WD Pro Series 8Tb NAS'. It transfers data quickly and is a useful single location for things. Very easy to set up as a mapped or network drive.
If you don't see that there is nothing, then you are kidding yourself.
Post Reply

Return to “Other Hardware: Field & Back Office”