AMD THREADRIPPER PRO 3995WX
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Still working on PC88, which is the workstation consisting of the AMD ThreadRipper Pro 3995WX with the ASUS Pro WS WRX80E-SAGE-SE MB. I decided to get the UBUNTU version installed first since there are some problems running both W10 Pro and UBUNTU on this original BIOS on the ASUS MB and there has been only one person at ASUS level one support who knew more than anyone at level two support in the last week.
Below is a view of the case showing the following from the top down:
1. DeepCool Castle 360EX Cooler (3X 120mm fans)
2. ASUS Pro WS WRX80E-SAGE-SE MB (US$90 ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card, included in box)
3. 4X128 GB ECC RDIMM=512GB RAM (will fit anywhere in the 8 RAM slots. I have 1 on top and 3 on bottom)
4. AMD 3995WX CPU (under cooler)
5. LSI AVAGO 9361-8i SAS RAID Controller, with 8X Seagate 12TB = 72TB (all not included in pricing table)
6. ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card with only Samsung 980 Pro M.2 shown below
7. NVIDIA GTX 1050Ti (decided to use this since it fits works for now and I do not need to steal a RTX2080Ti from another workstation)
8. EVGA 1300W G2 power supply
9. Not shown, Dual Intel 10GbE ports on ASUS MB.
The table below shows the cost of building the 3 workstations that we use to process our NavVis, other scanning/imaging projects, and other experimental visualizations.
Costs do not include Hawaii tax and freight (Free from Amazon and $34 two day from NEWEGG).
The cost difference between the two 3990X and 3995WX workstations is in the CPUs, RAM, and MBs. It is hard to believe that 256GB ECC RDIMM would be that much of a cost difference, but if you do not have it and you crash or have to break up you project in to smaller pieces, what is that cost? We can expand the RAM to 1.5TB with enough $$ and 4 empty RAM slots.
Below is a view of the case showing the following from the top down:
1. DeepCool Castle 360EX Cooler (3X 120mm fans)
2. ASUS Pro WS WRX80E-SAGE-SE MB (US$90 ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card, included in box)
3. 4X128 GB ECC RDIMM=512GB RAM (will fit anywhere in the 8 RAM slots. I have 1 on top and 3 on bottom)
4. AMD 3995WX CPU (under cooler)
5. LSI AVAGO 9361-8i SAS RAID Controller, with 8X Seagate 12TB = 72TB (all not included in pricing table)
6. ASUS Hyper M.2 X16 PCIe 4.0 X16 NVMe card with only Samsung 980 Pro M.2 shown below
7. NVIDIA GTX 1050Ti (decided to use this since it fits works for now and I do not need to steal a RTX2080Ti from another workstation)
8. EVGA 1300W G2 power supply
9. Not shown, Dual Intel 10GbE ports on ASUS MB.
The table below shows the cost of building the 3 workstations that we use to process our NavVis, other scanning/imaging projects, and other experimental visualizations.
Costs do not include Hawaii tax and freight (Free from Amazon and $34 two day from NEWEGG).
The cost difference between the two 3990X and 3995WX workstations is in the CPUs, RAM, and MBs. It is hard to believe that 256GB ECC RDIMM would be that much of a cost difference, but if you do not have it and you crash or have to break up you project in to smaller pieces, what is that cost? We can expand the RAM to 1.5TB with enough $$ and 4 empty RAM slots.
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Finally got most of the pieces together for my AMD 3995WX Threadripper Pro + ASUS Pro WS WRX80E SAGE MB + Raid 6 8X12TB=72 TB + W10 Pro + Ubuntu 18.04 + 512GB ECC DDR4 RAM with everything working. It is not every day that you see 128threads working at 100% using 300+GB of RAM.
Jed: I am still going to wait to see if anything better comes out on the GPU market without taking 3 PCIe slots, especially since I only have 4 empty PCIe-4 slots left. I could take the last (7) PCIe slot since the case has two empty positions below it.
Unfortunately, ASUS support + knowledge base needs significant help since this MB is nothing like the previous ones, especially if you have any legacy devices. You will have problems using the ASUS Hydrid NVMe card without reading the manuals in detail which do not relate with each other. It was trial and error for a week. I am using the NVMe card to boot W10 Pro off a Samsung 980 Pro 1TB and boot UBUNTU 18.04 off of a Samsung 980 Pro 2TB, which is easier to move NVMe sticks around since the MB NVMe have 3 locations, but only 2 can be active and are difficult to move around.
LESSON LEARNED: If you are going to use these type of multi-core CPUs with MB having many PCI-e-4 slots, it is best to check the MB manual for the number of power plugs to be used and counting power plugs to be used on the number of GPUs before deciding to purchase the MB and power supply (which both are expensive or difficult to find).
Here is the ASUS manual showing a minimum of 5 power plugs needed on the MB + what ever you will need for your GPUs, and the other image showing how many power plugs are on the EVGA 1300Watt PSU. With this configuration, you will only have 3 cables left for VGA/GPU.
Jed: I am still going to wait to see if anything better comes out on the GPU market without taking 3 PCIe slots, especially since I only have 4 empty PCIe-4 slots left. I could take the last (7) PCIe slot since the case has two empty positions below it.
Unfortunately, ASUS support + knowledge base needs significant help since this MB is nothing like the previous ones, especially if you have any legacy devices. You will have problems using the ASUS Hydrid NVMe card without reading the manuals in detail which do not relate with each other. It was trial and error for a week. I am using the NVMe card to boot W10 Pro off a Samsung 980 Pro 1TB and boot UBUNTU 18.04 off of a Samsung 980 Pro 2TB, which is easier to move NVMe sticks around since the MB NVMe have 3 locations, but only 2 can be active and are difficult to move around.
LESSON LEARNED: If you are going to use these type of multi-core CPUs with MB having many PCI-e-4 slots, it is best to check the MB manual for the number of power plugs to be used and counting power plugs to be used on the number of GPUs before deciding to purchase the MB and power supply (which both are expensive or difficult to find).
Here is the ASUS manual showing a minimum of 5 power plugs needed on the MB + what ever you will need for your GPUs, and the other image showing how many power plugs are on the EVGA 1300Watt PSU. With this configuration, you will only have 3 cables left for VGA/GPU.
You do not have the required permissions to view the files attached to this post.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: AMD THREADRIPPER PRO 3995WX
Glad to hear you're up and running, Dennis. I remember being gleeful in the early 80s when I got the 16k RAM pack for my ZX81 with 3.25mhz CPU, being delighted to have all that memory to wallow in. You have 19,660,800 times the amount of memory and about 62,828,257 the integer OPS (very many more FLOPS)dhirota wrote: ↑Fri Mar 26, 2021 1:32 am Finally got most of the pieces together for my AMD 3995WX Threadripper Pro + ASUS Pro WS WRX80E SAGE MB + Raid 6 8X12TB=72 TB + W10 Pro + Ubuntu 18.04 + 512GB ECC DDR4 RAM with everything working. It is not every day that you see 128threads working at 100% using 300+GB of RAM.
PCs and parts are getting scarce enough around this part of the world. Needed to update an old testing rig yesterday and found it is now cheaper to buy than to build, LENOVO P340 i9-10900K with 64gb cost me €1,450 on amazon.de. Self build would cost €1,650 using available parts delivered to Ireland in the same time frame. I'm lucky enough to have a spare GPU, these are silly prices right now. Planning to replace my main rig at some point soon but reckon I'll hold off for a few months more.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
I decided that running a GTX 1050Ti is OK some of the time to visualize 100Mpt, but if you have to visualize and render 500Mpts to 1.3Bpts you need something better. I did some adjustment on the 72TB SAS disk arrangement to get better cooling of the disks and gave me some extra room for the two blank spaces below the motherboard for the RTX2080Ti, which takes up three slots and 2 VGA (8pin cables, leaving me with only one VGA cable left)dhirota wrote: ↑Wed Mar 03, 2021 6:58 pm Jed
We have 5 Nvidia RTX2080Ti GPUs in the office that take up 3 slots. I will steal the one on our 18Core i9-9980XE until I figure out how to best use the 7 PCIe-Gen-4 slots with 128 lanes on the 3995WX and which GPUs make the most sense after the Bit-coin guys go broke.
Still waiting to see what/when NVIDIA will have beyond the RTX A6000.
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
This is probably the last add-on to this system, unless I can get a better NVIDIA GPU at MSRP.
I call this my GOLD 22 KT XBOX wireless controller. It took a little while to figure out how to get it running on some of the visualization software. It was straight forward once you get a game person in the room to help figure it out. It makes a significant difference in real time movement of 1.35Bpts on a 82 inch 4K screen versus a mouse to control everything.
ONLY "BENCHMARK"
1. NavVis VLX scan outdoors
2. Area =80,600 sq.meters
3. post-processed on 5mm grid, 1.35 billion points
4. importing 20GB E-57 file = 3 minutes
5. pan/zoom in real time for entire point cloud
I call this my GOLD 22 KT XBOX wireless controller. It took a little while to figure out how to get it running on some of the visualization software. It was straight forward once you get a game person in the room to help figure it out. It makes a significant difference in real time movement of 1.35Bpts on a 82 inch 4K screen versus a mouse to control everything.
ONLY "BENCHMARK"
1. NavVis VLX scan outdoors
2. Area =80,600 sq.meters
3. post-processed on 5mm grid, 1.35 billion points
4. importing 20GB E-57 file = 3 minutes
5. pan/zoom in real time for entire point cloud
You do not have the required permissions to view the files attached to this post.
- gsisman
- V.I.P Member
- Posts: 898
- Joined: Fri Oct 07, 2016 1:51 pm
- 7
- Full Name: Steve Long
- Company Details: Montgomery County DOT _ MD
- Company Position Title: Land Survey Supervisor
- Country: United States
- Skype Name: gsisman1
- Linkedin Profile: Yes
- Has thanked: 767 times
- Been thanked: 149 times
Re: AMD THREADRIPPER PRO 3995WX
Nice Dennis. I'd love to see a little video clip of the navigation of that puppy on that 82" screen 1.35bpts. That is a LOT of points for the VLX, My 3500' linear Street scape only had 531 milliondhirota wrote: ↑Fri Apr 02, 2021 11:01 pm This is probably the last add-on to this system, unless I can get a better NVIDIA GPU at MSRP.
20210402_105749.jpg
I call this my GOLD 22 KT XBOX wireless controller. It took a little while to figure out how to get it running on some of the visualization software. It was straight forward once you get a game person in the room to help figure it out. It makes a significant difference in real time movement of 1.35Bpts on a 82 inch 4K screen versus a mouse to control everything.
ONLY "BENCHMARK"
1. NavVis VLX scan outdoors
2. Area =80,600 sq.meters
3. post-processed on 5mm grid, 1.35 billion points
4. importing 20GB E-57 file = 3 minutes
5. pan/zoom in real time for entire point cloud
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Steve
I have not at least posted some screen shots because of the client requirements of not showing images to the public. We have other public VLX scans that we scanned last year that we need to re-process to improve color balance and visualization soon.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Steve
Here is a partial resort scan of 700 million points. I am not organized yet to generate great color balance and improved visualization, the "BENCHMARK" below is interesting to see a NVIDIA RTX 2080Ti working.
"BENCHMARK"
NavVis VLX scan, post-processed on 5mm grid, 700 million points
E-57 file = 10.2 GB
50% of 82 inch Samsung 4K screen displaying 30million pixels
NVIDIA RTX 2080Ti running at 80% to 97% with XBOX wireless controller
full screen of 82 inch Samsung 4K screen displaying 30million pixels
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
After searching the entire weekend trying to find another NVIDIA GPU, I just purchased a NVIDIA A6000 with 48GB ECC VRAM for MSRP US$4,650 and it will be delivered next week.
- TommyMaddox
- V.I.P Member
- Posts: 514
- Joined: Mon Apr 25, 2016 7:15 pm
- 7
- Full Name: Tommy R Maddox III
- Company Details: ONSITE3D
- Company Position Title: Director of Technology
- Country: USA
- Linkedin Profile: Yes
- Location: Calgary, Alberta
- Has thanked: 67 times
- Been thanked: 130 times
Re: AMD THREADRIPPER PRO 3995WX
Eager to see how RiScan Pro performs on the 48GB VRAM A6000 GPU. I know on our end the 1080Ti 11GB units we have perform better than the newer RTX series cards that have lower vram amounts just due to how RSP handles point display. Heavily considering getting that once prices become more justifiable.