AMD THREADRIPPER PRO 3995WX
- Carbix
- V.I.P Member
- Posts: 236
- Joined: Sat Mar 16, 2019 1:36 am
- 5
- Full Name: Daniel Loney
- Company Details: Excelsior Measuring
- Company Position Title: Owner
- Country: Canada
- Linkedin Profile: Yes
- Location: Canada
- Has thanked: 33 times
- Been thanked: 50 times
- Contact:
Re: AMD THREADRIPPER PRO 3995WX
RAM disk could work but tends to come with its own number of issues.
I would run a raid 5 with 4 NVMe Gen 4's and if you feel like living on the whiled side run it in raid 0.
Intel has their optane drives. I would see if they have a new Gen 4 full pci version. Optane is all about dropping IO
they ain't cheap
I would run a raid 5 with 4 NVMe Gen 4's and if you feel like living on the whiled side run it in raid 0.
Intel has their optane drives. I would see if they have a new Gen 4 full pci version. Optane is all about dropping IO
they ain't cheap
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Shanesmacl wrote: ↑Mon May 10, 2021 7:57 am Not sure if I've asked this before Dennis, but have you considered RAM disks. If you have an I/O bound process and available memory, these are likely to be an order of magnitude faster than SSD. The trick is figuring out how to divide the physical RAM out to leave enough space for the program and storage, may not be workable on larger jobs where you don't have the available excess.
We has discussed this over the years, and one of the references is listed below:
viewtopic.php?p=69651#p69651
Based on your 4X-6X SSD @ 500MB/second IO, I might be better off with RAID0 as the processing storage?
In feature evaluation of workflow using a processing workstation, I use the many years of observation from other historic systems that I have observed. One in particular, was the PLATO system from the University of Illinois. Most people on this LSF are probably not familiar with it since the majority were not on this planet at its inception in the 1960s. If my memory is still good, it used a multi-tiered system of storage.
https://en.wikipedia.org/wiki/PLATO_(computer_system)
PLATO @50 PLATO Computer Learning System 50th Anniversary
https://www.youtube.com/watch?v=THoxsBw-UmM&t=31s
To concentrate resources into the fastest area, RAM, but also the most expensive. The RAM DISK alternative has been considered as the cost of RAM decreases and the motherboards allow greater number of DIMM slots. In this example with the TR PRO 3995WX, it was important to move large data files from the AVAGO RAID6 PC88E(72TB) to the next tier, a RAID0 PC88D(4TB) or Samsung 980Pro (2TB) for processing storage because of the read/write speed. Shown below is the CrystalDiskMark (CDM) for PC88E and PC88D for the 64threads. As mentioned above, I was able to transfer an E57 20GB file from PC88E to PC88D in 25.31 seconds.
Shane: At this point, I could "sacrifice" 150GB from the 512GB of total RAM available, but I do not have the time to check out potential problems in W10 and Ubuntu to see if it will work with all our raw data set sizes and application programs. I am still evaluating many visualization applications that are running on W10 and Ubuntu, but most are running on W10, so the priorities may change and you will be the first to know.
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Shane:dhirota wrote: ↑Tue May 11, 2021 8:08 pmShanesmacl wrote: ↑Mon May 10, 2021 7:57 am Not sure if I've asked this before Dennis, but have you considered RAM disks. If you have an I/O bound process and available memory, these are likely to be an order of magnitude faster than SSD. The trick is figuring out how to divide the physical RAM out to leave enough space for the program and storage, may not be workable on larger jobs where you don't have the available excess.
We has discussed this over the years, and one of the references is listed below:
viewtopic.php?p=69651#p69651
Based on your 4X-6X SSD @ 500MB/second IO, I might be better off with RAID0 as the processing storage?
........
Shane: At this point, I could "sacrifice" 150GB from the 512GB of total RAM available, but I do not have the time to check out potential problems in W10 and Ubuntu to see if it will work with all our raw data set sizes and application programs. I am still evaluating many visualization applications that are running on W10 and Ubuntu, but most are running on W10, so the priorities may change and you will be the first to know.
Since the term RAMDISK was not a familiar in our office, I decided to research the best as rated by others. Depending on requirements such as cost, size, and installation there were about 20+ on the internet. Many were dated from 5+ years ago, when SSD and NVMe storage were difficult to obtain, HDDs were slower and smaller in size. Here is a YouTube example from February 5, 2014
https://www.youtube.com/watch?v=Sv8Ypj3iUWI
There was only one system that met my criteria of free, expandable to 1TB of RAM, and works (I tried another that worked at 4GB but it crashed at 64GB). Below are the CrystalDiskMark benchmarks running at 64threads showing from left to right:
1. PC88C- Samsung 980Pro-(1TB) attached to ASUS Hyper M.2X16 PCIe Gen 4
2. PC88D- 2X Samsung 980Pro-(2TB) total 4TB attached to ASUS Hyper M.2X16 PCIe Gen 4, using ASUS MB BIOS RAID0
3. PC88E- AVAGO RAID6- (8X12TB Seagate) total 72TB
4. PC88Z- RAMDISK 64GB using 4X2933MHz 128GB ECC DDR4 RAM total 512GB in 4 of 8 channels of AMD 3995WX memory.
As I said earlier, I do not have the time to check on the potential problems with applications and the RAMDISK. The other problem is the relative smaller size of the RAMDISK device created, which may limit the software application and the data set size. To acquire the I/O speeds of 2X Samsung 980Pro for US$800 in RAID0 and 4TB of storage versus the estimated US$20,000 cost of 2TB of ECC RAM at an unknown application speed.
That is about the end.
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
I started this thread almost 3 months ago as I started to build another AMD, this time a AMD ThreadRipper Pro. It has changed several times since the start and will probably change again many times, especially since I have still not received the NVIDIA RTX A6000 ordered over a month ago. The ability to run applications in both W10 and Ubuntu with 7 PCIe Gen4 X16 slots almost without worrying what will fit where is worth the extra cost of building such a research/production system. Over the many years of doing this it has enabled us not to worry about catching up with our competitors and providing clients with as much as we can afford.
I also track the number of visits to the threads by both LSF Registered Members (RM) and Guests. I am still not sure how all this works on the LSF, but the RM's apparently see all the tables and images that I produce with the significant information and Guest just see the text. Since the beginning of this thread there have been 7,283 views and at least 1,690 views by RM according to LSF. This means that 5,000+ Guests did not get the full value of the information that I posted. I am not sure why someone would spend the time to visit a forum and not access everything the forum had to offer.
I am also interested in the 1,000+ views by RM, since only a few comments or questions resulted from the 20+ RM responses. I have been waiting for someone to ask a question about the last posting on using RAMDISK. Most of the previous examples on the internet in the last 5+ years were 4-20GB for RAMDISK, which seemed too small for many applications. For my posted example, I decided to use 64GB, since I decided that some people could have 256GB of RAM available. There have been no responses or questions in the last week, so I have determined that either no one has any further interest or the presentation is irrelevant. I had a question in my own mind to document a larger size RAMDISK of 256GB since I had the resources, although I will probably not use RAMDISK in the future.
And finally, the system could have been built at a significantly lower cost, but whether this workstation would be limited by many of the larger applications we are beta testing in W10 is not worth the total cost of multiple research systems at this time. It seems from the threads on the LSF, most people are not willing to post research/testing information.
The shortage of electronic parts will continue into the future, so for those without parts to build, they will have to buy at a higher cost, or they will have to figure out a different workflow for many applications that they do not know about today.
I also track the number of visits to the threads by both LSF Registered Members (RM) and Guests. I am still not sure how all this works on the LSF, but the RM's apparently see all the tables and images that I produce with the significant information and Guest just see the text. Since the beginning of this thread there have been 7,283 views and at least 1,690 views by RM according to LSF. This means that 5,000+ Guests did not get the full value of the information that I posted. I am not sure why someone would spend the time to visit a forum and not access everything the forum had to offer.
I am also interested in the 1,000+ views by RM, since only a few comments or questions resulted from the 20+ RM responses. I have been waiting for someone to ask a question about the last posting on using RAMDISK. Most of the previous examples on the internet in the last 5+ years were 4-20GB for RAMDISK, which seemed too small for many applications. For my posted example, I decided to use 64GB, since I decided that some people could have 256GB of RAM available. There have been no responses or questions in the last week, so I have determined that either no one has any further interest or the presentation is irrelevant. I had a question in my own mind to document a larger size RAMDISK of 256GB since I had the resources, although I will probably not use RAMDISK in the future.
And finally, the system could have been built at a significantly lower cost, but whether this workstation would be limited by many of the larger applications we are beta testing in W10 is not worth the total cost of multiple research systems at this time. It seems from the threads on the LSF, most people are not willing to post research/testing information.
The shortage of electronic parts will continue into the future, so for those without parts to build, they will have to buy at a higher cost, or they will have to figure out a different workflow for many applications that they do not know about today.
You do not have the required permissions to view the files attached to this post.
- TommyMaddox
- V.I.P Member
- Posts: 514
- Joined: Mon Apr 25, 2016 7:15 pm
- 7
- Full Name: Tommy R Maddox III
- Company Details: ONSITE3D
- Company Position Title: Director of Technology
- Country: USA
- Linkedin Profile: Yes
- Location: Calgary, Alberta
- Has thanked: 67 times
- Been thanked: 130 times
Re: AMD THREADRIPPER PRO 3995WX
Dennis, I appreciate all of the testing and documentation you did for the community, beyond what words can convey.
I typed up a full 3 paragraph reply, but because LSF signs me out every few minutes, it was lost.
I typed up a full 3 paragraph reply, but because LSF signs me out every few minutes, it was lost.
- Carbix
- V.I.P Member
- Posts: 236
- Joined: Sat Mar 16, 2019 1:36 am
- 5
- Full Name: Daniel Loney
- Company Details: Excelsior Measuring
- Company Position Title: Owner
- Country: Canada
- Linkedin Profile: Yes
- Location: Canada
- Has thanked: 33 times
- Been thanked: 50 times
- Contact:
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Tommy and Daniel
The real trick to posting responses on any forum is to (1) compose it using a WORD PROCESSOR to create it; (2) login to the forum; (3) cut and paste the content to the forum; (4) edit on the forum; and finally (5) submit it.
Tommy
I thanked you in advance, but it would be interesting to see your comments since I appreciate them very much.
The real trick to posting responses on any forum is to (1) compose it using a WORD PROCESSOR to create it; (2) login to the forum; (3) cut and paste the content to the forum; (4) edit on the forum; and finally (5) submit it.
Tommy
I thanked you in advance, but it would be interesting to see your comments since I appreciate them very much.
- smacl
- Global Moderator
- Posts: 1409
- Joined: Tue Jan 25, 2011 5:12 pm
- 13
- Full Name: Shane MacLaughlin
- Company Details: Atlas Computers Ltd
- Company Position Title: Managing Director
- Country: Ireland
- Linkedin Profile: Yes
- Location: Ireland
- Has thanked: 627 times
- Been thanked: 657 times
- Contact:
Re: AMD THREADRIPPER PRO 3995WX
Hi Dennis,dhirota wrote: ↑Sun May 23, 2021 10:48 pmI am also interested in the 1,000+ views by RM, since only a few comments or questions resulted from the 20+ RM responses. I have been waiting for someone to ask a question about the last posting on using RAMDISK. Most of the previous examples on the internet in the last 5+ years were 4-20GB for RAMDISK, which seemed too small for many applications. For my posted example, I decided to use 64GB, since I decided that some people could have 256GB of RAM available. There have been no responses or questions in the last week, so I have determined that either no one has any further interest or the presentation is irrelevant. I had a question in my own mind to document a larger size RAMDISK of 256GB since I had the resources, although I will probably not use RAMDISK in the future.
As others have said, many thanks for your contribution here, it is greatly appreciated. Admittedly it has been some years since I've used RAM disks regularly and it looks very much like the software out there for them hasn't seen updates to take advantages of new architectures in some time. Out of interest, was the ramdisk solution that you tried out open source?
While I spend a whole lot of time on performance optimization, it is nearly all from the development side, focussed on algorithmics rather than hardware. The posts yourself and Daniel have made are very useful as they challenge (and often disprove) a number of my assumptions as to what the best hardware is to run the type of software that I'm writing. Many of my users are looking for a price / performance sweet spot in the context of being able to work with increasingly large point clouds in a cost effective manner. e.g. what is the minimum spec to work efficiently with a 1 billion, 2 billion, 4 billion or 8 billion points? Much of this comes down to analysis of specific workflows and identifying and removing bottlenecks only to have them move somewhere else in the workflow. If the software is hitting the disk when there is ample RAM available to the detriment of performance, it is really calling out for an optimization. FWIW, for most of the typical SCC workflows, something like an i9 10900k and 64gb of RAM is a decent budget option and I've been recommending the Thinkstation 340 as providing a lot of bang for your buck. Hard to believe the new i9 11900k is actually slower than it's predecessor on many common tasks.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
The RAMDISK is open-source as were all of the approximately 20+ applications that I reviewed. I do not know if there are other RAMDISK applications that might be faster, but this version claimed unlimited size (the 256GB example could have been larger) and the price was right. What I did not check was transferring my 20GB E57 file between the RAID0 NVMe and the RAMDISK. It will not matter since I am not planning to use the RAMDISK unless there are applications that will significantly improve our workflows. Having the ability to test and modify workflows without creating another research/production workstation is what this AMD3995WX and the 7 slot PCIe Gen4 ASUS MB are all about.smacl wrote: ↑Mon May 24, 2021 10:26 am
Hi Dennis,
As others have said, many thanks for your contribution here, it is greatly appreciated. Admittedly it has been some years since I've used RAM disks regularly and it looks very much like the software out there for them hasn't seen updates to take advantages of new architectures in some time. Out of interest, was the ramdisk solution that you tried out open source?
You do not have the required permissions to view the files attached to this post.
-
- V.I.P Member
- Posts: 958
- Joined: Sun Nov 01, 2009 11:18 pm
- 14
- Full Name: Dennis Hirota
- Company Details: Sam O Hirota Inc
- Company Position Title: President
- Country: USA
- Linkedin Profile: Yes
- Location: Hawaii, USA
- Has thanked: 87 times
- Been thanked: 379 times
Re: AMD THREADRIPPER PRO 3995WX
Since my last update, I have been installing more software on my TR PRO running W10 Pro installed on a 1 TB Samsung 980 Pro NVMe stick. Although it has a 5 year warranty, there is also an endurance limit that comes with the warranty. The majority of most people will probably be under the endurance limit in their use of the SSD/NVMe stick. Shown below are some of the parameters to consider.
I used the Samsung Data Migration software to copy all the information on the 1TB 980 PRO to the 2TB stick. If you have ever copied a boot disk with 200+GB on it, the time with a SSD might take 30 to 60 minutes. In this instance, it took a total of 6 minutes as seen on the screenshot below.
One of the significant reasons for the increased boot disk size besides speed is the accumulation of programs and data as well as endurance as shown in the table above. With the ability to track the health of our boot SSD/NVMe disks and raid individual HDDs it is important to check the remaining 5 year warranty to receive a better idea of system stability.
I used the Samsung Data Migration software to copy all the information on the 1TB 980 PRO to the 2TB stick. If you have ever copied a boot disk with 200+GB on it, the time with a SSD might take 30 to 60 minutes. In this instance, it took a total of 6 minutes as seen on the screenshot below.
One of the significant reasons for the increased boot disk size besides speed is the accumulation of programs and data as well as endurance as shown in the table above. With the ability to track the health of our boot SSD/NVMe disks and raid individual HDDs it is important to check the remaining 5 year warranty to receive a better idea of system stability.
You do not have the required permissions to view the files attached to this post.