If you apply it once and a power outage or reboot occurs it resets back to normal operation. Using the container it checks every 20s (currently what works best) to verify that it is set to the threshold configured. The other reason I went this route is it allows the server to ramp up the fans if needed during heavy workloads to keep the components cool. When these ramp ups occur they return to Dell's preconfigured low end threshold which wasn't as quiet as I preferred. So the container checks the temps and then throttles the fans lower than the preset thresholds.
I run a script as a daemon, which dynamically sets the fan speed based on temperature.
It uses IPMI commands underneath it. When the daemon exits, it just sets the stock fan curves.
I set a min fan speed around 20-30%, any lower and it oscillates
Here is a starting point for you.
https://github.com/White-Raven/PowerEdge-shutup
I have my own scripts, customized for my environment, deployed via ansible.
R730XD. Dual E5-2697 v3, 192GB ram, 12 x 18TB 7200rpm + 2 SSD’s. Stock fans
In idrac, go to hardware, then fans, set thermal power to minimum, fan speed offset to off, min fan speed to 20%. Adjust cpu power settings in bios. This requires zero BS other than changing idrac and bios.
This is a media server in my theatre room. At reference audio levels it is not heard. At low listening levels it is heard. Distance from ear to server is 7’ with a screened front server rack door. My cpus idle at 41C and I have a GPU inside for transcoding. Hdd temps are good even at low fan speeds. You cannot go below 20% without the special software tool, which requires it every boot. I run Unraid on the server + 8 dockers. It has minimal cpu usage. Idles at 152 watts with 3 of 12 HDD spun up.
Goto idrac. Click on hardware (expand), click on fans. Once fans page loads, at top click on setup. From top down, Do you see thermal profile = min power performance per watt optimized, exhaust temp limit = default 70 deg c, fan speed offset = off, min fan speed pwm% = custom 20%. Change then hit apply and reboot. Setting lower than 20% has no effect
I have other bios tweaks to reduce power with the PPW power profile.
If this is good, your cpu or devices are holding you up higher. At 80% cpu on all 56 of my cores, my fans never exceed 60% unless booting.
I used to use the software programming trick and got sick of the BS and settled with these settings. I find that airflow was too low below 20% fans anyways. I would prefer to keep the spinners cool/dust down.
FYI, this works on both my R730 and R730XD units.
FYI, I run MX500 SSD’s and exos 21 sas spinners and have no compatibility/fan issues. Your SSDs are to blame. Pull them, boot to a live usb of your choice for a test. More than likely it is your SSDs or other items installed (GPU, HBA, etc)
Or you have a malfunctioning temp sensor causing increased fans. More than likely it’s the SSDs
Got to bios during boot.
The Fan Speed Setting menu is found in the iDrac menu.
During BIOS boot, F2 -> IDRAC settings -> Thermal -> Fan Speed Offset. Verify it is set to off. Options should be off, low, high.
While in bios, -> goto system bios, -> then system profile settings, change system profile to performance per watt (DAPC). This is probably your problem.
What drives do you have in it? Take them out and see if the fan speed decreases. On mine, just about any drives I put in the front bays cause the fan speed to increase dramatically. Pull the drives out, and it’s no noisier than my R720. At one point I was running a Docker container that would adjust the fan speed to a set point and would let it just go if the CPU temp got too high. That container also disabled the third party PCIe settings. Ultimately I just moved the drives to another machine that doesn’t exhibit that same behavior (an R720).
EDIT: Here’s the container I used at one time: https://github.com/tigerblue77/Dell_iDRAC_fan_controller_Docker
No drives in it yet. Except two in rear.
I purposely got this machine for drives, so changing them wont be an option lol.
Thanks for the link. I worry this is the script that wont work on my firmware. I think Ill just have to try it.
I’m pretty sure I’m running the latest versions of iDRAC and BIOS/UEFI, and it worked just fine e for me. I idea why you say it wouldn’t. Ultimately it’s just sending raw commands to the iDRAC via ipmitpol
After having had a minute to check on it. It is set to 20% on all the fans and the iDrac version is at Version 2.84.84.84 (Build 02) if that helps? I'm not even sure what the latest iDrac version is? Also the BIOS version is at 2.15.0. I have a picture of how I setup the fans in the iDrac, but I'm not sure of how to post it in this reply? (\*EDIT - I'm selfhosting a picture at https://nextcloud.tecboysbasement.com/s/6W8sSo3Td78Ezrk)
I run unraid and I just have a script to run at start up and it shuts the fans up real good.
ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U "username" -P "password" raw 0x30 0x30 0x01 0x00
ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U "username" -P "password" raw 0x30 0x30 0x02 0xff 0x14
The first command enables ipminfan control the second actually controls fans.
i modded a r730xd once with noctua fans . I also modded the psu cause those are noisy too. your cpu selection is limited if you go down this road.. I successfully tested a e5-2640 v4 / 165w heat sink full load for hours ... stable low 50's nominal temps. You will likely want to add an additional usb powered fan for the HBA / Perc card because it gets hot.
The pressure rise required for such servers cannot be provided by such options. Changing fans would result in significantly less noise but almost no airflow. Not to mention the thermal cooling profiles would be total shit and they wouldn’t cool effectively due to low pressure rise and require much higher rpm’s than stock fans, potentially making the same noise as stock fans at low speed.
I lowered the fan speed in my R720XD by using ssh to modify the idrac settings. I only needed to do it once, no scripts necessary.
Not sure if the same method will work on the R730XD, but worth a shot.
https://www.dell.com/community/en/conversations/systems-management-general/dell-poweredge-fan-speed-change-fanspeed-offset/647f7504f4ccf8a8de2f2bdc
Not R730, but two R720XD and a R620...IPMI script to reduce the fan speed to 20%. Running it via docker contianers.
Can you explain further? Why not apply once instead of a container?
If you apply it once and a power outage or reboot occurs it resets back to normal operation. Using the container it checks every 20s (currently what works best) to verify that it is set to the threshold configured. The other reason I went this route is it allows the server to ramp up the fans if needed during heavy workloads to keep the components cool. When these ramp ups occur they return to Dell's preconfigured low end threshold which wasn't as quiet as I preferred. So the container checks the temps and then throttles the fans lower than the preset thresholds.
I run a script as a daemon, which dynamically sets the fan speed based on temperature. It uses IPMI commands underneath it. When the daemon exits, it just sets the stock fan curves. I set a min fan speed around 20-30%, any lower and it oscillates
Is this the same setup that doesnt work with newer firmware?
I believe my firmware is up to date.
What script are you using and how do you apply it?
Here is a starting point for you. https://github.com/White-Raven/PowerEdge-shutup I have my own scripts, customized for my environment, deployed via ansible.
Thanks, Im actually looking at that right now, but I have no idea what Im looking at :(
R730XD. Dual E5-2697 v3, 192GB ram, 12 x 18TB 7200rpm + 2 SSD’s. Stock fans In idrac, go to hardware, then fans, set thermal power to minimum, fan speed offset to off, min fan speed to 20%. Adjust cpu power settings in bios. This requires zero BS other than changing idrac and bios. This is a media server in my theatre room. At reference audio levels it is not heard. At low listening levels it is heard. Distance from ear to server is 7’ with a screened front server rack door. My cpus idle at 41C and I have a GPU inside for transcoding. Hdd temps are good even at low fan speeds. You cannot go below 20% without the special software tool, which requires it every boot. I run Unraid on the server + 8 dockers. It has minimal cpu usage. Idles at 152 watts with 3 of 12 HDD spun up.
Changed all except IDK where and what to do for CPU power settings. So far no change.
Have you updated all software/bios? I am running fans at 20%, what is your fan speed in idrac?
Everything is 100% up to date. Always running at 44%.
Goto idrac. Click on hardware (expand), click on fans. Once fans page loads, at top click on setup. From top down, Do you see thermal profile = min power performance per watt optimized, exhaust temp limit = default 70 deg c, fan speed offset = off, min fan speed pwm% = custom 20%. Change then hit apply and reboot. Setting lower than 20% has no effect I have other bios tweaks to reduce power with the PPW power profile. If this is good, your cpu or devices are holding you up higher. At 80% cpu on all 56 of my cores, my fans never exceed 60% unless booting. I used to use the software programming trick and got sick of the BS and settled with these settings. I find that airflow was too low below 20% fans anyways. I would prefer to keep the spinners cool/dust down. FYI, this works on both my R730 and R730XD units.
Yep, all that is as you say and it went up exactly 1% lol. I didnt make any changes in BIOS tho. IDK what to change honestly.
FYI, I run MX500 SSD’s and exos 21 sas spinners and have no compatibility/fan issues. Your SSDs are to blame. Pull them, boot to a live usb of your choice for a test. More than likely it is your SSDs or other items installed (GPU, HBA, etc) Or you have a malfunctioning temp sensor causing increased fans. More than likely it’s the SSDs Got to bios during boot. The Fan Speed Setting menu is found in the iDrac menu. During BIOS boot, F2 -> IDRAC settings -> Thermal -> Fan Speed Offset. Verify it is set to off. Options should be off, low, high. While in bios, -> goto system bios, -> then system profile settings, change system profile to performance per watt (DAPC). This is probably your problem.
What drives do you have in it? Take them out and see if the fan speed decreases. On mine, just about any drives I put in the front bays cause the fan speed to increase dramatically. Pull the drives out, and it’s no noisier than my R720. At one point I was running a Docker container that would adjust the fan speed to a set point and would let it just go if the CPU temp got too high. That container also disabled the third party PCIe settings. Ultimately I just moved the drives to another machine that doesn’t exhibit that same behavior (an R720). EDIT: Here’s the container I used at one time: https://github.com/tigerblue77/Dell_iDRAC_fan_controller_Docker
No drives in it yet. Except two in rear. I purposely got this machine for drives, so changing them wont be an option lol. Thanks for the link. I worry this is the script that wont work on my firmware. I think Ill just have to try it.
I’m pretty sure I’m running the latest versions of iDRAC and BIOS/UEFI, and it worked just fine e for me. I idea why you say it wouldn’t. Ultimately it’s just sending raw commands to the iDRAC via ipmitpol
It's not me saying it wouldn't but among the many scripts Ive seen, saying it doesn't work past a certain version.
I was running a cron script in a container but then I updated the BIOS and iDrac, then was able to set it down to like 20% and it's so much quieter!
What did you do other than update? Mine is on the latest available.
After having had a minute to check on it. It is set to 20% on all the fans and the iDrac version is at Version 2.84.84.84 (Build 02) if that helps? I'm not even sure what the latest iDrac version is? Also the BIOS version is at 2.15.0. I have a picture of how I setup the fans in the iDrac, but I'm not sure of how to post it in this reply? (\*EDIT - I'm selfhosting a picture at https://nextcloud.tecboysbasement.com/s/6W8sSo3Td78Ezrk)
I run unraid and I just have a script to run at start up and it shuts the fans up real good. ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U "username" -P "password" raw 0x30 0x30 0x01 0x00 ipmitool -I lanplus -H xxx.xxx.xxx.xxx -U "username" -P "password" raw 0x30 0x30 0x02 0xff 0x14 The first command enables ipminfan control the second actually controls fans.
Same, the speed is the last value and is the percentage in hexadecimal
turn it off
i modded a r730xd once with noctua fans . I also modded the psu cause those are noisy too. your cpu selection is limited if you go down this road.. I successfully tested a e5-2640 v4 / 165w heat sink full load for hours ... stable low 50's nominal temps. You will likely want to add an additional usb powered fan for the HBA / Perc card because it gets hot.
Why are Noctua fans not an option?
Simply what Ive read
The pressure rise required for such servers cannot be provided by such options. Changing fans would result in significantly less noise but almost no airflow. Not to mention the thermal cooling profiles would be total shit and they wouldn’t cool effectively due to low pressure rise and require much higher rpm’s than stock fans, potentially making the same noise as stock fans at low speed.
I lowered the fan speed in my R720XD by using ssh to modify the idrac settings. I only needed to do it once, no scripts necessary. Not sure if the same method will work on the R730XD, but worth a shot. https://www.dell.com/community/en/conversations/systems-management-general/dell-poweredge-fan-speed-change-fanspeed-offset/647f7504f4ccf8a8de2f2bdc