Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
mike-ravkine 
posted an update 4 days ago
Post
2710
There is no anxiety quite like powering up 2KW of basement compute after rewiring it all. Small bit of trouble with the horizontal 3090 because I misread my motherboard manual, but otherwise so far so good.. Next we see if I've built up enough cooling to hit my target TDP on those 3-slot nvlinked cards especially. The 4-slot bridges are much easier to work with but their prices went bananas and I couldn't acquire a second, so gotta get a little creative with intakes.
mike-ravkine

The top high density rack is successfully hitting 65C targets at 1.15kW TDP.
image (1)

Really impressive coolers on the FEs, there is barely 10mm of space but the sandwiched one still keeps up to unobstructed air coolers.

We are back, baby 🏇

asaid7

curious to hear about the thermals on the two 3090 FE stacked up like that. 4 slot nvlink is insanely overpriced right now.

·
mike-ravkine
•
edited 3 days ago

I am running 3x140mm 89CFM as intakes on the front and directly underneath the in-blowers of those two 3090FE is the main trick - a dedicated 140mm 140CFM blowing upwards to feed the intake blowers at the front/pcie edge of these coolers. The cross-blower air passes through both cards and then has space to vent their heat on the right.

At 280W load the 'outside' card is bored and sitting at 50C with the blower fans at 50% while the 'inside' card maintains 60-65C with it's blowers closer to 70-80%.

The main thing to be careful about with the FE dual-blower coolers is do not try to pump air into the 'front' (pcie side) of them! I see this configuration on many mining rigs and its fine for air cooled cards but the FEs actually have a blower venting out this pci-slot side so you HAVE to feed them from either the rear or underneath.

When I had the 4-slot bridge installed across them, the rear-feed alone was sufficient.

In this post