MagAO-X 2022A Day 5: Clogs and cables and comrades arriving

We’ve got MagAO-X mostly re-cabled in its temporary home in the LCO cleanroom, and Doctors Close and Knight are fresh off the plane and working on the optical alignment. But, earlier, we had a fun discovery: the instrument control computer (ICC) was getting almost no coolant flow.

Yesterday, we did some brain surgery on the real-time control computer. Today was more like heart surgery. We found that although our pump tried its hardest, almost no liquid coolant made it through the ICC, and temperatures remained stubbornly high. In other words, it was clogged. We really wanted the issue to be anywhere except the CPU liquid cooling block, so of course our troubleshooting pinpointed… the CPU liquid cooling block. Not any of the lines feeding it, but the very center of it.

Disclaimer: this is actually a picture of the other computer, but it looks cooler. They’re basically the same though.

See those three pink hoses in the center, under a bunch of crap? Those go to the blocks we removed. We took the computer out of the rack, the cooling blocks off the computer, then took them both out of the clean room entirely to try and blast the clog free.

But, to no avail.

After consulting reputable YouTubes, we were pretty sure these things came apart. The down-side is, according to the manufacturer, you lose your “leak-free guarantee.” (Well, it’s probably void after 5 years anyway.)

Readers, it was gross in there. We only do extreme adaptive optics, and this was extremely gross.

It turns out glycol does not enjoy being left in tiny channels without moving for a long time. And, while MagAO-X was on its two year shipping hiatus, it didn’t get the same twice-yearly flushing it evidently needed.

Fortunately, there was a solution: graduate student labor!

Photo by Jared Males

(Just kidding; it was a team effort.)

After we reassembled and pressure tested and reinstalled everything, we had great flow. We also had just spent a few hours on another unscheduled computer disassembly, and had to hustle to get the system ready for Laird and Justin. Fortunately, it was a two-viscacha day, which boded well for our efforts.

Another of today’s wins was figuring out what the “ultra-wide angle” camera on my phone is for: making an already long advisor look even longer.

Once the computers were back online again, we used 2.67 monitors per researcher in hopes of making everything go faster.

Fortunately, installing all of the cables between the electronics rack and the instrument went great. All our movable bits in the instrument moved when we asked them to, so we had time for a bit of sunset-watching before the clean room became part of Laird and Justin’s quarantine bubble.

It only looks like a romantic twosome because Jared had to go take the picture.

I also had a wistful moment, taking a selfie in the tail plate. I definitely did not imagine that I would be the only non-faculty repeat visitor from the original team.

We rigged up a lab laptop (labtop) and left it signed in to Zoom™ so that when Laird and Justin got in we’d see them. We’re remotely supporting their alignment efforts by sitting upstairs with our laptops to move mechanisms as needed. (Or write blog posts, when not needed.)

As I write this, they’re still at it. I admire their tenacity.

Late breaking news: we have pupil images on the pyramid wavefront sensor!

Song of the Day

Your song of the day is brought to you by ~*~*flow*~*~.