MagAO-X Engineering Run 2024B Day 1: No Longer A Couple

One of the big goals for this visit to LCO was to troubleshoot a coupled pair of actuators on our precious 2040 actuator “tweeter” deformable mirror (DM). Being coupled meant that when we poked actuator 498, actuator 650 moved (not 498). Actuator 650 also moved when we poked 650. We first noticed this problem after shipping MagAO-X in March, when we ran a post-cabling function test. However, digging through our archives we found that it showed up back in November, 2023, but we just didn’t notice.

Now our tweeter is expensive. So we don’t like it when things go wrong, and it makes messing with it really stressful. But having those two actuators acting up means MagAO-X didn’t work perfectly, so we had to do what we can to fix it. We deferred it until after our March and April runs to give us time to plan and do it right.

We can all breathe a sigh of relief: with only minimal shenanigans we got it fixed. After testing various potential cabling problems, we found that a circuit board just needed to be replaced. We had a spare, popped it in, and the couple had split up and gone back to behaving normally.

And we got that done in time for Empanada Sunday:

Empanadas are just a little bit better fresh, compared to at midnight.

After lunch we got onto the next set of tasks, which included disassembling a rack shelf so we can get better airflow.

The shelf in question is actually Jay’s baby. He designed the hold-down system for the components on it. That also means he had to take it apart and put it back together.

We’re also upgrading the Instrument Control Computer (ICC) to go faster and do more stuff. We like to liquid cool our CPUs and GPUs. Normally we rig that up in Tucson, but we couldn’t get all of the components from our usual supplier in time, so we shipped what we had (and found some substitutes on Amazon!) and are doing some clean room plumbing.

An NVIDIA RTX 4090 in the middle of having a water cooling block attached to it. The shiny square in the middle is the business end.

Jay has also been leading the assembly and testing of new ICC. Other than the liquid cooling parts, it was fully working in Tucson before we took it apart to ship it down here. We’re getting it all back together.

Jay hard at work screwing ICC components to the custom rack mount.

Even after our delayed departure, we are making good time thanks to the relatively easy fix to the tweeter. I haven’t had enough running on this trip yet, so I took some time before dinner to go circle around the 100″.

Start of run, heading to the 100″ telescope which is down and left from Magellan.
The view back from the 100″. I’m surprised this came out, it was so windy I couldn’t hold my phone still.
From 1966.

MagAO-X Maintenance 2024-1 Day 2: In-N-Out

Caught the 0830 transport down this morning. Irony of ironies, this may have been the most stunningly clear blue Chile-wide sky I’ve ever seen. Caveat: I’m usually coming off a weeks-long night-schedule average 4 or 5 hrs of sleep, so not usually awake for any of this trip.

Not a cloud in the sky at sunrise, but a few on the ground.
Last view of the telescopes (until next time)
First view of the ocean. About here is where your skin breathes again.
The forms must be observed. Even after such a short stay, this is the stuff.
The La Serena / Coquimbo shoreline. Should have gone to the beach.
The Andes look very imposing when you can see them.
Challenge: find Vera C. Rubin!
The SCL succulent wall looked gorgeous with all the sunlight.
A closeup.
Song of the day.

MagAO-X Maintenance 2024-1 Day 1: That Was Quick

My short visit was a success. Got our cooling system back to nominal, and installed and tested cables for our new cam-yo-fiz.

Clouds aren’t so bad when you’re above them
A good lunch to warm up with. It’s cold.
Beef meatballs on pasta, and some very garlicy tomatoes and avocado.
My PhD project sits forlornly in the parking lot, not even fully covered.
Vizzy. What a tail!
Time to go to work up there.
The song of the day

MagAO-X Maintenance 2024-1 Day 0: 40 to 40 in 1 day

I popped down to LCO from Tucson for a quick visit to fix some cooling issues and test some new cables. I left 40 C in Tucson and arrived to 40 F at LCO.

On my flight down I was somewhat surprised to see that Laird was coming with me.

Laird and Shannon were seated in the row in front of me. Who knew.
The Starbucks SCL Tradition. I think that’s Yeri.
I don’t know what this is about, but I know we’re fans.
Santiago is surrounded by snow-capped mountains, but it had just rained when I walked between terminals.
The La Serena area is stunningly green.
A familiar, but greener, view driving up the coast
Our old friend is still here waiting for us. I probably won’t make it up there.
The post-sunset sky on my walk down to dinner.

There are blog rules. But since I’m the only one here I’m keeping them to myself. Don’t worry, I’ll follow them.

The song of the day

MagAOX 2024Ab Day 16: I Still Believe

The Picard Principle states:

“It is possible to commit no mistakes, and still lose.”

That is life.

My wrap-up posts have always been somewhat triumphant. But not this time, ‘cuz that just sucked.

To be clear the MagAO-X team did what they always do, and the instrument was ready to rock like we always keep it. We’re all just bummed that we didn’t get to let it off the chain to go hunting planets and disks and other exciting things. We just sat there, sometimes actually hiding under a blanket.

A quote:

“Laird and I agree that this was the worst continuous stretch of bad seeing we have seen in all that time.”

Morons. Apropos from this run:

“Why would you say that?”

It can always get worse. One’s dome has to be open for one to measure seeing. As I type this in my room, body trying to decide what schedule we’re going to follow tonight, my window is vibrating as the wind climbs and the stars fiercely twinkle.

Why not blow us off the mountain?

As Leden said, Jay, Josh, and I stayed an extra day to get one more Empanada Sunday.

They’re even better with seafood soup.

Since we were here, and to make sure it was a business expense, we did take the time to re-cable MagAO-X. Best 15 minutes: split over two 7.5 minute moments when the MEMS deformable mirrors came alive with no problems and we didn’t have to re-cable.

If exao1 has your key, MagAO-X is all yours.

We’ll be back, probably in November. In the mean time, MagAO-X is available for any eXtreme Wavefront Control experiments you have. A few of us will stop by around September to tighten some bolts, etc.

Don’t worry. It can always get better too.

I must confess, I still believe.