MagAO-C 2019B Day 5: Alycia and the Baysides

Tonight was Alycia’s first night. I told Emily it would be more relaxed than last night with all Jayne’s weird rotations because Alycia’s observations are more in MagAO’s wheelhouse — Alycia is searching for faint companions; keeping the rotator off for ADI mode; and is a Clio and general observational expert. But… hah.

So on the first Pre-set of the night, the Bayside-X stage decided to not work. There are 3 “Bayside” stages — they are the motors that move the AO system around so that the star is always on the tip of the pyramid WFS even when the star needs to be nodded around the detector on Clio for sky subtraction. We have replaced the X and Y stages with higher-power versions after we had observers who wanted to keep the rotator on and tracking, which caused the stages to overheat and fail. So I wonder if it’s a coincidence that we spent all last night with the rotator on, putting weight on the X-stage, and then tonight the X-stage refused to move…

Laird and Jared are now on pseudo-day schedules (see Alex’s great MagAO-X post about the unpacking today!) but this happened at the start of the night so Laird was still at the summit and Jared was down in the clean room with MagAO-X but he came up to help post-mortem. Laird went out to the platform with a walkie-talkie and we did the usual dance of moving the motors from the control room and Laird telling us what he saw or heard in the Nas. The current would spike but the motor wouldn’t move. Small movements and homing didn’t work; eventually we power-cycled them and in the end the X-stage did start responding again.

WFS hardware GUI, on the Bayside Stages tab. Each stage can be moved manually here, and the position is reported in mm and current in amps. The Bayside X and Y stages nod in Y and X on Clio, respectively, and the Bayside Z stage moves the W-unit towards or away from Clio, effectively producing a focus movement for Clio. Amali edited the code to enable “Control always active” on the X-stage after a nod rather than “Stage” (brakes). [Image description: A GUI on a Linux computer with status and function boxes and buttons.]

But Laird heard a noise that made him think there was some friction affecting the motion of the X-stage. Since the stage worked normally after the power-cycle, we started thinking about all the work we have done on the Baysides over the years. Originally we used the setting “Control always active” as that would position them precisely to the micron. But this was thought to contribute to the over-heating problem when we had high currents at strange rotator angles with all the weight of the instrument on the Y-stage. So we swapped the original out for the higher-power version, but we also started applying the brakes and turning off the active control after each move (“Stage” check box). But now we are thinking… maybe the brakes themselves are starting to fail, and the friction was the X-stage rubbing along its own brake. So tonight, Amali went into the code and reverted the procedure for the X-stage to now enable “Control always active” rather than enabling “Stage” (applying the brakes) after each move. And all stages worked perfectly for the rest of the night.

This is the “Board GUI” that shows the current state of the various subsystems on the W-unit, which includes the WFS (left) and VisAO (right). The rectangle at the bottom shows the current position of the pyramid with respect to the telescope center. The Bayside X, Y, and Z positions are listed. The entire W-unit is moved relative to Clio when we move the Bayside motors. [Image description: An optical diagram of the WFS and VisAO. Light from the telescope comes in from the top center, goes through the ADCs, and then is split by the selectable beamsplitter such that some portion goes to the right to VisAO and some portion goes to the left to the WFS. The WFS arm includes a rerotator that keeps the pupils aligned to the actuators, and a camera lens that keeps the pupils registered to the pixels on the CCD39. The VisAO arm includes science filters. Below the optical diagram is a rectangle with cross-hairs and a red dot showing the relative position of the pyramid.]

After Jared and Laird went to bed, at one point our TO Alberto went down to get some coffee, and I snapped this pic of the all-women control room. What a big difference from when I was the “Only Girl In the World” at LCO!

Astronomers hard at work. From left: Alycia, Logan, Emily, Amali. [Image description: Alycia is on her phone (don’t worry, Clio and VisAO are actively taking data). Logan is analyzing some data on her laptop. Emily is studying the telescope system to make sure she understands the big picture of MagAO. Amali is on her laptop debugging MagAO software, probably the FITS headers of Clio that aren’t being fully populated with the correct AO parameters.]
Vizzy by the library on my way up to the top. [Image description: A Vizcacha sits on a window ledge in the diffuse glow of pre-sunset.]
The Clay telescope at sunrise. [Image description: Photo taken from slightly downhill of Clay, the dome is to the side glinting in the sunrise while the sky has a beautiful stripey glow.]
Walking back to the village at dawn. [Image description: A cluster of terracotta-roofed buildings down a slope with a winding road. Beautiful mountains and hills. Quiet and still.]
The telescope ridges at dawn. [Image description: A stripey glow on the horizon. A mountain peak and a ridge are both covered in telescopes that are closed up to sleep for the day.]

Classic MagAO song from Day 40 of the super long cold winter run with so many technical problems and such long nights that I actually worked, really worked, 112 hours a week (trouble-shooting all afternoon, observing all night, and choosing each day either a shower or dinner): This Must Be the Place/ Naive Melody by the Talking Heads:

MagAO-C 2019B Day 4: ExoSagan

Today was our first science night, with visiting observers Jayne and Ben from Amsterdam in collaboration with Sebastian from Arizona. Jayne, Jared, and I were all Sagan fellows before becoming faculty, and Sebastian is a current Sagan(-Hubble) fellow at Arizona.

A picture of the 3 Sagan alums in the Clay control room. [Image description: Jared is operating his phone in selfie mode so he’s up close. I’m sitting at the VisAO workstation. Jayne is behind me sitting at the Clio workstation. There are around 20 computer monitors mounted to the desks and walls in the background.]

There were a few hiccups associated with trying to keep the rotator tracking like all other instruments at Clay, but Emily and Amali managed to keep the loop closed and we got great data all night long.

Sunset at Clay. The MagAO-C and MagAO-X teams participated in the tradition of hoping for a green flash. [Image description: Several people stand on a catwalk watching the orange sun set over the distant horizon. The horizon has a orange glow and the distant mountains are blurry.]

The song of the day is the Britney Spears classic “Oops I did it again”:

MagAO-C 2019B Day 3: Closed loop Trapezium

Today was the super long day through night. Up at 7am for a quick breakfast, then hurried to the top for a day full of instrument removal and installation, then a full night of on-sky engineering tests. The crew removed MIKE, the MIKE guider, and the f/11. The crew plus the MagAO instrument team (including some Classico and some eXtreme) installed the Nas, the ASM, and Clio. I didn’t take many pictures, but I’m sure those will be coming once the new kids on the block start blogging. The live-view camera of the inside of the Clay dome was cool (see Jared’s MagAO-X post). Here’s what it looks like right now (while we’re still on-sky:)

Live view of the interior of the Clay dome while observing. [Image description: It is a black rectangle. A black rectangle with a border and official-looking timestamp that implies it could have come from a webcam.]

Then there was much cabling and testing of cooling, signals, connections, and functionality. This went on past sunset. Then through the night we further tested on-sky capabilities such as the guider, offsets, angles, volcanoes, scripting, focus, and modes.

A beautiful Clay sunset. [Image description: A horizon lit up with light blue and yellow. The yellow/orange sun is setting behind some low clouds/fog in the distance. In the foreground are some vehicles and gravel.]
We ironed out a few AO, telescope, and Clio bugs, and then were able to test some modes and do some engineering. Here we have closed the loop on Trapezium B. [Image description: A log-scale image of stars in the Trapezium cluster around Theta 1 Ori B. Negative star images are from where the sky was subtracted off.]

Turno changed over today too. Had another nice round of hellos with colleagues I haven’t seen in 18 months. It’s good to see everyone again!

The song of the day is a classical classic, Pachelbel’s Canon by the Canadian Brass. I really like their descant arrangement.

MagAO-C 2019B Day 2: Prep day

Yesterday was cooldown day; today was prep day; tomorrow will be the big day/night.
Tomorrow we will be busy from 7am with removing MIKE, the MIKE guider, and the f/11, and installing the NAS, Clio, and the ASM.
To prep today, we continued to test the Clio motors and code, and then we moved Clio from the Aux to the platform to start pumping on the inner dewar in order to solidify it:

Some of the team bring Clio up on the lift. [Image description: Point of view is from looking down an open lift shaft from above. The Clio instrument and its electronics rack are raised from the ground to the Nasmyth platform level.]

A herd of burros were milling about as I walked to lunch. [Image description: An adult and a juvenile burro stand in profile but look at the camera. They are brown with white undersides. Behind them are some roads, and off in the distance are at least 2 telescope domes on ridges. There is also a low-lying bodega. Finally, a light brown valley, some distant purple mountains, and the blue sky round out the scene.]
I think Vizzy is a fan of the blog. Because I haven’t seen a viz make this pose in several years, but yesterday I link to this image and all of a sudden Vizzy is making the same pose today! [Image description: A Tame Vizcacha on the Clean Room sits up on its haunches with its little paws in front like a classic Monte Python bunny rabbit.]

The song of the day is the MagAO-C Classic “Boom Clap” cover by Lennon & Maisy:

MagAO-C 2019B Day 1: “The empanadas have been taken care of”

Today was Day 1 of the 2019B MagAO-Classic run. Emily and I cooled Clio all day, while Amali and Laird cleaned optics, and Jared moved motors. Amali, Emily, and I are also working on porting the Clio user manual to a new site, after the zero server died. Finally, I fired up the Clio computers, clio1 and clio2, to check how they are working…

Emily did a great job getting Clio from 280 to 77 K today while taking excellent notes and working to port the user manual to a new site. [Image description: Emily is wearing safety glasses and blue gloves, and is holding a metal hose that connects to a black cylindrical dewar. The hose goes into the dewar, and there is boiling nitrogen steam coming out of the dewar in a little smoke plume.]

Today we learned that Clio missed us very much while we were gone. The last time we were here was May 2018. Then I buttoned up Clio and left it in the small room in the Aux. I left the new “clio2” computer plugged in so that we could trouble-shoot some of the new software from Arizona. We did a bit of that, then left it alone for about a year. When I tried to turn it on and check it out today, I couldn’t even ping it or log in remotely. Special thanks to Gabriel Prieto who came over and trouble-shot the ethernet connections, monitors, and finally booted up clio2 in safe mode to discover that poor clio2 had written 30 GB of logs while we were gone. Every day Clio was looking for the AO server, not finding it online, and writing an error. Perhaps every Hertz it was doing this.
So I contacted our software engineer and learned how to stop the process writing those logs (note to future self: touch /tmp/noindi), and Amali verified which logs we could safely delete (since the new clio2 computer is on the same set-up as the LBTI computers), and Gabriel checked and deleted them by size. And we realized that every day, Clio has been here reaching out, saying, “Hey, I’m lonely, is there an AO system around here that can talk to me? No? Ok I’m going to blog about it in my system logs.”
“Dear diary, today the AO system wouldn’t talk to me.”
“Dear diary, today the AO system wouldn’t talk to me.”
“Dear diary, today the AO system wouldn’t talk to me.”

We moved the ASM from the clean room up to the Aux today. Emily and Amali pushed while Felix and Muriel pulled and Juan supervised. [Image description: Inside a warehouse-y looking large room with lots of metal panels and instruments, 2 people are pulling a cart directly towards the camera, while 2 other people are on the other side pushing it. The cart is metal and hold a round adaptive secondary mirror wrapped in green plastic to keep it clean.]

Laird took this video the other day of a Wild Vizcacha hopping about on the hill side near the Magellan telescopes:

[Video description: A rabbit-looking animal with a squirrel-looking tail hops like a kangaroo over some large sharp grey boulders.]

And besides the Wild Vizcacha Laird saw, here are two clean room Vizzies. Can you find both? [Image description: A stone-masonry exterior with wood paneling and a tile roof. A vizcacha sits on one of the wooden beams. Another Viz is there too but a little more hidden…]
There were goats by the Clay at Sunset! [Image description: A somewhat blurry photo of the support posts for the telescope with a small herd of brown goats lit up by the setting sun, with the Andean valley in the background, and mostly clear skies.]

Today was Empanada Sunday and it was delicious! Jared asked me to help him stash some empanadas for later. Don’t worry, Jared, “The empanadas have been taken care of”.
This is a reference to a Classic blog story. See the set-up here regarding a theft of food in the ASB kitchen. And see the reference here: “The problem has been dealt with”.

The song of the day is a classic Lady Gaga song: Bad Romance