Amali’s blog post about the saga of the Alpaca was another creative expository piece that tonight we had a ceremony to award her the 2019B MagAO-C blog award:
Jared gives the citation and Amali graciously accepts the award [Image description: Amali and Jared stand on the Clay catwalk with the setting sun behind them. Amali is holding a metal item.]Here is the blog award close-up. [Image description: The metal item turns out to be a failed hard drive from the adsec supervisor computer, the very computer that Alpaca was impersonating in IP address. Writing on the harddrive in black sharpie in Jared’s handwriting says “adsecsup / Failed, replaced / 11/15/15”. The harddrive has been decorated with the 2019B Mission Patch sticker, and labeled “MagAO 2019B Blog Award / Awarded to Amali Vaz / 2019/11/11 [not shown]”.]
Tomorrow there will be a general strike throughout much of Chile. The turno (shift changeover) that was supposed to happen will be postponed until Wednesday. That means those who hoped to go home tomorrow must stay 1 more day. We wish our colleagues all the best — they are all extremely professional and we truly appreciate working with them here.
The MagAO-C and MagAO-X teams and observer: Emily, Katie, Laird, Kyle, Jared, Amali, Joseph, Alex (v. 2.0), and Blake. [Image description: A group of AOistas stand on the Clay catwalk in front of the setting sun]. Photo by Alberto Pasten.Blake didn’t get to do his targets to the North because of the wind. Also, the seeing got pretty non-LCO-ish at the end of the night, with a couple spikes above 1.5”. [Image description: Screen grab of the LCO weather plot showing seeing, temperature, wind, and pressure vs. time.]Moon set at dawn. [Image description: Foreground: A ridge with telescopes. Background: Purple mountains majesty. Sky: Band of Venus, orange of dawn, and Full moon setting.]
The song of the day is Michael Franti & Spearhead – Say Hey I Love You — a classic from MagAO 2015B:
This morning, Laird and Alex rotated the K-mirror to a more optimal position and re-aligned the rest of the optical train. I switched out a board in an ALPAO driver to one that lets us power the NCPC (non-common-path-corrector) DM remotely, which saves us many hypothetical trips to and from the Nasmyth platform. Jared did…well, a lot of everything.
Alignment specialists, aligning.
The innards of an ALPAO driver
One tool that has proven useful in the alignment process is an “F”- and “X”-shaped pattern on the 2K deformable mirror (previously featured here), which we can view in high resolution on our LOWFS (low-order wavefront sensing) camera.
After Alex and Laird were happy with the alignment, I drove out (most of?) the remaining system aberrations with the “eye doctor” script, which attempts to find the combination of Zernike modes on the deformable mirrors that maximizes our Strehl.
Alignment F-Test on the 2-Kilo DM, as imaged on the pupil viewer
Instrument PSFs on the pyramid tip (left) and science camera #1 (right), post alignment and eye doctor. We have airy disks!
Reinforcements in the form of Joseph Long arrived this afternoon, although fumigation delayed his luggage in Santiago and left him (temporarily, we hope) dispossessed of his worldly goods (and a few supplies for MagAO-X). The MagAO-C team joined us after dinner for a tour of MagAO-X.
The united MagAO-C and MagAO-X teams walk up to the halfway house after dinner.
The tour.
After the tour completed and the crowds dispersed, Alex and Laird reinstalled the panels, shielding the instrument from prying eyes (and unwanted particles—of both the massive and massless variety). Jared installed the blower (the large white tube entering the instrument table from the left) to supply the optics with a soothing breeze.
After spying a guanaco in the far distance earlier in the week, I finally encountered two up close yesterday. They were surprisingly…pungent.
A guanaco who may be Gary. Yes, I recorded vertically.The guanacos survey their domain.
As always, we stopped to watch the sunset.
Another beautiful LCO sunset, another disappointing lack of green flashes
For the song of the day, I present “Stairway to the Stars” by the Queen of Jazz herself, Ella Fitzgerald.
Tonight was finally the promised night of relaxed AO operation after multiple nights of troubleshooting. It is much more to an AO operators delight to only need to open and close the loop for new targets with the occasional MatLab pop up forcing a reset on the tip tilt. Alberto, our telescope operator, was able to tell us his operational comedies of the past and share pictures of his lovely family. Later today is the Mercury transit! Tonight is also my first blog post so an introduction to the outside world is due.
I’m Emily Mailhot, a new AO operator for not only MagAO-C but also LBTI and eventually MMT Maps, both in Arizona. My equally new counterpart, Jared Carlson, and I have been learning the ins and outs of adaptive secondary operations, concepts, and frustrations – and are excited to do so.
This trip to LCO has created numerous “first experiences” for me personally. First time leaving the United States, first time on a B787-8, first time seeing the southern sky, first time experiencing sub-arcsecond seeing (both AO operationally and visually), the list goes on. Considering this, a life at LCO post from brand new eyes seemed appropriate.
Food at LCO
When I was informed that our meals would be served on site I was both relieved and skeptical. At LBTI we are responsible for feeding ourselves through the night, vending machines on the 5th floor of Steward Observatory get a lot of traffic. But would this provided food be delightful or would I need to start preparing for numerous high school band-camp cafeteria flashbacks? I am happy to announce that the food is absolutely superb. From the empanadas to the night lunches to the cookie filled dome kitchens – you have to try very hard to go hungry. Meals are at set times, 730 breakfast, 1230 lunch, and 1830 dinner, but for most of the run breakfast and lunch is sleep for the observers. Instead we fill out night lunch forms which are located in the main kitchen for observers to fill out by 1600 each day. It’s a checklist of various sandwich combinations or your can write “plato de cena” if you’d prefer a plate of that evenings dinner. It is the telescope operators job to collect all the night lunches after dinner and bring them to the telescope.
For those at home wondering what we are eating here’s a selection:
Empanadas are only served on Sunday for lunch so you better get your fill while you can. They come in two forms, carne or con queso, both equally delicious. This particular Sunday there was also veggie and seafood soup.
The kitchen below the Clay Telescope control room.
The life sustaining cookie cabinet.
The “claimed” MagAO table.
Delicious soup.
Avocado at every breakfast.
Colorful dinners.
The TO’s night lunch basket.
Classy night lunch sandwich.
Clearly more than enough food to go around.
Wildlife at LCO
The wildlife here at the summit is diverse. This blog has many posts about the endearing vizzy’s, but the burros, horses, and guanacos are just as abundant. Today on a run around the grounds I encountered both guanacos and horses running along the ridge below the road. They seemed to be chasing each other in what resembled a wild wild west setting. It was so magnificent to capture that I stopped the video clip too soon and I instead watched in awe.
On another run earlier in the week, towards the du Pont Telescope, I encountered a large herd of burros controlling the road. One was particularly annoyed that I was trying to pass and gave me a stiff snort.
A herd of burros serving a strong stare-down.
Of course, this wouldn’t be complete without a vizzy. This sunrise viscacha (a Chilean cocktail?!) watched us sleepy observers wander back down the summit, only to be met by this eight legged, fanged creature. He was small but mighty, creeping up toward my hand each time I tried to open my door. Not usually being afraid of spiders, a small fear for them was gained after the long dance it took to be allowed into my home, room 5.
A sunrise vizzy.
It took a while to get into my room this morning.
The fox on the other hand kept distance and the goats clumsily climbed to the summit with us.
A fox waiting for lunchtime.
Goats on a guardrail.
The goats followed us all the way to the Clay Telescope dome where inside the control room us tired observers tried to troubleshoot through our engineering night after a long install day.
Goats under Clay.
Wild control room, feat. Amali, Laird, and Katie
Work at LCO
Of course the animals only provide us entertainment between times of work – like our extremely long install day and engineering night (see Day 3!). Here’s a glimpse of the Clay control room, AO work station, and some pre-science pupils.
An image of the control room before sunset. The back left station is for science camera, Clio. To its right is the telescope operators station. Left front is the VisAO station and finally next to it is the adaptive operators station. During troubleshooting events this room can be full of people, standing room only.The AO station! GUI’s upon GUI’s upon GUI’s.
The pupils before dark, secondary baffles visible.
The pupils watch the dome open!
Beauty at LCO
LCO is nestled in the most beautiful landscape, best described through photographs.
The first sunset of the trip from the porch of the “hotel” dining room.Taken from behind the Summit near sunset. This was the first clear night since we had arrived.The peak for the future Giant Magellan Telescope taken from the telescope operators housing. Clay dome open for science.Clay still at the zenith, the ASM backlit by the sky.Baade and Clay awaiting a night of science.A picture of the open dome from the nasmyth platform, taken during the night of the misbehaving baysides.
And my favorite photograph of all, taken after dinner on our long day transitioning into our long night.
The stairway to the stars.
Keeping with classic song theme, here’s Led Zeppelin performing Stairway to Heaven.
MagAO-X is alive! After being boxed up, shipped from Tucson to Phoenix to LA (we think — a little fuzzy) to Miami, with a long pause, then to Santiago, braving the dangers of revolution (and customs (and customs strikes!)), and a trip by truck (always touchy, this time with road blocks!), and finally being craned out of its box and carefully reconstituted, we can still close the loop.
Needless to say we are happy, a little bit relieved, and excited to get our new instrument on the Magellan Clay Telescope.
A clear example of this phenomenon is when we had to roll our electronics rack out of the clean room so Nelson and Emilio could cut off a stripped 3/8-16 bolt. The Aluminium L-bracket you see there is part of the earthquake restraint system, which, as Amali pointed out last night, is always a thing here.
A pano of the cleanroom when we were just getting started this morning. That’s Kyle checking on the DM humidity before turning it on.
So here’s the big moment when we finally knew that we didn’t f it all up:
And here you can see our vAPP coronagraphs looking good in a deep exposure:
vAPPs looking eXtreme. Laird wants you to know that there are no optical ghosts in the dark hole. Maybe the eye doctor has a little work to do though.
Moonrise over the Andes on my hike up.
Laird and Alex watching tonight’s sunset. Photo by Kyle, I’d already given up — obvious non-green-flash night. That’s our electronics shipping crate, btw.
Tonight’s song is “Wanted Dead or Alive”, by Bon Jovi:
Tonight, gentle readers, a tale of drama and derring-do! A mystery was solved, a dastardly plot uncovered, an insidious impostor unmasked, several computers rebooted, a culprit nabbed, and, despite its most vicious efforts, the criminal’s final revenge borne without injury. I present: The Alpaca Saga!
THE CASE
Yesterday night the adaptive secondary — aka adsec, asm, shell, TS1, and as of last week Lollipop — began to display some troubling symptoms. For instance: every so often, even in the gentlest of conditions, the shell would suddenly RIP (don’t worry, the falling-to-pieces is only metaphorical). That in itself isn’t terribly uncommon, but the circumstances were a bit strange: the adsec would briefly report missing values for elevation and for windspeed. A shriek into the void! The wail of an injured creature in the darkness! But then the elevation and windspeed would both return, pretending that nothing was amiss.
Obviously, something was amiss.
Simultaneously, in the once-peaceful environs of the Clio workstation, bayside stage focus moves were being sent, as messenger pigeons into a clear blue sky…and then snapped up by an invisible space dragon, never to return. By which I mean, the bayside stage at the WFS never moved. And yet, it could be moved without problems from the WFS directly.
THE INVESTIGATION
What do absent elevation, unreadable wind speed, and unheard Clio commands have in common? They’re all communications sent through the computer that masterminds the MagAO processes: the Magellan adsec supervisor, or magadsecsup.
A CLUE!
In the dead of night, Jared bravely took his life in his hands and went down to the equipment room — is that its name? the one with the MagAO computers and racks and occasional unstable ladders — anyway, Jared found A CLUE!! Which was: the little blinky lights on the ethernet ports connecting magadsecsup and the network switch were neither happy-working-green or transmitting-information-amber, but instead an alarming shade of ORANGE. So Jared and Emilio beheaded a cable and put it in a different switch port.
But the crimes did not stop!
We hobbled through the night. The adsec wept. The adsec RIPped (still metaphorically). The notion of a criminal in our midst weighed heavily upon our minds. Dawn came and we fell into uneasy slumber…
A BREAKTHROUGH!
During the day, while the rest of us slept, Jared (life still in hands) disconnected magadsecsup. AND YET…he could still ping its IP.
Note, gentle readers: if you remove a computer from the network, you should most certainly NOT be able to ping it. Which meant that, instead of only magadsecsup responding to madadsecsup, we had … AN IMPOSTOR! GASP! SHOCK!
We put the sleuths of LCO onto the case. At dinnertime we received word that the culprit had been found…and it was none other than Alpaca, one of the TCS computers at Baade!
THE PART WHERE THE DETECTIVE EXPLAINS EVERYTHING
Somehow, alpaca.lco.cl had been given the same IP address — one ending in 35 — as is statically assigned to magadsecsup.lco.cl. Consequently, when the Clay TCS sent information — elevation, for instance — to magadsecup, they were sometimes intercepted by this nefarious agent and never made it to the real adsec. Chaos! Upheaval!
Here I provide a helpful diagram to assist in distinguishing a genuine adsec from a sneaky alpaca.
A guide to distinguishing a true adsec from an alpaca impostor.
Anyway, once the impostor was summarily booted from the scene of the IP, and Lollipop comforted and cajoled into returning to service, the night proceeded without further loss of elevation.
Emilio says the real solution is to update the hosts file on alpaca.
ALPACA’S LAST KICK
But! Just as we returned to science, we were subject to one last act of villainy! In a fit of rage and envy, deprived of its chance to be cool like magadsecsup, Alpaca threw a fearsome tantrum. Being somewhat larger, the actuators of an alpaca are capable of a much larger throw, and can cause much larger aberrations. And, because it’s networked, this particular alpaca managed to cause severe turbulence IN THE VERY EARTH, somewhere 33km northeast of Vallenar…
In other words, we had an earthquake! It was fascinating and bizarre! As though the world was swimming. I got to push the “earthquake button” on the adsec GUI, which Laird had pointed out to me only a few hours earlier!
Apparently the “Force enable TSS” button works by setting the shell windspeed reading to 500 m/s. Seeing that number was honestly more alarming than the earthquake…
Anyway, here are a few more helpful diagrams to cement the reader’s understanding of the situation. One is from the Chilean national seismological center. The other is…not.
Why yes, it was a perceptible earthquake. Even the Earthquake Officials agree.ALPACA SMASH
MORAL OF THE STORY
Don’t judge a villain by its actuators. And watch your IPs.
SONG OF THE DAY!
Here’s a song about another creature up to no good. May all of us here at MagAOs both -C and -X have more luck with our problems than the hapless Inspector Clouseau.
