MagAO-X 2022A Day 4: The Backup Burrito

We hit a pretty major roadblock today. Due to the sky high fuel prices around the globe, Las Campanas Observatory has had to drastically reduce use of most forms of energy, including electricity for cranes, propane for forklifts, and gas for trucks, except as needed for nighttime operations. So we essentially have no power when the sun is up.

Our assigned backup burrito for the day. She was game, and did all the heavy lifting we usually have a forklift and crane do. Thanks friend.

Luckily, as we always do on the MagAO-X and LCO teams, we had a contingency plan. This is, of course, the burros who hang around the mountain top. The young lady above was our assigned crane and forklift motor for the day. It took a little longer than normal, but in the end we managed to get MagAO-X unpacked.

Sebastiaan inspecting the inside of MagAO-X. Looks like nothing is broken.
Our only small problem was a broken dessicant bag, which dribbled a bit. Here Joseph is holding a side panel up while Sebastiaan vacuums.

Yesterday Sebastiaan mentioned our misbehaving graphics cards (GPUs). When we turned the electronics on, 0/4 GPUs that we use for our super-fast real-time computations were alive. There is not much in common between these 4 devices in terms of where/how they are installed, so it seemed extremely unlikely that they all just quit on us. So after none of us getting much sleep, in part due to racing minds trying to figure out what it could be, we divided our duties today so Joseph could troubleshoot while Sebastiaan and I worked with the crew on unpacking.

In the end it was two different problems. One of them just needed a stern talking to (nerd code for we took it out and put it back in, and it worked). The other 3 were more complicated. We use a PCIe expansion system, which lets us attach more crap to our computers. The backplane of that failed, but luckily we had a backup burrito for this too, and once it was put into service we now have all 4 GPUs working.

Our control system electronics, with the comforting glow of a 2080 Ti FTW card visible.

When we went up to the top we saw a vulture soaring on thermals. It was coming right over us, and we could hear a whistling sound from its feathers.

We have some good close ups, but I like this background.

The best news of the day was our friend Vizzy the cleanroom vizcacha making an appearance. This was when I knew the GPUs would be ok.

Vizzy!

The song of the day accurately reflects my sentiments after what was a damn stressful 24 hours.

MagAO-X 2022A Day 2: Way To Go Past Us

When we last left LCO we expected to be back in just 3.5 months. We’re nerds of the science and engineering type, so we tend to take lots of notes and pictures because we think we want to remember this or that. But there are always those things where you don’t even think about not remembering it because it’s so obvious, or around here it’s because we have done it that way every 6 months for the last 10 years.

Well, guess what. 2+ years of being totally distracted by global catastrophe fries your brain and you forget a lot of things. So I spent a lot of time today rummaging around all of our junk at LCO trying to piece my brain back together. Several times I came across a nicely labeled box, or a place where someone collected all the bolts that go with the thingy, and said out loud “way to go past us!”.

But also there are cases where there’s nothing on the hard drive and I just stand there like this:

You can briefly see Sebastiaan and Joseph sitting next to the fires of AOC there at the end wondering why I’m mumbling to myself

The big accomplishment of the day was unpacking our AO Operators Computer (a.k.a. AOC, a.k.a. exao1) and getting it set up. Joseph and Sebastiaan assembled Megadesk (without breaking it) and got AOC plugged in, on the network, and monitors sorted out.

Somehow, even after all of this time, you always see something that you haven’t before.

We also had our first Earthquake of the run. I was up in the cleanroom after dinner, and announced it on Slack. Sebastiaan was down in the lodge and saw my slack message before he felt it.

Just a long slow rumble that made the roof rattle, nothing moved though.

We also unveiled the new attire for cool-kids at LCO

MagAO-X rests in the background, waiting for crane day.
These are the handiwork of MagAO-X’s own lithography expert Avalon McLeod. I’m 100% sure she’ll write a post about how she does it once she gets down here. (hint hint)

There are some Burros around, as you’ve seen in previous posts. Today I’d like to highlight the Burritos. There is one hanging around with Ma up here at the lodge, and another down at the Bodega.

Lodge Burrito (right)
Bodega Burrito (Left)
Hungry Lodge Burrito

And of course, there are the sunsets. Oh how I have missed the LCO sunset ritual.

So good to be back.
In 10 days we won’t get to see this, cuz we’ll be working already.

Us first 3 have just 1 more day of “bubble”. Assuming negative tests tomorrow evening we’ll get to join normal life at LCO. So far, we’ve just been able to say “hi!!!” to our friends from a distance. We’re really looking forward to actually getting to reconnect.

The song of day’s video sorta captures what it feels like to look back through a decade’s worth of blog posts to find that thing that you forgot.

Watch for Lynx!

Desert Hackberry Pie: A Perilous Quest

This is a recipe post in the traditional internet style.

Part I: The Rains

We have had a fairly incredible monsoon season. It just keeps raining and raining here in Tucson, and everything around us is enjoying it. Since this is a science blog, we should start with evidence for these assertions.

The plot shows our last 2.8 years of rain according to our backyard rain gauge. This year, we have had two months which individually exceed all of 2020. Combined those two months exceed all of 2019. And it might not be over.

My house is at the outlet of Ventana Canyon, and the Ventana Wash runs through our backyard. These next two videos are from a game camera we use to monitor the various goings on back there. The first one shows one of our neighborhood bobcats trotting by, with the wash down to just a trickle:

We can go several months at a time with at least some water flowing. Since the Bighorn Fire last year, the flow has gotten exceptionally strong at times due to the increased runoff. Next is just a day or so after the bobcat video, showing what happens after an inch of rain falls in the canyon (same camera, same spot):

Now all of this rain has led to an abundance of rainbows and a wonderful lushness to our desert foliage.

The view towards Sabino Canyon. At left is a big old Mesquite Tree, right (light green) is a very happy Palo Verde tree. In the middle of the frame is the star of this blog post: a thicket of Desert Hackberry.

Part II: The Hackberry

We have two big bushes, and one thicket with several bushes, of Desert Hackberry. Here’s a view from the back of the thicket in the previous snap:

I’ll send you over to the Arizonensis page to learn more about the natural history of this plant. The main takeaway is that you can eat the berries, but I also noticed the butterflies. I actually witnessed the butterfly combat described, and hanging out around the thicket was lovely:

You can hear the running water in the background.

This is the first year we even noticed the berries, and we have TONS. So of course I decided to start picking and, well, we’ll get to that.

Hackberry harvesting. Note the toes.

The above picture shows the fruits of my labor, mid-harvest. This much takes about 30 minutes. We have a saying in our house: “Everything outside wants to kill you, even the plants.” The hackberry is no exception. Thorny, and the berries tend to be embedded behind small twigs and the thorns.

So I ended up spending about 4 hours on labor day afternoon, sun on my back, butterflies flitting about (and fighting), the wash running next to me, picking hackberries. It was wonderful!

Part III: The Guardian

After my wonderful nature-full day, I decided to do a little maintenance in the back yard. The Hackberry thicket tends to get a little overgrown, and I have trimmed it back a few times to keep the lower parts under control. This time I also had in mind that I was now an expert Hackberry picker, and wanted to optimize things.

So I grabbed my big pruning shears and my nata tool and set to work. I had gotten a good chunk out, and was getting down low to go under to cut from the inside out a bit. I stuck the shears in to get a big branch and . . . MONKEY BRAIN CIRCUITS ENGAGED. JUMP JUMP JUMP was the response from my brain as my highly tuned image processing CNN took over when my eyes focused on this:

The biggest freaking diamondback I have yet to interact with personally. Hackberries for scale, and to document how long she was there!

She (I’m going with she) is absolutely gorgeous, and please note just how well camouflaged. And gorged! Her mid region (under the head) looks very bulging, so I’m guessing one of the bunnies that triggers the trail camera frequently is no more. She was also super chill, no rattle, just flicked the tongue and watched.

Here’s the thing: I was within 1 meter of her for several hours. My bare, flip-flopped feet were maybe closer depending on the lay of various branches. She never made a peep, and could have easily struck many times. I feel like I was judged worthy. Or maybe she was just too sleepy to care…

Interlude and Song of the Day

Part IV: The Pie (Finally)

So after all that picking, I next had to clean. Luckily it turns out Hackberries sink. So a water bath for about 1 cup at a time allows lots of the twig and leaf matter to float and be strained out. They are also just large enough that a standard slotted spoon works as a strainer. Perhaps the most tedious part was berry-by-berry inspection to get residual stems off.

After all that, I had a little more than 4 cups of Desert Hackberries!

My harvest.

Now what to do? The internet was not super helpful. Some discussion of foraging and eating them while hiking, a few hints at making energy bars and wine, but nothing really satisfying.

The berries have a hard pit, but you can crunch it and eat it. The taste is complicated: sweet, but with a sort-of green apple + green tea aftertaste or bite (I’m not sure that’s right, but it’s hard to describe). I decided that I wanted to get the pits out, and after a bit of kitchen lab work, found that our food processor handled this perfectly:

in motion

and a salad strainer worked perfectly to drain:

Separated

and in the end about 4 cups of berries made 2.25 cups of berry pulp:

The juice that was worth it

Well great. Now we have 2.25 cups of Hackberry pulp but still no clear direction. So we decided to try a blueberry pie recipe because what the hell. It was a bit of a meander, I document it here for posterity.

2.25 cups Desert Hackberry pulp (about 4 cups whole berries)

1/3 cup granulated sugar

2 tbsp tapioca flour to thicken (or corn starch)

1/3 tsp salt (cut in half next time)

1/3 tsp cinnamon (tasted like too much before baking, maybe ok after. TBC)

1 tbsp lemon juice. this is common in blueberry pie, but did NOT work here.

1/3 cup granulated sugar (to counteract the lemon juice)

Note: error bars on the above quantities are estimated at +/- 5% unless otherwise specified.

This will fill one small pie pan:

Finally, I dotted the pie with 1 tbsp of butter chopped into flakes. Then baked at 375 for 50 minutes. And here’s the result:

That’s supposed to be a Saguaro.

Verdict: it was ok! We had a friend over, the one who first told us about Hackberries in fact!, and went for it `a la mode. Vaguely pumpkin pie in texture and smell. The cinnamon was a little overdone. The main challenge is to figure out what to do with the green-apple/green-tea aftertaste I described in the fresh berries. I’ve decided that, based on the lemon juice mistake, that the issue might be being to acidic. So note for next time: try adding some baking soda to knock it down.

Plan for next time is to: (1) inspect under all bushes thoroughly and (2) make small batches in ramekins to experiment with different buffers and spices.

Epilogue

It’s now been a couple days and I’m eating the last piece. Also my fingers have started to heal from the thorns, but a few splinters are still working their way out. The pie is still good. I def think the baking soda is worth a shot. I feel like I’m on to something, this might be the pumpkin pie of the Sonoran Desert!

I’ve also been thinking about the snake, a lot. That was quite a scare, and frankly I’m still a bit jumpy outside. But also, we interact with them fairly often up here in the hills. I encounter them when on runs, crossing our driveway, and we had one in our compost bin once. Only one time have I heard the rattle, and that was from a baby that I was chasing away from our front door with a looooooong pole. There’s this idea that rattlesnakes are evolving to rattle less, but I’m skeptical. She wasn’t scared and not rattling because I might hurt her, she just didn’t care. I leave you with this link, and note that “peaceful sighting” is a great way to describe this: Are rattlesnakes evolving to rattle less, or losing their rattles?

The Monsoon Begins

After a long, dry, and recently HOT year, it appears that the monsoon has begun!

Today felt like the first snow used to feel when I lived somewhere more North.

The only downside to the monsoon is that the storms often keep us from operating our sensitive equipment. Here’s a real-time lightning map from this afternoon, showing the storms advancing on Tucson from the South.

Lightning strikes shown as the colored dots, white most recent, getting dark with age. That’s Tucson in the middle of the image.

Thanks to Joseph’s hacking kills, our friendly lab-assistant Vizzy keeps us informed if there is lightning nearby.

We had to shutdown our fancy deformable mirrors and sensitive cameras this afternoon, but there’s always software to debug and documentation to update and tests to study for and referee comments to answer. So we enjoyed the thunder and were happy for the rain.

The Sabino Rainbow made an appearance. I went for a blessedly cool run into the canyon, but didn’t find the end.

The song of the day is about the monsoon rains, and it also needs to be turned into a movie. It sounds like an intense adventure.

Sunset Here But Not There

The sun has just set here in Tucson. But we were supposed to be watching the sunset at LCO tonight, taking our usual break from preparing our instrument for a night of observing. On the schedule, this was to be our first of eleven nights exploring the sky with MagAO-X.

An LCO sunset taken from the Clay catwalk, from what seems like forever ago:

Spoon – “Got Nuffin”