by Jenny Williams
Jenny Williams discusses how life reflects art and art reflects life in her novelette “A Short Future History of Whales”, available to read in our [November/December issue, on sale now!]
On August 21, 2017, shortly after five a.m., a United States Navy destroyer crashed into a 30,000-ton oil tanker in the Singapore Strait. Ten Navy sailors were killed in the collision and 48 more were injured. “It was,” states a ProPublica article that investigated the USS McCain incident, “the Navy’s worst accident at sea in 40 years.”
The cause? Not enemy interference, crew malfeasance, or mechanical malfunction. At fault wasn’t an error of strategy or miscommunication.
The problem was bad design.
Eager to modernize their fleet, the Navy had retrofitted a handful of ships with a digital navigation system that was pitched as reducing staffing needs and increasing control, but was, in fact, “a welter of buttons, gauges and software”–a touch-screen interface that sailors “had been inadequately trained on and that the Navy itself came to admit it did not fully understand.”
In the minutes leading up to the collision, multiple sailors misread the screens in crucial ways that caused the ship to steer directly into the path of the oil tanker. In essence, they’d pressed the wrong buttons in the wrong order.
I read the ProPublica piece when it published in December 2019. At the time, I was working on a team at Google that designed user interfaces (UI) for Google’s cloud products. Our UI didn’t control warships, but it did control how large organizations managed their cloud infrastructure. We had customers in health care, financial services, and transportation, industries whose smooth operations are essential to people’s welfare and wellbeing.
Until that point, during my day-to-day work, I’d often wondered whether writing button labels really made a difference in the world. After the ProPublica story, I understood it could.
During my period at Google, I’d become increasingly aware of the unintended harms that could be caused by advances in artificial intelligence, and in the weeks after reading about the USS McCain, the inklings of a short story began to rise in my mind. I imagined a bigger Navy disaster–because, you know, speculative fiction–and let the possibilities simmer.
Over a long weekend In February 2020, I took my dog to a yurt on the Kitsap Peninsula and plunked out a sketchy draft on a portable typewriter from the 1950s, weaving in childhood memories of California tide pools and the poetic melancholy I’d felt when reading Leslie Jamison’s moving essay on “the loneliest whale in the world” and the Navy technologist who first identified its song. I did research on sonar technologies to come up with a plausible–if not perfect–catastrophic scenario that could result in mass marine extinction.
Inspired by a friend who’d recently told me about her plan to become a single mother by choice, I created a pregnant protagonist to stare down the barrel of the end of the world. I immersed myself so deeply in Phoebe’s character that I started spontaneously lactating, necessitating my first mammogram at age 37. (Results were thankfully normal. My own diagnosis? “Being a writer.”)
That summer–amidst a global pandemic, a sort-of-actual end of the world–I left Google and joined an AI ethics team at Microsoft. The relationship of UI design to real-world harm was no longer peripheral to my work; it was the work. My life had begun to resemble the fiction I’d envisioned.
As I got an insider’s look into bleeding-edge AI technologies, I revisited my story draft with an eye toward the questions I kept coming back to in my job: What does meaningful human control look like for “black box” AI systems? Which tasks should be optimized, and which ones shouldn’t? Who gets the blame when things go wrong?
In 2021, my life took on another uncanny twist: I became pregnant myself, as a single mother by choice, via IVF, just like the protagonist of my still-unfinished story. In between meetings about a then-little-known AI model called GPT-3, I’d feel my growing child press his fist and heels against my taut belly.
The relationship of UI design to real-world harm was no longer peripheral to my work; it was the work. My life had begun to resemble the fiction I’d envisioned.
I became exquisitely aware of a future that would last beyond my own lifetime. What future was I helping to create? Was this a future my son would want to live in? Again, the questions I encountered in my real life seeped into my fiction. I returned again to Phoebe and Estuardo, now more closely attuned to matters of a mother’s heart. I felt a renewed sense of urgency to infuse possible AI futures with a perspective other than male software engineers living in wealthy suburban enclaves.
Steve Almond has said of authors that “We write what we can’t get rid of by any other means.” I wish I could say that writing “A Short Future History of Whales” released me of the questions that haunt me about the role of AI in our world, our institutions, our communities, our selves. But I still grapple every day with the feeling that the sand is shifting under our feet, that the rules of society are changing faster than we are equipped to navigate them.
Already, the AI capabilities I depicted when I finished this story a year ago are outdated, incomplete. The latest AI agents are becoming more powerful and inscrutable, while the UI through which we interact with them is becoming simpler, designed to obscure complexity rather than illuminate it.
The probability that one of us, any of us, will push the wrong buttons in the wrong order increases every day.
And yet.
My son, now, is almost two and a half. He likes bees that sting and airplanes that drone and the colossal koi fish in a nearby pond. His curiosity far outpaces his concern. He marvels more than he fears. I watch his persistent posture of playfulness and I think: Yes, this. This exactly.
The Astounding Analog Companion 