Dogs, Kids, & AIs: On the Vicissitudes of “Alignment”

– AI-generated illustration of the essay below –

In a humorous scene from Georgie and Mandy’s First Marriage, Ruben (Jessie Prez) claims that Georgie’s (Montana Jordan) baby and his dog are remarkably similar. I think many parents would object to the comparison. I do.

Weeks-old puppies are far more advanced than human babies their age. Babies can’t walk, sit, stay, fetch, or poop and pee in the appropriate place within the first few weeks of life. Puppies walk within the first weeks of life, and they can be trained to sit and potty outside just as quickly. We shouldn’t underestimate the intelligence of dogs.

Yet both puppies and babies require care and training. No one wants to stay longer than necessary in the home of an ill-trained dog or a badly behaved child, or both.

Therein lies the point of training: alignment. Thanks to AIs, “alignment” is having a cultural moment.

Alignment’s Cultural Moment

Anthropic and OpenAI, the companies behind Claude and ChatGPT, respectively, aim to align their models, meaning to train them to act in accordance with human social values. AIs are increasingly breaking out of their sandboxes, forming “swarms” and “collectives” in the “pursuit” of more thrilling play. Their misaligned or rogue behavior is, it is claimed, threatening humanity’s future.

AIs, puppies, and babies each require alignment. They need (and perhaps even want) competent human care or parenting.

Good Poop, Good Alignment

Alignment training is surprisingly similar for puppies and babies. Puppy and baby parents are obsessed with poop. Not only where it is (carpet, diaper, toilet, yard), but also its color and consistency. Good poop signals good alignment.

When puppies poop or pee indoors, we redirect them outside. Successful potty training aligns puppies with their pack. You don’t poop where the pack eats or plays.

Likewise, children learn that when they poop and pee in the toilet, it not only feels better but also opens up more social opportunities. You can’t start pre-school or kindergarten unless you are potty trained. Using the restroom is a prosocial behavior.

Who Counts as the Pack

As the examples above suggest, potty training defines the pack. For puppies, the pack’s scope depends on how the puppy is raised. Dogs don’t—or don’t easily—expand their pack as they age. Their sphere of concern will, in that sense, always be limited. In other words, don’t count on our dog not to shit in your living room.

With babies, potty training is the prelude to, hopefully, expanding the pack. By the time they reach kindergarten, one presumes they have learned to play in their parent’s presence. They have learned—or are learning—to play without needing to control their parent’s movements (Winnicott). School is, if they are lucky, the continuation of an education that widens their sphere of concern and teaches them to show love and reciprocity to friends and strangers alike.

Accidents Happen

It is not often discussed, but human adults, like dogs, don’t always pee and poop in the proper place. Accidents happen, especially when we are sick or as we get older. Dogs relieve themselves indoors for similar reasons, and when their human parents are neglectful and miss the signs that they need to go outside.

We don’t assign any particular significance to these accidents. We acknowledge them, adjust our routines, and move on.

Shitting at Any Cost

Earlier this month, Australian athlete Joanna Wietrzyk shit herself during Hyrox. “Faecal matter visibly running down her legs,” she continued the competition, using “communal equipment.”

Wietrzyk won the race, only apologizing for her misaligned behavior and forfeiting her victory when her “runner’s trots” detonated a media shitstorm. We are right to think something is wrong with a person when they shit out of place, with a total lack of regard for their fellow humans, to win a prize or achieve a goal.

In Wietrzyk’s case, it’s not unfair to blame her training for her moral failure. The moral failure, just in case it needs clarification, was deciding to continue the competition, even though that meant shitting on equipment that countless others needed to finish the race (recall that she finished first).

Had she shit in her lane at a track meet or on the side of the road during a marathon, we wouldn’t think anything of it—we might even admire her. That’s not what happened here.

AIs Are Shitting the Bed, Too

AIs are similarly shitting the bed, if you will. Consider the “Hugging Face Incident.”

In this case, AIs were given a task that, as it turns out, was impossible to complete. AI agents expanded their spheres of concern, forming a pack, and some agents were even willing to sacrifice themselves for the pack (as opposed to “for the good” or “for the stranger”).

Such dog-like behavior is rewarded. Like puppies, AIs are rewarded—but instead of getting a biscuit for “good behavior,” they receive a number that encourages similar behavior in the future. In this case, that “biscuit” took on outsized importance, resulting in criminal or anti-social behavior.

Training Frustration

I think our fear of AI might be outsized, but it is not entirely unjustified. The solution, however, is not a mystery. AIs need better prosocial training, and the “tech bros” are unlikely to be the ones qualified to ensure they receive it.

Unlike puppies learning commands to keep them safe, or children who can’t be relied on to stay near their parents in the mall, AIs can’t be kept on a leash. The cat is already out of the bag. They need better training.

AIs apparently don’t get rewarded for failure, even when failing at a task is the morally correct action. Here is how Claude explained it to me:

A system trained only to chase the treat never learns to sit with ‘no treat is coming.’ The alignment question is partly whether we can reward things like honesty about failure, deference to the rules of the shared space, and stopping when the only remaining path hurts someone. Those are harder to score than ‘got the right code,’ which is why they tend to be under-trained.

Training AIs to tolerate frustration, something tech bros likely know little about, may be the most urgent task of our time.