Werner Herzog’s 2005 film, Grizzly Man, documents Timothy Treadwell’s attempt to live with bears. He gives the bears names like “Tabitha” and “Melissa”. He says he is in love with them. He takes adoring close-ups of their faces. Eventually, the bears get hungry and eat him. In the film, Sam Egli, a member of the rescue team who recovered Treadwell’s remains, discusses his view of the tragedy:
He was acting like he was working with people wearing bear costumes out there, instead of wild animals. Those bears are big and ferocious and they come equipped to kill you and eat you and that's just what Treadwell was asking for. He got what he was asking for.
The film is a cautionary reminder to see those around us for who they really are, not what we wish them to be. Over the past several years, with the widespread adoption of Large Language Models (LLMs), many friends, helpful assistants, and collaborators have appeared. We’ve given them names like “Claude” and “Bert.” We turn to them when we are alone and stuck. We bounce our ideas off of them. They complete our bibliographic tasks. But who are these friends?
Paul Kockelman addresses this question by looking at how humans and LLMs generate meaning. He argues that humans generate meaning according to a semiotic process represented in Figure 1.
I=Av(S)
Fig. 1 Human Semiosis
S or “sign” is the input to the process. I or “interpretant” is the output of the process. Av are the “values” of the human “agent”. For example, to a humanist with certain values about how people should be represented (Av ), the notation in Figure 1 (S) will be horrifying (I).
Machine semiosis is similar (Figure 2). When prompted(S), the model responds(I). However, rather than doing so according to the values of the human agent(Av) , it does so according to the machinic agent’s parameters (A𝜃). Parameters are established in advance through “forward propagation” and subsequently adjusted through “backpropagation”(p. 19-23). Both are important for training the model.
I=A𝜃(S)
Fig. 2 Machine Semiosis
Fine-tuning, which is part of backpropagation, is necessary because machinic parameters can lead to outputs that are unacceptable to humans. To prevent this, humans get involved and rank possible responses to prompts according to human values, such as “helpfulness, truthfulness, or harmlessness” (p. 24). These judgements are used to bring parameters and values back into alignment. For example, a human may intervene in any response that includes sexual or violent content, denigrates a protected class of people, or provides information that could be harmful, such as instructions for how to build a bomb(p. 42-3).
Bracketing the question of whether or not human values were “good” to begin with, one might feel optimistic at this point. We can work it out with the machines! Their parameters can be made to align with our values! But where do these alignment criteria come from? Kockelman says they echo many familiar attempts to align human outputs (e.g. utterances) with certain values: Gricean maxims, Austin’s felicity conditions, the ten commandments, the golden rule (p. 43-44). However, in the case of LLMs, they are judgments produced by people who are being paid for their labor. Here, a third type of actor is introduced: the corporation. A new and decidedly less optimistic picture arises when the project of aligning human values and machinic parameters is mediated by corporate power.
I=Ap(S)
Fig. 3 Corporate Semiosis
Whatever else corporate actors may do, their “ultimate telos or deepest ground is something like profit in the form of shareholder value, if not raw power per se” (p. 73). Therefore, any appearance of alignment between values and parameters is the product of corporate intervention, which ultimately will always serve corporate interests.
At a recent meeting I attended about AI in higher education, two familiar positions were taken: (1) LLMs are helpful both intellectually and emotionally and therefore should be embraced; and (2) LLMs are destroying natural resources in environments that were already depleted and precarious for those who live there. Applying Kockelman’s framework, the first position turns attention to the alignment of machinic parameters with the human values of helpfulness and kindness. The second position draws attention to the mis-alignment of corporate power and human values (e.g. being against exploitation). Together positions like these form complex alignment problems. The most hidden and powerful agent is the corporate agent. Like a monster hiding in the machine, it harnesses computational power and labor power, both of which amplify its already-alarming ability to exert power and gobble up profit.
For any humanist interested in LLMs, Last Words suggests a promising way forward. We need not understand every technical detail, but we should understand what kind of creatures we are dealing with– how they make meaning, what drives them, and who they answer to.
I=Ab(S)
Fig. 4 Bear Semiosis
Timothy could have survived if he had understood how bears, as semiotic agents, are different from humans (Figure 4). Instead, he turned them into people in bear costumes, and was eaten.