By Mikael Brunila
Brunila, M. (2025). Cosine capital: Large language models and the embedding of all things. Big Data & Society, 12(4), https://doi.org/10.1177/20539517251386055. (Original work published 2025)
Large language models (LLMs) have taken the world by storm since the release of ChatGPT in November 2022. The promise of Artificial Intelligence (AI) evolving into Artificial General Intelligence (AGI) is prompting companies to invest increasingly substantial sums in these technologies. This mania is only paralleled by quickly growing fears of automation and replacement among intellectual and creative workers of all stripes.
Looking beyond these dominant narratives on AI, I wanted to take a closer glimpse at the first principles of language modelling to understand (1) how they make sense of the world and (2) what kind of fundamental power asymmetries might result from their proliferation. Through examining Word2Vec, an early neural language model, I show that language models (whether large or small) extend a metaphor of communication that was established during early information theory and the encoding of data as “bits”. Instead of reducing the world into zeros and ones, LLMs encode words into “embeddings”, a series of hundreds of numbers that are “learned” as the model is trained on massive troves of textual data. What is more, any sequential data, not only text, can function as the input for producing embeddings. Text, images, behavioral profiles of swipes on dating apps, listening preferences on streaming services, clicks in browser sessions, and DNA sequences can all be “embedded”.
To describe the consequences of this new regime of modeling, I introduce the concept of “cosine capital”. This term is informed by two lines of inquiry. On the one hand, “cosine” refers to the measure used to evaluate how similar two embeddings are. On the other hand, “capital” describes the manner in which embeddings are accumulated over time. As technical systems increasingly rely on embeddings, our interactions with these systems end up producing data for the fine-tuning of more and “better” embeddings. This is what happens when we use ChatGPT, for instance. This movement, where companies that control a technology end up accumulating more and more of it, is reminiscent of Marx’s understanding of value as something that is always in motion. Cosine capital, then, is my attempt to theorize how LLMs act as harbingers of a new paradigm of quantifying the social world.
The article is supplemented with a GitHub repository that dives deeper into the technical relationship between bits, entropy, and embeddings.
