Investors are spending a great deal of time debating the scale of the AI build-out. Questions about how many data centres will be needed, how much electricity they will consume, and whether future revenues will justify the capital being committed are dominating discussions across markets. Increasingly, the word “token” is at the centre of these conversations, yet what it actually represents is rarely understood.
This matters because the same unit is now appearing on both sides of the financial equation. Tokens are becoming a cost of using AI within organisations and a key measure of future demand for AI services. To make sense of the economics shaping the AI revolution, it is worth understanding precisely what an AI token is.
What is a token?
A token is a small unit of text that an AI model reads or produces. It is not always a complete word. Short words may consist of a single token, while longer words can be split into multiple tokens. Punctuation, numbers, and even spaces can also contribute to token counts. As a result, a single page of text can contain hundreds of tokens.
Token usage has two visible sides: input tokens and output tokens. Input tokens represent the information sent to the model, while output tokens are the content the model generates in a response.
Generally, output tokens are more computationally demanding because they are generated progressively, one token at a time. Some models may also use additional reasoning tokens during processing to help formulate a response, although these remain invisible to the user.
Why the bill grows
The surprising part is not how tokens are counted, but how often they are counted again. To the user, a conversation feels continuous. In reality, the model is typically provided with relevant portions of the conversation history each time a new prompt is submitted, allowing it to maintain context and understand what has already been discussed.
This means that by the tenth message in a conversation, the model may be processing much of the previous nine exchanges before it even begins generating a response. As the conversation grows, each new interaction can include more context than the last, causing token consumption to rise faster than the visible exchange might suggest.
Agentic AI systems can extend this process even further: searching, analysing, calling tools, and repeating those steps several times before returning a result. What appears to be a single request can therefore involve several layers of processing and significantly more AI consumption than the user ever sees.
Using AI more deliberately
The solution is not to start counting every token, but to use AI more intentionally. Starting a new conversation when the task changes can often be more efficient than carrying forward unnecessary context. Providing clear, complete instructions upfront also reduces the amount of repeated context. Where possible, large documents should be narrowed to the relevant sections, and specifying the length or format of a response can help to avoid generating unnecessary output.
The model selection matters as well. More advanced models are useful for complex reasoning, but not every task requires that level of sophistication. Choosing the right model for the job is increasingly becoming an important aspect of cost management, much like choosing the appropriate level of computing infrastructure for a particular job. By matching complexity to need, organisations can improve both efficiency and value from their AI investments.
The token behind the AI trade
The same token that appears in a firm’s AI usage also sits behind the investment case for much of the AI build-out. Chips, cloud infrastructure, data centres, and power generation are being financed on the assumption that future AI consumption will be significantly higher than it is today. In effect, every AI-related investment contains an implicit forecast for future token demand.
However, increased token volumes do not automatically translate into more revenue. While AI usage appears to be growing, the cost of processing each token has generally been trending lower, and it seems that not every token is directly monetised. Telecoms showed that traffic can surge even as the underlying network becomes commoditised. AI may face a similar dynamic, where consumption expands rapidly while the question of where economic value ultimately accrues, remains far less certain.
For businesses adopting AI, the token is becoming a fundamental unit of consumption and cost. For investors, that same token underpins many of the demand assumptions driving investment in the AI ecosystem.
That is what makes the token worth understanding. The very unit companies are working to manage as an expense is the same unit markets are underwriting as a source of future growth. While the pace of token consumption matters, the more important question may be who ultimately captures the value created by that growth.
Abdur Amod is the head of technology at Prescient Securities.
Disclaimer: This article reflects the personal views of the author and does not necessarily reflect the views or position of Moonstone Information Refinery. It is provided for general information and market commentary purposes only and does not constitute investment advice, an offer, solicitation or recommendation to buy, sell or hold any financial product, or to adopt any investment strategy. Readers should seek appropriate professional advice before making any investment decisions.
