Bolmo and the Hidden Trade-Off in How AI Reads

Key takeaways

• Most LLMs process text as tokens whose boundaries depend on the tokenizer.
• Byte models use a common encoding unit across languages, though characters can require different numbers of bytes.
• Converting an existing LLM to bytes requires adapting both its inputs and its outputs.
• Longer sequences can increase costs, so comparisons need to measure actual work completed.

A million-token context window sounds generous until you ask how much actual text fits inside it. The answer depends partly on the tokenizer, which breaks your documents into the pieces a model reads. Bolmo explores changing those pieces to bytes while carrying over the capabilities of an existing LLM.

The tokenizer shapes what fits

Before a typical language model can process a sentence, a tokenizer splits the text into pieces and assigns each piece a numerical ID. A token might represent a whole word, part of a word, punctuation, or another frequently occurring string.

This is useful compression. If a familiar sequence fits into one token, the model can process it as one unit instead of working through every character.

But the vocabulary of available pieces reflects the tokenizer’s training data and design. The same content can require different token counts in different languages. An unfamiliar name or a typo can also produce a very different split from a familiar word.

That makes character count a poor guide to token count. It also means the model must recognize meaning across whatever boundaries the tokenizer happens to give it.

For anyone feeding multilingual documents or code into an LLM, those boundaries have practical consequences. They help determine how much material fits into a context window.

Bytes offer a simpler foundation

A byte contains 8 bits, giving it 256 possible values. A byte model uses these small encoding units as its basic representation of text, reducing dependence on a learned inventory of word fragments.

The distinction between bytes and characters matters. In UTF-8, the English letter “A” takes 1 byte. The Korean syllable “한” takes 3 bytes. Switching to bytes does not give every written character an equal-sized representation.

What it does provide is a common encoding system. A new product name or a sentence mixing several languages can be represented through the same byte values, with less riding on which strings were bundled into the tokenizer’s vocabulary.

Existing tokenizers can already handle unfamiliar words, often by splitting them into smaller pieces or falling back to bytes. The change is making bytes the default processing unit.

This is also what to keep in mind when someone says “tokenizer-free.” Text still needs to become numbers. The learned word-fragment vocabulary loses its usual role; the encoding step survives.

Changing the alphabet without losing the knowledge

The central idea behind Bolmo’s approach is byteification: converting an existing language model to operate on bytes.

Starting with a trained model offers an obvious attraction. It has already learned language patterns and developed capabilities that might be carried into the new representation. Preserving those capabilities could reduce the burden of starting from scratch.

The catch is that a model’s reading and writing machinery is built around its original tokens.

On the input side, it has learned to work with numerical representations associated with those tokens. On the output side, it predicts the next item from that token vocabulary. Changing the unit requires adapting both ends.

Knowing a language does not instantly make you fluent in a new writing system. A similar distinction matters here: an existing model’s knowledge does not automatically become accessible through a new input format.

Successful conversion therefore needs to demonstrate more than the ability to accept bytes. It needs to show how much knowledge and reasoning ability survives, and how much additional training the transition requires.

Smaller pieces can create a larger bill

The trade-off starts with sequence length.

A string that previously occupied one token may require several bytes. If the architecture processes each byte individually, the longer sequence can increase computation and memory use. It can also change how much actual document content fits within a fixed sequence limit.

Generation has a similar constraint. A model producing one byte per step may need more steps to finish the same sentence. Designs that group bytes can change that calculation, so speed depends on the architecture.

This makes tokens per second a potentially misleading comparison across models that use different units. The useful question is how long each model takes—and what it costs—to read the same document and produce a comparable answer.

Multilingual quality needs the same scrutiny. A common byte representation does not guarantee equally efficient processing across languages, much less equal understanding. Training data, its quality, and the amount of it still matter.

Bolmo’s central question is whether an existing model can change how it reads while keeping the capabilities that make it useful. The payoff will show up in accurate answers, usable document capacity, and manageable costs—the things people notice after the architecture diagram leaves the screen.