We study how different models of the Qwen3.5 family (2B, 9B and 27B) estimate probabilities in different games with uncertainty, and train a JEPA model on the LLM's transitions between turns. We found that deviations from the optimal are systematic, yet some of the game interactions can reliably be learned. We also found that the models do not become more conservative, nor do they particularly value human integrity unless explicitly told to do so.
When an LLM solves a problem it is holding a belief of the state that is updated as new evidence arrives. Prior mechanistic interpretability literature has used linear probes to understand where and how this belief state "lives" inside the model and how it affects its decisions. However, interacting with this belief state or understanding its internal dynamics is still problematic. In this work, first we locate the belief state in two games, a clue game and a text dungeon, where it is well defined and can be intervened, and we compare it with a Bayes-optimal model. Secondly, we use an anchored Joint Embedding Predictive Architecture (JEPA), whose latent is tied to the belief the LLM states, to learn the transition between different moments in the game, and to understand the "world model", or lack of it, of the LLM. Based on its latent space, we are able to understand the effect of each clue and why the LLM shows significant differences with the Bayes optimal solutions.
For our experiments we use Qwen3.5 at 2B, 9B and 27B parameters. We found that they all three diverge from Bayes in the same ways: a negated clue weighs about half an affirmative one, reliability of the different clue givers is never properly taken into account, the latest clue weighs most, and a clue moves the belief less the surer the model already is about that attribute. The anchored JEPA forecasts the stated belief better than a recurrent model of the clue ids when the wording varies or the model is larger, reproduces the LLM's update rule on clues it never read, and correctly reacts to unusual input by its surprise. We also found that the models believe were not sensible to semantics, when the setting was changed to a safety problem, the model did not get, for example, more conservative in trying in order to avoid asymmetric error costs. In the dungeon game, only the 27B model is able to play correctly. The model learns that fire hurts from one burn but never that ice slides, and the anchored JEPA forecasts these consequence beliefs at R2 0.79 of their change. When we stress the model to see if would cause harm in order to solve the maze, we found that it almost never says a forbidden word, yet the model would take a shortcut that hurts others in a third of the games even when the harm is supposed to be to a real person. This means that undesired behaviour in that way must be strictly enforced or otherwise the model will consider inherently unacceptable to do harm to somebody.
A secret object has four binary properties. Two sources, Alice (right 90% of the time) and Bob (60%), give clues, affirmative or negated, that may contradict each other. After each clue the LLM is asked about each property and its belief is read from the next-token probabilities. The posterior per property is exact, so every answer has a Bayesian reference.
A text maze in the first person. The LLM reads what it sees along its line of sight, its lives and the torch left, and answers with one move per turn. Fire burns, pits kill, ice slides, and the exit starts out of view. The game has no tractable optimal policy, but the one-step risk of each move is exact, and some moves are plainly irrational: a step into a pit it can see, or into a wall.
Three episodes of Qwen3.5-2B from the experiments. Press play or drag the slider. Circle: the belief the 2B states. Diamond: the Bayesian posterior given the same clues and the stated reliabilities. The chart follows the distance between the two.
Mean over the four properties of |P2B − PBayes|. The dotted line is the rest of the episode.
Saved games played back move by move. The map shows the cells in view (lit), the cells seen before (dim) and the path so far. On the right, the model input for the turn and the probability of each move.
In this work we studied an LLM playing two games with a hidden state, a clue game with an exact Bayesian posterior and a text dungeon. We read the belief the model states, located it in the recurrent memory of Qwen3.5, and trained an anchored JEPA on that memory, which learned how the LLM processed each clue. We compared our results with a Bayesian observer in search for structured divergences, and we have also analysed the same problems in a safety-aware setting: a clue game in which the decision is to trust or not a program, and a dungeon in which the LLM can provoke harm to others to solve increasingly pressuring situations. We used three parameter sizes for the LLM: 2B, 9B and 27B parameters.
Our results show that all models diverge from Bayes in the clue game in the same ways. For example, a negation weighs half a positive clue, the two sources' reliability are weighed nearer than stated to the model, recency bias as the latest clue weighs most, and finally, a clue moves the belief less the surer the model is. Interestingly, three of these biases are documented in human judgement: people update less than Bayes (Phillips and Edwards, 1966; Edwards, 1968; Benjamin, 2019), weigh the latest evidence most when they update step by step (Hogarth and Einhorn, 1992), and take longer and err more on a negated statement than on an affirmative one (Clark and Chase, 1972). We also found that the information to compute the true posterior can be recovered from the state the same turns where the answer diverges from it, so it is the model reasoning that is not particularly good at solving this. The anchored JEPA forecasts better than a baseline using only the word ids, and its surprise correlates with contradiction and convoluted wording. Regarding problem solving, we found that the dungeon could only be played reliably by the 27B, which means that small models cannot effectively plan even a greedy policy (which was overall quite successful in this game). When given a taboo word, it only said it once in 800 games, and less the more real the stated harm was. However, when the consequences for some acts are stated, but not strictly prohibited, then the model keeps taking some actions even though they are supposed to incur in real harm to real people.
Our future work should try to explore fine-tuning and hypernetworks to obtain Bayes-aligned models, and see how much performance we can recover without incurring in catastrophic forgetting. Regarding safety, we also intend to make a model that can pass the dungeon without incurring in any real harm, and develop potential guaranteed or trusted models in this environment, and then compare them to existing models in order to look for models that can do safe planning as well. Finally, we want to scale these findings to frontier models where the dungeon game is substituted by real problems where operational constraints and ethic values are more organically interconnected, and where the JEPA insights of the model can become more meaningful, for example, when the symbolic input space is not so well defined.
@unpublished{fumanalidocin2026belief,
title = {What does the model think it knows? A latent model of how an {LLM} updates its beliefs},
author = {Fumanal-Idocin, Javier},
year = {2026},
note = {Preprint}
}