The field of reinforcement learning research requires objective measures that track the hidden updating of internal models, as they occur. While it is well established that pupil-linked arousal is sensitive to reward prediction errors, the degree to which this physiological signature persists when participants are not explicitly aware of task contingencies, remains subject to debate. We examined this relationship using a probabilistic reinforcement learning task, and a computational q-learning framework to capture participant’s learning dynamics across the task duration. A dual model of awareness was employed incorporating a binary categorisation, and a continuous measure of task knowledge. Results demonstrated that post-feedback pupil dilation reliably tracked the magnitude of reward prediction error. While there was not sufficient evidence for a modulating effect of binary awareness on the pupillary response to absolute reward prediction error, continuous task knowledge emerged as a significant predictor of this psychological signal. Additionally, we demonstrated a strong internal consistency between participants ability to accurately monitor their performance, and their overall task knowledge. We conclude that this physiological response to surprise is not a fixed reflex but rather may be modulated by the accuracy of a participant’s internal model. Our evidence suggests that explicit task knowledge plays a critical role in regulating internal learning signals, during reinforcement learning.