Understanding how agents learn and make decisions under uncertainty is a fundamental question in many fields, with applications including real options, R&D, and entrepreneurial ventures. The conventional approach formulates this learning process as an optimal stopping problem within a Bayes framework, assuming agents possess the cognitive sophistication to continuously update their beliefs based on statistical principles, thereby rigidly locking their decisions to these updated beliefs and forcing a strict, deterministic threshold rule. This paper develops a continuous-time reinforcement learning framework for sequential experimentation that formally separates beliefs from actions. By decoupling the evaluation and policy processes, we provide a unifying framework that yields both normative benchmarks and flexible positive dynamics. Normatively, using the workhorse Gaussian bandit model, we prove that by properly tuning learning rates, the framework achieves a logarithmic regret bound, matching the efficiency of Bayesian rationality. Positively, the decoupled policy generates distinct and testable predictions, such as experience-driven, path-dependent quitting dynamics, even when the belief is consistent with its Bayesian counterpart. Crucially, we prove the structural identifiability of these hidden learning dynamics. By utilizing the method of simulated moments, we demonstrate how this framework can be structurally estimated directly from censored observational field data and extended to general jump-diffusion bandits.