I have been reading Sutton’s new book Reinforcement Learning: An Introduction(2nd edition) for many days, and what made me confused was the deterministic model. What is the deterministic model?
Mathematical model in which outcomes are precisely determined through known relationships among states and events, without any room for random variation. In such models, a given input will always produce the same output, such as in a known chemical reaction. In comparison, stochastic models use ranges of values for variables in the form of probability distributions.
In deterministic models, the output of the model is fully determined by the parameter values and the initial conditions. For example, the multi-arm bandits problem in Sutton’s book, if the value of an arbitrary action is selected according to a normal distribution, say with mean 0 and variance 1, the model is stochastic, if for each action, the value is fixed for example, that is deterministic.