A glimpse of Markov property

Markov property is a core property in Markov Process, understanding it will give you a broader horizon on Reinforcement Learning. It’s simple that Markov Process doesn’t care about the past, however it is the past that definite the present, which means present is the outcome of the past. Nevertheless, the only thing we should do is focus on the present, because the present will be the past.

So, what we should take into consideration? Remembering all the past is not a ideal method, we should summarize them. From Sutton’s book “What we would like, ideally, is a state signal that summarize past sensation compactly, yet in such a way that all relevant information is retained. … A state signal that succeeds in retaining all relevant information is said to be Markov, or to have Markov property.”




电子邮件地址不会被公开。 必填项已用*标注