Abstract
If
X is a countable state space and
g is a bounded reward function on
X
n
, then say
g has the Markov-adequacy property if every strategy has a corresponding randomized Markov strategy which gives
g the same integral as the original strategy. A complete characterization of functions having the Markov-adequacy property is given. In particular, if
g is permutation-invariant and
X has at least three elements, then
g has the Markov-adequacy property if and only if
g has the linear sections property, a condition which is easy to verify.
