Abstract
In this paper, we study learning and knowledge acquisition (LKA) of an agent
about a proposition that is either true or false. We use a Bayesian approach,
where the agent receives data to update his beliefs about the proposition
according to a posterior distribution. The LKA is formulated in terms of active
information, with data representing external or exogenous information that
modifies the agent's beliefs. It is assumed that data provide details about a
number of features that are relevant to the proposition. We show that this
leads to a Gibbs distribution posterior, which is in maximum entropy relative
to the prior, conditioned on the side constraints that the data provide in
terms of the features. We demonstrate that full learning is sometimes not
possible and full knowledge acquisition is never possible when the number of
extracted features is too small. We also distinguish between primary learning
(receiving data about features of relevance for the proposition) and secondary
learning (receiving data about the learning of another agent). We argue that
this type of secondary learning does not represent true knowledge acquisition.
Our results have implications for statistical learning algorithms, and we claim
that such algorithms do not always generate true knowledge. The theory is
illustrated with several examples.