Discerning Context Through Reinforcement Learning

Santiago, Roberto A., George G. Lendaris (2005), “Discerning Context Through Reinforcement Learning,” Proceedings of the American Association of Artificial Intelligence Conference 2005 (AAAI’05), Pittsburgh, Pennsylvania, July. Submitted

This paper presents a method for using reinforcement learning (RL) to construct an artificial agent capable of applying learning knowledge in a contextually appropriate way (Contextual Reinforcement Learning). In the proposed method, the stream of inputs and feedback used by the machine learning algorithm is treated as a context to the optimal parameters as determined by the algorithm. Thus, each set of parameters has a context in which they are most appropriate. The reinforcement learning aspect of the method seeks to build an algorithm which works across contexts and extracts those features which are most informative about the optimal parameter settings. Thus for novel context, the proposed methods works to extract context features and translate them into an “educated guess” of the appropriate parameter settings.

Relation to my thesis: This paper was one inspiration for my doctoral school paper on Improving Location-Aware Applications Through Reinforcement Learning. I find machine learning interesting because it could give a sense of evolution of the system according to its usage and environment. Interaction is not only done via an interface, the core of a context-aware application could co-evolve with the user.