Simplifying the Creation and Management of Utility Models in Continuous Domains for Cognitive Robotics

Abstract

[Abstract] Establishing goal/sub-goal hierarchies in robotic motivational systems for open-ended learning situations and modelling this utility in a manner that is useful for robots is an open research problem, especially when the robots’ state-spaces are continuous and may present ambiguities. In these cases, directly obtaining value functions, and in particular, precise Artificial Neural Network based value functions, becomes very difficult. In this paper, this issue is addressed through a new type of coarse utility functions for the representation of motivation. The proposed approach can be used as an intermediate step in order to be able to produce more consistent data for the subsequent training of precise value functions when and where it becomes necessary. This type of coarse utility functions, called Separable Utility Regions (SUR), are based on the use of the variation of sensor values as clues to the position of goals in state space. Moreover, areas in the state-space must be established where there are correlations between the desired direction the system should follow in its state-space towards a goal, and the direction of variation of the values of a particular sensor. The main focus of this paper is on the process of creating sub-goal hierarchies that permit leading the system in a consistent manner towards goals, so that it can autonomously learn to achieve them whatever its starting state. To this end, an approach to sub-goal determination and chaining based on a recursive establishment of consolidated goal domains as new goals for new utility functions is described. The approach is tested on a real robotic system and the results are extensively analysed and discussed.