Learning Visual Routines
Visual routines extract the required information from a scene which is needed for a vision agent to solve tasks ranging from visual search to execution of user-specified commands such as to move objects around. We are looking at the learning of routines that can tell an agent what aspects of its visual input to focus on , with motivations drawn from cognitive theories of representation. These visual routines then act as the feature extraction units that the higher levels in the architecture choose from.
THE BIGGER PICTURE
Our work in this area can be organized as several levels in the cognitive stack. At the bottom level comes the extraction of base patterns in an image which can then serve in object recognition at the next level. We look at autonomous characterization of object features by supervised/unsupervised learning methods. The next level involves composition of objects and identification of the sequence in which they are to be focused on , using reinforcement learning methods. We also look at deictic representations for selective attention on the relevant parts of the image. A skills library can be built on top of this layer to provide different functionalities to the agent , in an incremental way. The topmost level concerns planning the execution of the task at hand. This may require planning with gaps which will again be filled in by learning methods. On the whole, we are working on formulating efficient methods for solving a query/task.
- Sponsored Project: Learning Visual Routines using Reinforcement Learning, funded by UKIERI, Jointly with Prof.Jeremy Wyatt, University of Birmingham.
VISION GROUP MEETING
Monday Feb 1, 2010
- Discussed the bigger picture of our work
- Decided action plan for next week
Tuesday Feb 9, 2010
- Discuss on the work done by University of Birmingham
- Will be discussing the thesis proposal of Jose (Phd Student, Birmingham)