The Robot Task Controller (RTC) is the decision-making brain of the robot that performs computationally expensive near real-time tasks like sensor fusion, localization, continuous task optimization, and analysis of visual data. The RTC communicates with one or more state agents to gather sensor data and to control actuators.