Alexander Perzylo

RoboEarth meets the Internet of Things (IoT) at the PICNIC Festival in Amsterdam (Update)

/in , /by

At this year's PICNIC Festival in Amsterdam, RoboEarth held a joint workshop on Robots and the Internet of Things (IoT) with Council, a think tank part of the High Level Expert Group (EG IoT) on the Internet of Things of the European Commission.

The event met with unexpectedly high attendance, resulting in a room packed with an interested and engaged audience and resulting in lively discussion and debate. The main topics centered on how robots could enrich our lives through the Internet and the challenges both communities face to make a vision where the Internet gets hands through robots, and robots greatly benefit from the Internet become a reality.

As the chairman Rob van Kranenburg introduced: 'Rather than programming robots to handle every potential situation, the Internet of Things could create an environment in which the objects themselves inform robots of their purpose and usage. Tomorrow's smart objects can provide sensing, robots can act, processing can be on the robot or in the Cloud. To accomplish this, the fields of robotics and IoT need to define common standards for knowledge storage, representation and communication.'

The topics of debate had clear connection points, and pointed to potential future research questions for RoboEarth, including:

  • Tomorrow's smart objects can provide sensing, robots can act, processing can be on the robot or in the Cloud (e.g., using RoboEarth's Cloud Engine)
  • Rather than programming robots to handle every potential situation, the Internet of Things could create an environment in which the objects themselves inform robots of their purpose and usage.
  • Both the IoT and RoboEarth encode knowledge. The fields of robotics and IoT need to define common standards for knowledge storage and representation.
  • The IoT, robots, and humans need to communicate. The fields of robotics and IoT need to define interfaces and common standards for communication.

For more information, have a look at the article Enlisting Robots - Once robots are integrated into the Internet of Things, they can perform tasks automatically published in the RFID Journal.

Update (Feb 27, 2013):
Even more information can be found in the article The Internet of Things: Robots, RFID & Co-operation published in the December 2012 issue of Elektor.

Alexander Perzylo

Third Internal RoboEarth Workshop (Update)

/in /by

Update (Sep 11, 2012):
Finally, we compiled a video of the demonstrator we created during the workshop including additional explanations of what is going on behind the visible actions of the robots:

The third internal RoboEarth workshop took place  at the Technical University of Munich from February 8th to 12th, 2012, and was directly followed by RoboEarth's second Annual Review meeting on February 13th, 2012.

The RoboEarth Team

The RoboEarth demonstrator developed during the week-long workshop showed how two robots with different hardware and in different locations could use RoboEarth to share knowledge.

First, a PR2 robot in downtown Munich was ordered to serve a drink to a patient, who was resting in a bed in a mock-up hospital room. As a related semantic task description was available in the RoboEarth database, the PR2 could download this information and infer whether its capabilities comply with the task's requirements and what other knowledge it was missing to execute the task, e.g. object detection models and environment maps. It successfully checked the availability of the missing components on RoboEarth, downloaded them and as a result could start executing the task. As the drink was stored inside of a cabinet behind a closed door, the PR2 had to learn the articulation model for that door. After completing the learning process, the PR2 annotated the object model of the cabinet with the learned articulation model for the door and updated it on the RoboEarth database.

Then an Amigo robot in a similar (but not identical) hospital room environment in Garching close to Munich was given the same command of serving a drink. The robot could download needed knowledge from RoboEarth like the PR2 did. This time the articulation model was included, so that during the execution of the task Amigo didn't have to learn it by itself. Amigo was able to grasp the handle of the door and open it right away.

Amigo opening a door

This demonstration showed what a shared knowledge base like RoboEarth including its reasoning services can add to the development of robots:  Robots were able to navigate, recognize objects and perform complex manipulation tasks without being explicitly pre-programmed for these tasks beforehand.

To achieve this goal, all of the involved PhD students and several professors gathered in Munich to work on tomorrow's cloud robotics solutions. The week was characterized by a large amount of work and a limited amount of sleep - and a joint evening at a Bavarian restaurant.

Some RoboEarth members having dinner

Alexander Perzylo

RoboEarth at IROS 2011

/in , /by

Members of the RoboEarth team contributed seven papers to the IROS'11 conference, which took place in San Francisco (USA) from September 25-30th. In addition, RoboEarth supported a workshop on Knowledge Representation for Autonomous Robots.

During the workshop Jos Elfring gave an introduction to RoboEarth's approach to world modelling. It uses a multiple hypothesis filter (MHF) to keep track of objects over time and introduces techniques to improve the probabilistic models by taking prior knowledge about objects into account, e.g. object dynamics, expected locations, relations between object classes and detector characteristics. For more details on this topic take a look at the corresponding paper, Knowledge-Driven World Modeling.

Other papers presented during the regular paper sessions were:

Alexander Perzylo

European Robotics Forum 2011

/in /by

Logo of the euRobotics project

At the European Robotics Forum (Västerås, Sweden, April 6-8, 2011), formerly known as the EURON/EUROP Annual Meeting, the RoboEarth team organized a workshop together with the BRICS and the Rosetta projects, entitled Knowledge engineering in robotics. In this workshop, we informed and discussed with the participants on various aspects of knowledge representation

euRobotics Forum plenary

Topics included:

  • Terminology. E.g., what are task descriptions, action recipes, skills and other primitives, in the context of each project, and what are their relationships?
  • Conventions. Are there shared definitions, conventions (e.g., coordinate systems, units), and data structures between the projects?
  • Scene graphs. How should data be represented (maps, objects, actions)? What data should be annotated and how? What kind of reasoning is performed or needed?
  • Reuse of knowledge. How can a robot decide which knowledge (e.g. map or skill) to reuse in a new situation?
  • Reuse of tools. What existing software modules, algorithms, libraries, or APIs can be reused?
  • Knowledge engineering. How will/should the knowledge base grow? What are the processes leading to creation of a substantial knowledge base useful in real applications, i.e. the bootstrapping of the KB?
Alexander Perzylo

Industrial Advisory Committee workshop

/in /by

An international committee has been formed to have a direct link to the industrial world. This committee consists of representatives of leading industries world wide. On February 7-8, 2011, a dedicated workshop has been organized at the ETH in Zurich, with the following objectives:

  • Transfer of (long term) industrial requirements to the project.
  • Transfer of new scientific knowledge from the RoboEarth project to industry.
  • Exploring opportunities for implementation of the project results.
    • Alexander Perzylo

    Second internal RoboEarth workshop

    /in /by

    The second RoboEarth workshop took place from January 24th to 28th in Eindhoven, The Netherlands. The goal, which was successfully achieved, was to use the AMIGO robot, that was built at the Technical University of Eindhoven, to serve a drink to a patient in a hospital room environment. Therefore the robot should benefit from knowledge shared through RoboEarth.

    The robot itself only had certain basic skills like navigation, moving its arms or using the grippers. These so called action primitives were addressed by corresponding items of a pre-shared action description, also known as action recipe, which was downloaded from RoboEarth by the robot. Starting from the mentioned basic skills, complex actions can be composed that are far beyond the initial capabilities of the robot. The robot's execution engine was responsible for translating the semantic descriptions into physical actions.

    To deal with the challenge, the members of the RoboEarth team had to integrate their different software components and refine the interfaces between them. The time was also used for inspiring discussions and the exchange of views. Despite the amount of work that had to be done, there was some time left for amusement at the local bowling center, where the team could prove its sportiness.
    The great outcome of this strenuous week is an indicator of what RoboEarth can do for robots. It excels at broadening their knowledge and capabilities and increasing their level of proficiency.

    Alexander Perzylo

    Workshop at IROS 2010

    /in /by

    At the IROS 2010 workshop, members of the RoboEarth team participated in the discussion on how the autonomous robotics community may tackle more and more complex manipulation activities, such as doing household chores, in more and more realistic operating environments such as human working and living environments. One of the biggest challenges in such applications is the open-endedness of the task domains and the enormous amount of knowledge needed to achieve reliable task success. Luckily almost every robot has access to the World Wide Web, the world's largest knowledge and information source.

    Alexander Perzylo

    First internal RoboEarth workshop

    /in /by

    'Birth of RoboEarth'

    The first internal RoboEarth workshop took place from July 19th to 23rd 2010 in Eindhoven. All PhD students and several professors met to discuss and work on the initial RoboEarth implementation. Apart from intense work on integrating the different parts, the week also offered the opportunity to discuss interfaces between components, research challenges and future directions. But there were also relaxing moments, like a visit to the museum of the DAF trucks manufacturer, a lab tour through the mechanical department, and a joint dinner.

    During the workshop, a first working system that uploaded and downloaded a recipe has been realized. Though the example scenarios, replaying a joystick-controlled trajectory and learning the path through a maze, are still rather trivial, this showed the general feasibility of the RoboEarth approach.