Our team NimbRo@Home won the RoboCup@Home Open Platform League (OPL) competitions of RoboCup 2024 in Eindhoven, NL. Our
final score was 8,852 points. Runner-up was Tidyboy (Korea, 7,495 points), followed by SocRob@Home (Portugal, 6,901 points).
21.7.2024
In the final, we used two robots simultaneously. They demonstrated
open-set object recognition, human-robot interaction by speech and
gestures, mobile manipulation, and integration of large languague
models. For this, they received the hightest score, both from the league-internal and from the league-external jury members.
20.7.2024
In the Restaurant test, our robot served two guests and
received the
highest score among all 17 RoboCup@Home teams (OPL + DSPL + SPL) that
advanced to Stage 2. n
the Stickler to the Rules test, our robot detected and resolved rule
violations by party guests. NimbRo@Home scored more than Tidyboy and is
now leading the RoboCup@Home OPL competition. Our robot also scored at
the Extended General Purpose Service Robot (EGPSR) test. After Stages
1+2, NimbRo@Home is leading the RoboCup@Home OPL competition.
19.7.2024
In the Serve Beakfast test, our robot grasped a spoon
and a bowl and plece them on the table. Our robot received the highest
score of all 29 RoboCup@Home league teams (OPL + DSPL + SSPL) in the
General Purpose Service Robot test. It solved the task to find the
smallest object on the kitchen counter completely and solved the task
of getting the pear from the office partially. After Stage 1, our
team NimbRo@Home is ranked 2nd, next only to last year's winner Tidyboy
(KR). In the Clean the Table test, our robot grasped a fork and placed
it in the dishwasher.
18.7.2024
In the Carry My Luggage test, our robot grasped a bag referended by a
pointing gesture and followed the user with it.
In the Receptionist test, our robot welcomed two guests and introduced
them to the other guests. Our main @Home robot suffered a major
hardware defect. With the prompt
help of PAL Robotics, we could identify and fix the problem within a
day. Many thanks to PAL! The issue caused a low score in the Storing
Groceries test.
17.7.2024
Our team NimbRo participates in the RoboCup@Home
Open-Platform competition (OPL) in Eindhoven, NL.
NimbRo won the competition with a final
score of 86 followed by ToBI (University of Bielefeld, 63). [Press Release]
20.4.2024
In the Final, our robot demonstrated indoor navigation and the use of a
foundation model for open-world object recognition and multimodal
human-robot interaction including speech-based dialogue to support
persons for making dinner.
This demonstration was ranked first by both the internal and external
jury members.
In the last Stage II test "Stickler for the Rules", our robot had to
find guests violating house rules, and help them comply, e.g. by
guiding them to the entrance to take off their shoes.
After Stages I+II, NimbRo is leading with 3331 points, followed by ToBI
with 2445 points.
19.4.2024
After the second competition day, our team leads with 2431 points,
followed by ToBI (1745
points) and SCC.
18.4.2024
Our team participates at the RoboCup
German Open
competition in Kassel. After the first competition day, NimbRo is
leading with 1271 points, followed by ToBI (Bielefeld University, 445
points) and SCC (Ravensburg Weingarten U.a.S., 355 points).
8.7.2023
In the Restaurant test, our robot detected a waving
customer in the unknown restaurant, navigated to it, took an order,
retrieved it from the bar and handed over one ordered item to the
customer. In the overall Open Platform League (OPL) ranking, NimbRo
reached the 4th place.
[RoboCup_2023_NimbRo@Home_Highlights.mp4]
7.7.2023
With the third highest number of points, our team advances to Stage II
of the competition.
6.7.2023
Our @Home robot performed well on the first competition
day. It scored 775 points in the Receptionist task and 500 points in
the Carry my Luggage task.
4.7.2023
Our team NimbRo participates at the RoboCup@Home competition in
Bordeaux, France.
Our team NimbRo won the 5M Grand Prize in the ANA Avatar Xprize
Competition.
Futher information: NimbRo Avatar
 |
|
7.11.2016
The article by Jörg Stückler, Max Schwarz and Sven Behnke:
"Mobile
Manipulation, Tool Use, and Intuitive Interaction for Cognitive Service
Robot Cosero"
appeard in Frontiers in Robotics and AI, vol. 3, 2016.
24.7.2014
In the final, our robot Cosero demonstrated the use of tools. It grasped a dustpan and a swab in order to clean some dirt from the floor. Unfortunately, the dirt detection failed. The robot executed the cleaning motion and continued the demo by pouring out the contents of the dustpan into the dustbin. It placed the tools back on a table and continued the demo by starting to make caipirinha. For this, it used a muddler to muddle lime pieces. Our robot came in third in the overall ranking, behind Wright Eagle (China) and TU/e Eindhoven, Netherlands.
23.7.2014
In the Coctail Party test and the Restaurant test, our robot had problems with object recognition and height estimation of the table, respectively. In the Demo Challenge, Cosero answered the door and received a package. He signed the delivery protocoll and carried the parcel with both arms to the user. We also demonstrated localization of objects tagged with Bluetooth low energy senders in the environment, which was instrumented with four receivers.
22.7.2014
The Basic Functionality test went very well. Our robot Cosero demonstrated object recognition and grasping, navigation in the arena where an additional obstacle was placed and a door was closed, and question answering (speech recgnition and synthesis). In the Open Challenge, Cosero demonstrated the use of tools: It grasped a bottle opener and a bottle and opened the bottle. There was not enough time for picking a saussage from the BBQ, which we also had prepared. After Stage 1, our team NimbRo is leading the competition with 3756 points, followed by Wright Eagle (China, 3755 points) and TU/e (Eindhoven, Netherlands, 3401 points). In the Enduring General Purpose Service Robot test, our robot recognized two complex speech commands and carried out the requested actions.
21.7.2014
On the first competition day, our robot Cosero handed out sweets to spectators in the Robot Zoo test. In the Follow Me test, Cosero learned the face of a human guide, followed it even though another person walked between the guide and the robot, went into an "elevator" with the guide and followed it out again, and relocated the guide successfully again after the guide went through a crowd of people.
17.7.2014
Our team NimbRo@Home participates in the RoboCup 2014 @Home Leauge
competition in Joao Pessoa, Brasil.
Press
release.
5.4.2014
Our team NimbRo won the @Home league of the service robots at RoboCup German Open 2014.
In the final, our robot Cosero demonstrated the use of tools. It opened a bottle and took a saussage from the grill. The robot also demonstrated bimanual grasping and transport of a tray. Our team reached a perfect normalized score of 100, followed byTech United Eindhoven (59 points) and ToBI (Bielefeld University, 50 points).
4.4.2014
Our team NimbRo participates at the RoboCup German Open
competition in Magdeburg in the @Home League. After Stages I and II, we
are leading the competition with 7919 points, followed by Tech United
Eindhoven (3611 points) and ToBI (Bielefeld University, 3069 points).
30.6.2013
Our team NimbRo won the @Home League competition at RoboCup 2013!
In the final, our cognitive service robot Cosero demonstrated tool use. He used a pair of tongs to cook a sausage on the grill and a bottle opener to open a beer that he served to a person. This demonstration convinced the high-profile jury which awarded the highest number of points. Together with the results of Stage 1 and 2, the final normalized score was 99 points for NimbRo, followed by Wright Eagle (China, 86 points) and TU Eindhoven (Netherlands, 73 points).
29.6.2013
Our robot Cosero performed two tests. In the Restaurant test, it followed a guide and learned a model of an unknown environment. It also used speech recognition to take orders. In the Demo Challenge the motto was health care. Our team demonstrated a handheld teleoperation interface. The robot also moved a chair to its location. Cosero also attempted to open a bottle with a tool.
28.6.2013
Today, our robot Cosero scored in all three tests. In the Emergency Test, it guided a person who did not the way to the exit of the arena. In the Clean-up test, it found two objects and put one of them on its place. In the EGPSR test, it understood a complex speech command and fetched an object.
27.6.2013
The highlight of today's Open Challenge was the demonstration of grasp transfer. Our robot Cosero registered a model of a known watering can with an unknown, different watering can. The non-rigid registration allowed to transfer grasp points and the tooltip position to the new watering can. Our robot grasped the can successfully with both hands, used it to water a plant, and placed the can back on a table.
26.6.2013
Our team NimbRo@Home participates at the @Home League competition of
RoboCup 2013 in Eindhoven with the robots Dynamaid and Cosero.
On the first day, the robots went through inspection and registration.
28.4.2013
Our team NimbRo@Home won RoboCup German Open 2013.
In the final, our robots Dynamaid and Cosero cooperated. Dynamaid
took an order, transmitted it over WiFi to Cosero who delivered a drink.
Cosero also pushed away a chair and Dynamaids delivered drinks and
snacks to the jury.
Our performance received the highest score from bot the execs and the
external experts. Together with our lead from Stages 1 and 2, we
reached the maximal normalized score of 1, followed by SmartBots@Ulm
(score .67) and team Homer (Univ. of Koblenz, score .61).
27.4.2013
In the Demo Challenge, Cosero was teleoperated by an immobile
user who could command the robot navigation and manipulation on three
levels of autonomy using a tablet. Cosero fetched items and handed them
over to the user.
In the Extended General Purpose Service Robot test, Cosero had to
understand several complex speech commands and to solve the requested
tasks.
In the Restaurant test, Cosero followed a human guide and build a map
of an unknown environment with marked object positions to which it
navigated on spoken request.
After Stage 2, our team NimbRo is leading with 6297 points, followed by
SmartBots@Ulm (3242 points) and ToBi (3198 points).
26.4.2013
In the Clean-up test, our robot Cosero showed that it can recognize
object which were distributed in the arena. It grasped a drink and
medicine from the fridge and delivered it to their respective
locations. While placing the beer on the couch table, it spilled it,
however. Afterwards, it found two more drinkson the dinner table and
attempted to store them when the test timed out.
In the Emergency Situation test, Cosero guided a person to the exit.
Our robot served as a butler in the Open Challenge, delivering drinks
to waving persons. After Stage 1, NimbRo is leading with 4247 points,
followed by SmartBots@Ulm (2792 points) and TOBI (2748 points).
25.4.2013
Our team NimbRo participates at the @Home League competiton of RoboCup German Open 2013 in
Magdeburg.
On the first competition day, our robots Dynamaid and Cosero registered
for the competiton by handing over the registration form.This RIPS test
was evaluated by the team leaders and the Technical Commitee. Our
robotsreceived the top score of 925 points.
A mechanical problem in Cosero's torso prevented to participate in the
Follow Me test.
The Cocktail Party test went very well. Cosero took orders, fetched and
delivered drinks.
After the three tests, our team is leading (1775 points) followed by
SmartBots@Ulm (1561 points) and TOBI (Univ. of Bielefeld, 1471 points).
23.6.2012
Our team won the @Home League competition.
In the final, our robot Cosero demonstrated the approaching, bi-manual grasping and moving of a chair to a target pose. It also approached and grasped a watering can with both hands and watered a plant. After this demonstration, our robot Dynamaid fetched a drink and delivered it to the jury. In the meanwhile, Cosero approached a transport box, from which it grasped an object using grasp planning. This demonstration convinced the high-profile jury, which awarded the highest number of points in both categories (league-internal jury: scientific contribution, relevance, presentation and performance; external jury: originality, usability, difficulty and success). Together with the lead after Stage 2, our team received 100 normalized points, followed by eR@sers (Japan, 74 points) and ToBi (Univ. of Bielefeld, 64 points).
22.6.2012
In the General Purpose Service Robot test, our robot Cosero had to
recognize complex speech commands. It correctly recognized two
commands, which gave the most points in this test.
In the Demo Challenge, an immobile person used a handheld PC to
teleoperate the robot. The person sent the robot to fetch a drink. The
robot recognized that the requested drink was not available and the
user could select another drink in the transmitted camera image. After
the robot delivered the drink, it recognized a pointing gesture and
navigated to the referenced object in order to pick it up from the
ground.
In the Restaurant test, our robot Cosero way guided through a
previously unknown bar. The guide showed the robots where the shelves
with drinks and the individual tables where. Our robot built a map of
this environment and took an order. Afterwards, it navigated to the
food shelf to search for requested snacks.
After Stage 2, NimbRo is leading with 6938 points, followed by
WrightEagle (4677 points) and eR@sers (Japan, 3547 points).
21.6.2012
In the Clean Up test, our robot Cosero had to find objects that
were distributed in the apartment, recognize them, and bring them to
their place. Our robot detected three objects, from which two were
correctly recognized as unknown objects. It grasped all three objects
and deposited them in the trash bin.
In the Open Challenge, Cosero demonstrated that it could recognize a
waving person. It took over an empty cup from this person and threw it
into the trash bin. Afterwards, it approached a watering can and
watered a plant. After finishing all test of Stage 1, NimbRo is
leading (5071 points), followed by WrightEagle (3398 points) and ToBi
(2627 points).
20.6.2012
Our robot Cosero performed the new Follow-Me test. It learned the
face of a previously unknown person, followed it although its
line-of-sight to the person was blocked temporarily, and went with the
person into a small elevator. At another floor, the robot went out of
the elevator again.
In the Who is Who test, Cosero performed very well. He learned the
faces of three persons, took an order, fetched three drinks, and
delivered two of them, before the time run out. After the third test,
NimbRo is leading with 2399 points, followed by WrightEagle (China,
2098 points) and ToBi (Bielefeld, 1283 points).
19.6.2012
Our team participates at RoboCup 2012 in Mexico City. On the second setup-day, the robots had to register for the competition while Jörg presented our team poster to the other team leaders.
1.4.2012
Our team won the RoboCup German Open @Home competition.
In the final, our robot Cosero showed the pushing of a chair and the
watering of a plant. For both tasks, it used real-time registration of
3D models with the current RGB-D camera images. Our robot Dynamaid
delivered some snacks to the jury. Jury members could raise their hands
to get a snack. This performance was awarded the highest number of
points by both parts (exec and external) of the jury.
The team b-it-bots (Hochschule Rhein-Sieg,Sankt Augustin) reached the
second place in the @Home competition. Team Golem from Mexico came
in third.
31.3.2012
In the Demo Challenge, our robot Cosero recognized colored and white
laundry shown to it, recognized pointing gestures, and picked up
laundry from the floor. It also put away two kitchen items into
the sink.
Cosero performed the Enhanced Who Is Who test almost perfectly. It
learned names and faces of three persons, understood a drink order,
fetched three drinks and delivered two of them to the correct person.
The Shopping Mall test also went very well. Here, Cosero learned the
map of an unknown environment by following a guide, took a shopping
order and collected two of the three requested objects.
Consequently, after Stages 1 and 2, our team is leading with 8550
points, followed by b-it-bots (4781 points) and SmartBots (4291 points).
30.3.2012
In the Who Is Who test, our root Cosero found four persons and
recognized three of them.
In the General Purpose Service Robot test, Cosero understood three
complex speech commands and found the requested objects.
The last test today was the Open Challenge. Here, Cosero was guided by
a human by taking its hand, carried a table together with the guide,
and pushed a chair to the table. After the second day, we were leading
the competition with 5300 points, followed by SmartBots (Ulm, 3491
points) and b-it-bots (Sankt Augustin, 3431 points).
29.3.2012
Our team participates at the RoboCup
German Open 2012 competition in Magdeburg.
On the first day, our robots had to register themselves and to perform
the Follow Me and Go Get It tests.
In Follow Me, our robot Cosero learned the face of a previously unknown
person and followed him trough a parcour. The robot was not disturbed
by another person walking in between the two, recognized its leader
again after he disappeared, and recognized a stopping gesture.
In Go Get It, Cosero found the requested object in the kitchen, but
could not reach it, because it was outside the workspace of its arms.
After the first day, our team is leading with 2340 points, followed by
SmartBots (Ulm, 2004 points) and Golem (Mexico, 1604 points).
11.7.2011
At the 15th RoboCup International Symposium, the paper Jörg Stückler and Sven Behnke: "Compliant Task-Space Control with Back-Drivable Servo Actuators" received the Best Paper Award.
10.7.2011
We won the @Home competition.
In the final, we used both robots. Cosero carried a table, together with a person who guided the robot by pulling and pushing the table. Cosero also turned on the cooking plate, fetched some egg dough and poured it into a pan to make an omelet. In the meanwhile, Dynamaid got a bottle of orange juice from the fridge. This performance was awarded the highest score from all three parts of the jury. Together with the top score from Stage II, this resulted in an optimal 300 points.
9.7.2011
The General Purpose Service Robot test went very well. Our robot Cosero
understood complex speech commands, fetched the requested object, and
was not confused by a command to fetch an object that was not at the
specified location. In the Demo Challenge, Cosero cleaned the
apartment. A human could show the robot where white and colored loundry
was to be put and the robot collected some clothes. It also cleaned up
other objects and put them at their place. This was awarded with the
highest number of points of all remaining 12 teams. After Stage II, our
team is leading with some distance, followed by WrightEagle from China.
8.7.2011
In the General Purpose Service Robot test, Cosero partially understood
a complex command and went to the correct place. The Go-Get-It test
went well. Cosero found a known object and delivered it. In the
Shopping Mall test, Cosero had severe difficulties with speech
recognition. Hence, it could not take the order which items should be
shopped and received only partial points.
7.7.2011
Cosero demonstrated in the Follow-Me test that he can follow a
previously unknown person, recognize gestures and distinguish this
person from another person. Due to the noisy environment, Cosero had
some difficulties to understand names in the Who-is-Who test.
Nevertheless, our robot detected and recognized four persons correctly.
In the Open Challenge, Cosero prepared a breakfast. It opened a bottle,
poured milk in a bowl and fetched a spoon. This performance was
rewarded with the highes number of points of all teams.
6.7.2011
Our team NimbRo participates at RoboCup
2011, which takes place at the Expo Center in Istanbul.
In the evening, our @Home robots Dynamaid and Cosero registered
themselves for the competition in the Robot Inspection and Poster
Session test. They reached the highest number of points of all 19 teams
in this test.
3.4.2011
Our Robots Cosero and Dynamaid won the @Home league!
In the final, our robots prepared a breakfast. Cosero opened a milk bottle, poured out the milk over the cereal bowl, and threw away the bottle into the trash bin. In the meanwhile, Dynamaid got the orange juice from the fridge. Cosero also could fetch a spoon, which had been placed in arbitrary position on a table by a jury member. Finally, the robot recognized a pointing gesture of a jury member showing the exit door. SmartBots (Ulm) came in second, b-it-bots (Sankt Augustin) reached third place.
2.4.2011
Our robot Cosero performed the Enhanded Who is
Who test, the Shopping Mall test, and the Demo Challenge. Im Shopping
Mall test, our robot learned the map of a previously unknown
environment, was introduced to objects and delivered two of the three
requested objects. In the Demo Challenge, Cosero recognized objects,
interpreted pointing gestures and tidied up the apartment by moving
objects into shelves.
1.4.2011
Cosero got the most
points in the Who is Who test by finding four persons. It got the
second most points in the Open Challenge. Dynamaid was the only robot
to score in the General Purpose Robot Test II by understanding an
underspecified command, asking for the missing parts, and working on
the task.
31.3.2011
We participate at RoboCup German Open in
Magdeburg in the @Home league with the service robots Cosero and
Dynamaid.
The robots had to show their skills in the Stage 1 tests Robot
Inspection and Poster Session, Follow Me und Go Get It!.
24.6.2010
In the @Home League final, Dynamaid recognized an object showing
gesture and the object shown to her. She also went to the fridge,
opened the fridge door, and grasped a can of beer. Afterwards, she
closed the fridge again and delivered the drink. Finally, our robot
went to a shelf to get some snack. [Video]
In the final ranking, the Japanese team Er@ser was still ahead of
NimbRo. NimbRo won the second place
in the @Home League.
23.6.2010
Dynamaid did very well in the Shopping Mall test. This test was
conducted in a real toy store. Our robot learned a map of the
environment, remembered the location of multiple objects and then went
to collect a requested object. [Video]
In the Enhanced-Who-is-Who test, our robot learned the appearance and
the names of three persons. One of these persons was recognized later
when ordering drinks. In the Demo Challenge, Dynamaid successfully
recognized pointing gestures, recognized objects shown to her, and
picked-up objects from the floor that she deposited in a thresh bin.
After completing all Stage II, tests NimbRo had the second most points
in the overall ranking, next only to to the Japanese team Er@sers.
22.6.2010
Dynamaid performed very well in the Open Challenge. She recognized
gestures, got something to eat and to drink for a guest and picked-up
an object from the floor. After Stage I, NimbRo is second in the
overall ranking, next only to team Er@sers from Japan. In the
General-Purpose-Service-Robot test, Dynamaid recognized complex
commands and asked for clarification in underspecified commands.
21.6.2010
Dynamaid did the FollowMe test perfectly. During the test, she
successfully recognized stop gestures. After the GoGetIt test, NimbRo
still has the highest total number of points.
20.6.2010
Our team participates at the RoboCup
2010 competition in Singapore in the @Home League with
the robot Dynamaid. In the first test, Dynamaid registered herself
successfully for the competition while the team poster was presented.
NimbRo reached the highest number of points for this test.
5.5.2010
Our robot Dynamaid participated in the ICRA Mobile Manipulation
Challenge.
It fetched an object requested by the user by means of a pointing
gesture.
The user could also request a drink by showing it to the robot.
Finally, Dynamaid collected an object from the table and moved it
towards a trash bin.
18.4.2010
Our robots Robotinho and Dynamaid participated at German Open 2010.
Robotinho performed the registration. Dynamaid detected and reconized
all pesons in the Who-is-Who test and interpreted pointing gestures
correctly in the Open Challenge. Robotinho and Dynamaid cooperated
in the Demo Challenge.
In the final, Robotinho greeted a guest and guided him to a table.
There, the guest could select something to eat using a pointing
gesture. Also, the guest ordered a drink by showing it to Dynamaid.
Overall, our team reached the second place in the @Home
league.
6.7.2009
Our robots Robotinho and Dynamaid participated for the first time at
the RoboCup@Home international competitions.
They reached the highest score in the Introduce test. Dynamaid
successfully performed the Follow-me and the Who-is-Who test. Both
robots reached the second highest score in the Open Challenge, where
Robotinho gave a home tour to a guest while Dynamaid delivered a
drink.
In Stage II, Dynamaid did the Walk&Talk test, the Supermarket test and the Party-bot test very well. Both robots were used in the Demo Challenge. After Stage II, our team had the second most points, almost on par with the leading team.
In the final, Dynamaid detected persons and delivered drinks to them.
Overall, our team reached the third place in the @Home competition.
We won also the innovation award
for "Innovative robot body design, Empathic behaviors, and Robot-Robot
Cooperation".
The Video is also available in WMV-Format.
NimbRo participated for the first time in the @Home league at RoboCup German Open 2009 during Hannover Fair.
In Stage I, we used Robotinho for the introduce task. It explained
itself and Dynamaid and interacted with a human in a natural way. For
the follow me task, we used Dynamaid. She was able to quickly follow an
unknown human though the arena, outside, and back into the arena. She
also could be controlled by voice commands. Dynamaid also did the fetch
and carry task very well. She took spoken orders from the human
user and delivered reliably the requested object.
In Stage II, Dynamaid did the walk and talk task perfectly. A human
showed her places in the apartment that she could visit afterwards as
requested by spoken commands. In the demo challenge, the users could
order different drinks that she fetched quickly and reliably from
different places in the apartment.
In the final, Robotinho gave a tour though the apartment while
Dynamaid fetched a drink for a guest [video].
Overall, the NimbRo@Home team reached the second place,
only a few points behind b-it-bots.
Address:
Prof. Dr. Sven
Behnke
Rheinische
Friedrich-Wilhelms-Universität Bonn
Institut für
Informatik VI
Friedrich-Ebert-Allee 144
53113 Bonn
Tel: +49
(0) 228 73-4422
Fax:
+49 (0) 228 73-4425
Email: behnke _at_ cs.uni-bonn.de
Universität Bonn, Institute for Computer Science, Departments: I, II, III, IV, V, VI; Team NimbRo; Robotics in Bonn
