|
|
 |
| 1.10.2021 - 30.9.2025 Förderkennzeichen: 01IS21080 |
The objective of this project is to learn human-like grasping with a
highly sensorized, compliant anthropomorphic hand from cluttered
containers. Grasping policies will be based not only on feedback from
RGB-D sensors, but also on force-torque measurements in the wrist,
force measurements in the fingers, and tactile measurements on the hand
surface.
The learned grasping strategies will be interactive, and thus include
non-prehensile manipulation actions, like pushing, rearranging, and
singulating objects, and multi-step plans like in-hand object
reorientation during grasping and regrasping for caging.
In order to achieve data-efficient learning from limited experience, we
will develop a structured approach for a scene parsing, prediction and
tracking that is based on visual and haptic measurements. Interactive
grasping strategies will be learned using deep reinforcement learning
methods in a physics-based simulation and with the real hand-arm
system.
Malte Mosbach and Sven Behnke:
Grasp Anything: Combining
Teacher-Augmented Policy Gradient
Learning with Instance Segmentation to Grasp Arbitrary Objects
In Proceedings of IEEE International Conference on Robotics and
Automation
(ICRA), Yokohama, Japan, May 2024.
[Video] [Project Page]
Arul Selvam Periyasamy and Sven Behnke:
MOTPose: Multi-object 6D
Pose Estimation for Dynamic Video Sequences using Attention-based
Temporal Fusion
In Proceedings of IEEE International Conference on Robotics and
Automation
(ICRA), Yokohama, Japan, May 2024.
[Video] [Project Page]
Efficient Multi-Object Pose
Estimation using Multi-Resolution Deformable Attention and Query
Aggregation
In Proceedings of IEEE International Conference on Robotic Computing
(IRC),
Laguna Hills, USA,
December 2023.
Arul Selvam Periyasamy, Arash Amini, Vladimir Tsaturyan, and Sven
Behnke:
YOLOPose V2: Understanding and
Improving Transformer-based 6D Pose Estimation
Robotics and Autonomous Systems, 168:104490, Elsevier, 2023.
Angel Villar-Corrales, Ismail Wahdan, and Sven Behnke:
Object-centric Video
Prediction via Decoupling of Object Dynamics and Interactions
In Proceedings of IEEE International Conference on Image Processing
(ICIP), pp. 570-574,
Kuala Lumpur, Malaysia, October 2023.
[Video] [Project Page]
Malte Mosbach and Sven Behnke:
Learning Generalizable Tool
Use with Non-rigid Grasp-pose Registration
In Proceedings of IEEE 19th International Conference on Automation
Science
and Engineering (CASE), Auckland, New Zealand, August 2023.
Best Conference Paper Award.
[Project
Page]
Dmytro Pavlichenko and Sven Behnke:
Deep Reinforcement Learning
of Dexterous Pre-grasp Manipulation for Human-like Functional
Categorical Grasping
In Proceedings of IEEE 19th International Conference on Automation
Science and
Engineering (CASE), Auckland, New Zealand, August 2023.
[Video]
Arul Selvam Periyasamy and Sven Behnke:
Towards 3D Scene Understanding
Using Differentiable Rendering
SN Computer Science 4(3):245, March 2023.
Arul Selvam Periyasamy, Luis Denninger, and Sven Behnke:
Learning Implicit
Probability Distribution Functions for Symmetric Orientation Estimation
from RGB Images Without Pose Labels
In Proceedings of 6th IEEE International Conference on Robotic
Computing (IRC),
Naples, Italy, December 2022.
Malte Mosbach and Sven Behnke:
Efficient
Representations of Object Geometry for Reinforcement Learning of
Interactive Grasping Policies
In Proceedings of 6th IEEE International Conference on Robotic
Computing (IRC),
Naples, Italy, December 2022.
[Project
Page]
Malte Mosbach, Kara Moraw, and Sven Behnke:
Accelerating Interactive
Human-like Manipulation Learning with GPU-based Simulation and
High-quality Demonstrations
In Proceedings of IEEE-RAS 21st International Conference on Humanoid
Robots (Humanoids), Ginowan, Okinawa, Japan, November 2022.
[Project
Page]
Angel Villar-Corrales, Ani Karapetyan, Andreas Boltres, and
Sven Behnke:
MSPred: Video Prediction
at Multiple
Spatio-Temporal Scales with Hierarchical
Recurrent Networks
In Proceedings of 33rd British Machine Vision Conference (BMVC),
London, UK, November 2022.
[Project Page] [Supplementary
Material]
Benedikt T. Imbusch, Max Schwarz, and Sven Behnke:
Synthetic-to-Real Domain
Adaptation using Contrastive Unpaired Translation
In Proceedings of 18th IEEE International Conference on Automation
Science and Engineering (CASE), Mexico City, Mexico, August 2022.
[Video]
Arash Amini, Arul Selvam Periyasamy, and Sven Behnke:
YOLOPose: Transformer-based Multi-Object 6D Pose Estimation using
Keypoint Regression
In Proceedings of 17th International Conference on Intelligent
Autonomous
Systems (IAS), Zagreb, Croatia, June 2022.
Best Paper Award.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Iterative 3D Deformable
Registration from Single-View RGB Images Using Differentiable Rendering
In Proceedings of 17th International Conference on Computer Vision
Theory and Applications (VISAPP), February 2022.
Finalist for Student Paper Award.
Diego Rodriguez, Florian Huber, and Sven Behnke:
Category-level
Part-based 3D Object Non-rigid Registration
In Proceedings of 17th International Conference on Computer Vision
Theory and Applications (VISAPP), February 2022.
Best Poster Award.
[Poster]
2018-2021 Amazon supported our research on perception of bin picking scenes with Amazon Research Awards.
Amazon continues to support our research on interpreting bin picking
scenes with a 2019
Amazon Research Award.
In the new project, we go beyond the passive observation of scenes by
considering scene changes caused by the actions of a
robotic manipulator arm. We will learn predictive models of the
effects of manipulation actions and use these to maintain the
structured scene model while the robot is manipulating.
Supported by an 2018 Amazon Research Award, we continued research in the area of interpreting cluttered bin picking scenes.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
SynPick: A
Dataset for Dynamic Bin Picking Scene Understanding
In Proceedings of 17th IEEE International Conference on Automation
Science
and Engineering (CASE), Lyon, France, August 2021.
[Projekt page with
data set, code, and videos}
Moritz Zappel, Simon Bultmann, and Sven Behnke:
6D Object Pose
Estimation using
Keypoints and Part Affinity Fields
In Proceedings of 24th RoboCup International Symposium, June 2021.
Finalist for Best Paper Award
[Video]
Diego Rodriguez, Florian Huber, and Sven Behnke:
Category-Level
3D Non-Rigid Registration from Single-View RGB Images
In Proceedings of IEEE/RSJ International Conference on Intelligent
Robots
and Systems (IROS), Las Vegas, USA, October 2020.
[Video]
Sven Behnke and Max Schwarz:
Generalizing
Scene Parsing for Cluttered Bin Picking
Presentation at Amazon Research Award Symposium (ARA), Boston, MA, USA,
October 2019.
Max Schwarz and Sven Behnke:
Stillleben:
Realistic Scene Synthesis for Deep Learning in Robotics
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Paris, France, May 2020.
[Source code] [Presentation video]
Catherine Capellen, Max Schwarz, and Sven Behnke:
ConvPoseCNN:
Dense Convolutional 6D Object Pose Estimation
In Proceedings of 15th International Conference on Computer Vision
Theory and Applications (VISAPP), pp. 162-172, Valletta, Malta,
February 2020.
Umashankar Deekshith, Nishit Gajjar, Max Schwarz, and Sven Behnke:
Visual
Descriptor Learning from Monocular Video
In Proceedings of 15th International Conference on Computer
Vision Theory and Applications (VISAPP), pp. 444-451, Valletta, Malta,
February 2020.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Refining 6D Object Pose
Predictions using Abstract Render-and-Compare
In Proceedings of IEEE-RAS International Conference on Humanoid Robots
(Humanoids), Toronto, Canada, October 2019.
[Video]
Starting from our work in the Amazon Picking and Robotics Challenges, we investigated the parsing of complex bin picking scenes with support of an Amazon Research Award.
Sven Behnke and Max Schwarz:
Learning
Semantic Perception for Cluttered Bin Picking
Presentation at Amazon Research Awards Fall Symposium (ARA), Boston,
MA, USA, October 2018.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Robust 6D Object Pose
Estimation in Cluttered Scenes using Semantic Segmentation and Pose
Regression Networks
In Proceedings of IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), Madrid, Spain, October 2018.
[Video]
Our team NimbRo Picking came in second at the Amazon Picking Challenge 2016 and the Amazon Robotics Challenge 2017.
Volker Krueger, Francesco Rovida, Bjarne Grossmann, Ronald Petrick,
Matthew Crosby, Arnaud Charzoule, German Martin
Garcia, Sven Behnke, Cesar Toscano, Germano Veiga:
Testing the Vertical and
Cyber-Physical Integration of Cognitive Robots in Manufacturing
Robotics and Computer-Integrated Manufacturing, vol. 57, pp. 213-229,
Elsevier, June 2019.
Max Schwarz, Christian Lenz, German Martin Garcia, Seongyong Koo,
Arul Selvam Periyasamy, Michael Schreiber, and Sven Behnke:
Fast Object Learning and Dual-arm
Coordination for Cluttered Stowing, Picking, and Packing
Accepted for IEEE International Conference on Robotics and Automation
(ICRA), Brisbane, Australia, May 2018.
Max Schwarz, Anton Milan, Arul Selvam Periyasamy, and Sven Behnke:
RGB-D Object Detection and
Semantic Segmentation for Autonomous Manipulation in Clutter
International Journal of Robotics Research (IJRR), Sage Publications, available online,
June 2017.
Max Schwarz, Anton Milan, Christian Lenz, Aura Munoz, Arul Selvam
Periyasamy, Michael Schreiber, Sebastian Schüller, and Sven Behnke:
NimbRo Picking: Versatile Part
Handling for Warehouse Automation
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Singapore, May 2017.
[Video]
 |
This project is funded by the FP7 ICT GA 610917. |
The goal of STAMINA is to develop a fleet of autonomous and mobile industrial robots for jointly solving three logistic and handling tasks: de-palletizing, bin-picking, and kitting. The main task of our group is the grasping of parts from transport containers. This includes the detection and pose estimation of parts as well as the grasp and arm motion planning for a robot manipulator.
Volker Krueger, Arnaud Charzoule, Bjarne Grossmann, Cesar Toscano,
Francesco Rovida, German Martin Garcia, Germano Veiga, Matthew Crosby,
Ronald Petrick, and Sven Behnke:
Testing the Vertical and
Cyber-Physical Integration of Cognitive Robots in Manufacturing
Accepted for Robotics and Computer-Integrated Manufacturing, Elsevier,
to appear 2019.
Dmytro Pavlichenko, Germán Martín García, Seongyong Koo, and Sven
Behnke:
KittingBot: A Mobile Manipulation
Robot for Collaborative Kitting in Automotive Logistics
Accepted for 15th International Conference on Intelligent Autonomous
Systems (IAS-15), Baden-Baden, Germany, June 2018.
Seongyong Koo, Grzegorz Ficht, Germán Martín García, Dmytro
Pavlichenko, Martin Raak, and Sven Behnke:
Robolink Feeder: Reconfigurable
Bin-Picking and Feeding with a Lightweight Cable-Driven Manipulator
In Proceedings of 13th IEEE International Conference on Automation
Science and Engineering (CASE), Xi'an, China, August 2017.
[Video]
Dirk Holz and Sven Behnke:
Fast Edge-Based Detection and
Localization of Transport Boxes and Pallets in RGB-D Images for Mobile
Robot Bin Picking
In Proceedings of 47th International Symposium on Robotics (ISR),
Munich, Germany, June 2016.
Dirk Holz, Alexandru-Eugen Ichim, Federico Tombari, Radu B. Rusu,
and Sven Behnke:
Registration with the Point Cloud
Library - A Modular Framework for Aligning in 3-D
IEEE Robotics and Automation Magazine, 22(4):110-124, December 2015.
Dirk Holz, Angeliki Topalidou-Kyniazopoulou, Joerg Stueckler, and
Sven Behnke:
Real-Time Object Detection,
Localization and Verification for Fast Robotic Depalletizing
In Proceedings of IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS), Hamburg, Germany, September
2015.
[Video]
Dirk Holz, Angeliki Topalidou-Kyniazopoulou, Francesco Rovida,Mikkel
Rath Pedersen, Volker Krüger, and Sven Behnke:
A Skill-Based System for Object
Perception and Manipulation for Automating Kitting Tasks
In Proceedings of 20th IEEE International Conference on
Emerging Technologies and Factory Automation (ETFA), Luxemburg,
September 2015.
Dirk Holz and Sven Behnke:
Approximate Surface Reconstruction
and Registration for RGB-D SLAM
In Proceedings of European
Conference on Mobile Robots (ECMR), Lincoln, UK, September
2015.
Dirk Holz, Matthias Nieuwenhuisen, David Droeschel, Jörg Stückler,
Alexander Berner, Jun Li, Reinhard Klein, and Sven Behnke:
Active
Recognition and Manipulation for Mobile Robot Bin Picking
In: Gearing up and accelerating cross-fertilization between academic
and industrial robotics research in Europe - Technology transfer
experiments from the ECHORD project, vo. 94 of Springer Tracts in
Advanced Robotics (STAR), pp. 133-153, 2014.
Alexander Berner, Jun Li, Dirk Holz, Jörg Stückler, Sven Behnke, and
Reinhard Klein:
Combining
Contour and Shape Primitives for Object Detection and Pose Estimation
of Prefabricated Parts
In Proceedings of IEEE International Conference on Image Processing
(ICIP), Melbourne, Australia, September 2013.
Matthias Nieuwenhuisen, David Droeschel, Dirk Holz, Joerg Stückler,
Alexander Berner, Jun Li, Reinhard Klein, and Sven Behnke:
Mobile Bin Picking with an
Anthropomorphic Service Robot
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Karlsruhe, May 2013.
Matthias Nieuwenhuisen, Jörg Stückler, Alexander
Berner, Reinhard Klein, and Sven Behnke:
Shape-Primitive
Based Object Recognition and Grasping
In Proceedings of 7th German Conference on Robotics (ROBOTIK),
Munich, May 2012.
Dirk Holz, Matthias Nieuwenhuisen, David Droeschel, Jörg Stückler,
Sven Behnke, Alexander Berner, and Reinhard Klein:
Active
Recognition and Manipulation of Objects Based on Shape Primitives
Poster at 5th International Conference on Cognitive Systems (CogSys)
Vienna, February 2012.
University of Bonn, Institute for Computer Science, Computer Science VI - Intelligent Systems and Robotics Impress Data Privacy Statement ; Robotics
![[Poster]](../papers/VISAPP_2022_Rodriguez_Poster.svg){kind=link}
