Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needles

Albert Wilcox, Justin Kerr, Brijen Thananjeyan, Jeffrey Ichnowski, Minho Hwang, Samuel Paradis, Danyal Fer, Ken Goldberg

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Scopus citations

Abstract

Robotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handover of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that uses a learned active sensing policy with a stereo camera to iteratively localize and align the needle into a visible and accessible pose for the other gripper. To compensate for robot positioning and needle perception errors, the algorithm then executes a high-precision grasping motion that uses multiple cameras. Physical experiments with the da Vinci Research Kit (dVRK) suggest a success rate of 96.7% on needles used in training, and 75 - 92.9% on needles unseen in training. On sequential handovers, HOUSTON successfully executes 32.4 handovers on average before failure. To our knowledge, this work is the first to study handover of unmodified surgical needles. See https: / /tinyurl. com/houston-surgery for additional materials including details about offline datasets and model architectures.

Original languageEnglish
Title of host publication2022 IEEE International Conference on Robotics and Automation, ICRA 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages9637-9643
Number of pages7
ISBN (Electronic)9781728196817
DOIs
StatePublished - 2022
Event39th IEEE International Conference on Robotics and Automation, ICRA 2022 - Philadelphia, United States
Duration: 23 May 202227 May 2022

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Conference

Conference39th IEEE International Conference on Robotics and Automation, ICRA 2022
Country/TerritoryUnited States
CityPhiladelphia
Period23/05/2227/05/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Fingerprint

Dive into the research topics of 'Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needles'. Together they form a unique fingerprint.

Cite this