A new framework for vision-enabled and robotically assisted minimally invasive surgery

This paper presents our on-going research at bringing the state-of-the-art in vision and robotics technologies to enhance the emerging minimally invasive surgery, in particular the laparoscopic surgical procedure. A framework that utilizes intelligent visual modeling, recognition, and servoing capabilities for assisting the surgeon in maneuvering the scope (camera) in laparoscopy is proposed. The proposed framework integrates top-down model guidance, bottom-up image analysis, and surgeon-in-the-loop monitoring for added patient safety. For the top-down directives, high-level models are used to represent the abdominal anatomy and to encode choreographed scope movement sequences based on the surgeon’s knowledge. For the bottom-up analysis, vision algorithms are designed for image analysis, modeling, and matching in a flexible, deformable environment (the abdominal cavity). For reconciling the top-down and bottom-up activities, robot servoing mechanisms are realized for executing choreographed scope movements with active vision guidance. The proposed choreographed scope maneuvering concept facilitates the surgeon’s control of his/her visual feedback in a handless manner, reduces the risk to the patient from inappropriate scope movements by an assistant, and allows the operation to be performed faster and with greater ease. In this paper, we describe the new framework and present some preliminary results on laparoscopic image analysis for segmentation and instrument localization, and on instrument tracking.