The automation technology is very promising in surgical robotic applications especially for the repeated tasks that need high accuracy because the autonomous robots can perform task with higher precision and speed than human operators. However, there are many disturbance and noise in the information that the robots can use, and this may lead to failures in automation. On the other hand, human operators are good at perceiving unstructured environments and immediately reacting to unexpected events. By combining strengths of autonomous robots and humans operators, this dissertation presents a human-robot collaborative robotic system for automated surgery. In the collaborative system, desired trajectory is abstracted and learned from preoperative demonstrations by surgeons, and the robot executes the learned trajectory autonomously. The planned trajectory is overlaid on the microscopic view, and the operator of the robot can approve or modify the trajectory by applying force to the input device. This collaboration method improves the safety in the surgical automation by fully exploiting the robot's advantages and surgeons experience.

This kind of human-robot collaborative system requires the system to be integrated with several functional components. Therefore, four research keywords are defined in this work such as perception, manipulation, automation, and intervention. The perception is an issue to estimate the state of the robotic forceps. The manipulation is a positioning function of the robotic forceps based on visual information. The automation is learning and reproduction methods of surgical tasks. Finally, the intervention is an interface between human operators and robotic systems to guide the automated robotic motions by surgeons.

The first research topic is perception problem. The robust forceps tracking method under a microscope is proposed for this issue. Forceps tracking is an indispensable function of high level surgical assistance such as visual servoing and motion analysis. Therefore, a robust and efficient tracking algorithm capable of estimating the forceps tip position in an image space is required. Our approach to cope with this problem is to fuse visual tracking data with kinematic information. In visual tracking, the full state parameters of forceps are estimated using the projective contour models generated from the 3-D CAD model of the forceps. The likelihood of the contour model is measured using the distance transformation to enable fast calculation, and the particle filter estimates the full state of the forceps. For more robust tracking, the visual tracking result is combined with kinematic data that is obtained from forward kinematics and hand-eye transformation. The fusion of visual and kinematic tracking data is performed using an adaptive Kalman filter, and the fused tracking enables the reinitialization of visual tracking parameters when a tracking failure occurs. Experimental results indicate that the proposed method is accurate and robust to image noise, and forceps tracking was successfully carried out even when the forceps was out of view.

For the manipulation issue, autonomous positioning system using visual servoing is proposed. Positioning of surgical robotic forceps based on visual feedback is the first step toward autonomous surgical systems that can make full use of the robotic system. To achieve high positioning accuracy, visual servoing technique is utilized in many applications. However, in visual servoing, we cannot control the trajectory of motion in Cartesian space. Undesired behavior of forceps may make damage to tissue or blood vessels. To cope with the problem, we propose a vision-guided positioning system by separating z-axis motion from x-y plane motion. The interaction matrix is decomposed into positioning (x-y plane) and approaching (z-axis) components. Approaching velocity is computed on the basis of difference of disparity that is a dominant factor for approaching, and then positioning velocity is yielded by reformulated interaction matrix. The experimental results showed that the proposed servoing method can control the trajectory in Cartesian space by adjusting approaching coefficient. Using the proposed method, a simple point-to-point task such as needle gripping is successfully performed.

Thereafter, the third research topic regarding automation is addressed. The automation technique is one of the solutions to improve the precision and speed of a variety of the tasks. Therefore, it is widely used for industrial robots in the past decades, and it can be also used for surgical robots to a certain extent. In surgical automation, two functions should be implemented; learning and reproduction. For learning method, we use Gaussian Mixture Model to learn the necessary trajectory for a certain surgical task. After training the task, Gaussian Regression provides an optimal trajectory from several demonstrations. Once the optimal trajectory obtained, the reproduction is carried out with the forceps tracking and visual tracking by tracing the whole desired trajectory. By integrating the functions of forceps tracking, visual servoing, and task learning, the automated task can be realized with the advantages of the reduced completion time and low motion deviation. However, the constraint for the automation is the structured environment, and it is difficult to apply the full automation to surgical tasks which contains many unpredictable conditions.

The fourth research topic is intervention for human-robot collaborative system. In the collaboration method, a commercial haptic device is utilized. The haptic device has two basic interfaces with the operator; force feedback and force input. In our method, the robot provides the operator with force feedback to indicate the planned motion from demonstrations. The surgeon inputs force to the haptic device so that he/she can supervise the automated robotic motion. The operator can approve, accelerate, modify, and terminate the automated motion by judging the current situation from the microscopic view. The prototype of the human-robot collaborative system was developed, and the feasibility of the developed system was demonstrated using simulated surgical tasks.

By integrating the four research topics aforementioned, I proposed the new concept of human-robot collaboration, and developed a platform of human-robot collaborative surgical system toward automated operations. In the experiments, the several surgical tasks were conducted to validate the feasibility of the proposed system. The experimental results demonstrated that unpredictable and undesirable situations can be dealt by the surgeon's guidance. The human-robot collaboration can provide a new way toward the clinical use of autonomous manipulation since its safety is ensured by the surgeon while benefiting from precision and speed of autonomous systems.

従来のマスター・スレーブ型手術支援システムは,医師に高精度,器用さを提供することで高度な手術を可能としてきた.現在は,手術時間の短縮や医師の負担の低減など,手術支援システムのさらなる高機能化のために,手技の自動化に関して多く研究されている.本研究では,自動化されたシステムで起こりうる安全性の問題を人間の監視機能を加えることで解決をはかり,人間とロボットが協調する手術支援システムを提案した.

特に本論文では,人間・ロボット協調手術システムのために必要となる4つの要素を定義した.その要素とは、認識,制御,自動化,そして介入である.そして、それぞれの課題を明らかにし,その課題の解決手法を提案する.

第1章では,既存のマスター・スレーブ型手術支援システムの特徴とそれに伴う限界点を述べ,それを乗り越えるために人間とロボットの協調システムが必要であることを述べる.

第2章では研究目的を述べ,人間・ロボット協調手術システムを開発するためのアプローチを提案する.また,4つの必要な要素を定義し,その実装手法を挙げている.

第3章では本研究でプラットフォームとして用いる微細手術用のマスター・スレーブ型手術支援システムを開発し,吻合実験を通してシステムの検証を行う.また、マスター・スレーブ型システムの限界点である、精度、速度、安全性の問題に関して考察を行う.

第4章では認識の問題に対し,ロボット鉗子の3次元モデルを画面上に投影した輪郭をデータベース化することで,画像上で鉗子のトラッキングを行う手法を提案する.また,画像情報のみを使用する場合には,鉗子が画面の外側に出た時,トラッキングできない問題がある.そこで,画像でのトラッキング情報とロボットの運動学情報を融合するロバストな鉗子トラッキング手法を提案する.

第5章では、検出された鉗子の情報に基づいたビジュアルサーボに関して述べる.既存のビジュアルサーボでは3次元の軌道の計画が不可能であり,これは手術ロボットでは致命的な弱点になる.この問題を解決するために,制御の平面位置決め,奥行き位置決めを分離することで,3次元軌道の計画の可能とした.

第6章では混合ガウシアンモデルを用いて手技を学習し、ビジュアルサーボを用いて学習した手技の自動化を行う.手技を自動化することで人間よりも速くタスクを完遂することができ,動作の精度も向上することを確認した.しかしながら,両手の動作が必要となる糸を巻き付けるタスクに対しては,糸を持つ角度が変化した場合にタスクを完遂できないことを確認し,手技の自動化システムにおける問題点を明らかにした.

第7章では触覚デバイスを用いてロボット軌道に沿った力覚フィードバックを付加し,医師がロボット鉗子の軌道を変更することで,自動化システムへ介入する方法を提案する.提案した手法を用いて,糸を巻きつける動作で自動化システムでは対応できなかった環境変化に対して、解決の可能性を示した.また、医師の意見に基づいた定性的な評価を行い,介入方法に関してはやや満足度が低く,人間・ロボットインタラクション(HRI)の今後の課題を明らかにした.

第8章では,本研究で提案した手法の限界となる手術環境の認識問題や高次元タスクモデリングに関して述べている.そして,本研究の展望について考察している.第9章では,本研究の結論が述べられている.

以上をまとめると,本研究は既存のマスター・スレーブ型手術支援システムにおいて,精度,速度,安全性の問題を解決するために,手技の自動化に向けた人間・ロボット協調手術システムを提案し,そのシステムを実装した.さらに,手技を模擬したタスクに対し検証を行うことで,協調手術支援システムのあり方を提案・検証した.

よって本論文は博士(工学)の学位請求論文として合格と認められる.