The continuous progress in semiconductor VLSI technologies during the past several decades has provided the opportunity of realizing real-time intelligent image processing systems such as image recognition, object tracking, motion recognition, etc. However, the traditional approach of running image processing algorithms on general purpose processors is not practical for building efficient systems at rational costs with low power-consumption. Therefore, a number of VLSI chips having parallel processing architectures such as graphics processing units (GPUs) have been developed to enhance the performance. Although the processing time can be reduced greatly, such approaches are not still efficient enough due to the complex and expensive image processing algorithms which usually include a number of floating point operations. In order to resolve the problem of such a large gap between the algorithms and their VLSI implementation and to maximally utilize the power of semiconductor technologies, we try to develop algorithms which are compatible with the physical characteristics of VLSI circuits.

The robust nature of the human brain in visual information processing has been attracting a lot of researchers to discover better ways of image processing. Physiology research has revealed that the directional edge information in images is utilized as the most important clue in visual object recognition. Being inspired by such a biological principle, a series of direction-edge-based VLSI-implementation-adapted intelligent image processing algorithms as well as the corresponding VLSI circuits and systems have been proposed and developed in our laboratory.

This work succeeds the research in such bio-inspired algorithms, circuits, and systems. In order to minimize the latency caused by the image data transfer between the image sensor and the processing circuits, the most serious bottleneck in such systems, digital-pixel-sensor-embedded (DPS-embedded) processors were proposed and designed. The performance of such processor has been verified by building a real-time image recognition system with a very low latency. In addition, a directional-edge-based object tracking algorithm was also proposed and partially implemented in an object tracking system by building processing circuits on FPGAs. In the followings, the work is described in more detail.

Firstly, a DPS-embedded global feature extraction VLSI processor for real-time image recognition has been developed. By combining the block-readout architecture of DSP and parallel processing elements, the latency of local feature extraction has been markedly reduced. By adapting the rank-order filter algorithm to hardware implementation, global feature extraction is accomplished in only 11 cycles. A prototype chip was designed in a 0.18-μm five-metal CMOS technology. The measurement results show that the VLSI processor can extract features more than 400 times faster than software processing running on a 2-GHz general-purpose processor when operating at 60 MHz.

Then, a DPS-embedded early-visual-processing VLSI processor for real-time intelligent image processing has been developed. Compared with the first chip, the enhancement in the functionality of processing elements in the global image processing block further improves the programmability of the processor. As a result, such a chip can handle multiple algorithms efficiently. A prototype chip was designed in a 65-nm 12-metal CMOS technology. The simulation results show that this VLSI processor can achieve all expected functions.

In order to demonstrate the power of such chips, a real time image recognition system has been developed. The system is based on a VLSI-implementation friendly image recognition algorithm. By using the global directional-edge-feature extraction VLSI processor, the latency between the image capture and the final recognition as small as 906 μs has been demonstrated. The merit of the global feature extraction algorithm that it can focus on more significant features automatically has also been experimentally verified.

In the research on algorithms, a directional-edge-based object tracking algorithm was developed. By using directional-edge-based feature vectors, the system has been made robust against illumination variation. The on-line learning technique and the statistical multiple-candidate-location generation have further improved the performance, making the system robust against object size variation, partial occlusion, and object deformation. The performance was verified by experiments under varying disturbing conditions.

Finally, a simple real time object tracking system based on a restrained version of the prior algorithm has been implemented successfully. By experimental results, this system shows satisfying performance in simple tracking tasks by employing only eight candidate locations. Thanks to the fine-grained VLSI-implementation of the object tracking algorithm implemented in an FPGA, the total processing time for the tracking task has been reduced to about 0.1 ms when the system is running at a frequency of 60 MHz.

In this work, circuits and systems for brain-mimicking algorithms have been developed based on a very naive model of the brain. In the algorithms of image processing, directional edge information plays an essential role for perception of still images as well as moving images. In these systems, the vast amount of subconscious processing in the mind has been implemented by VLSI chips or FPGAs. In order to build "real-time responding human-like intelligent systems" with small hardware volume and low powers, such development of hardware-friendly algorithms and their VLSI implementation in fine-grain parallel architectures are most essential.

本論文は,VLSI Circuits and Systems for Directional-Edge-Based Intelligent Image Processing(和訳:方向性エッジ情報に基づく知的画像処理回路・システムに関する研究)と題し,人間のように柔軟な画像認識システム構築を目指し,対象物の画像より方向性エッジ情報を高速で抽出可能なデジタル方式CMOS特徴抽出イメージセンサーの開発,及びこれを用いた実時間認識システムの実現,またエッジ情報を用いたロバストな物体追跡ハードウェアアルゴリズムの提案とFPGAを用いたシステム構築に関する研究成果を纏めたもので,全文7章よりなり英文で書かれている.

第1章は,序論であり,本研究の背景について議論するとともに,本論文の構成について述べている.

第2章は,A Digital-Pixel-Sensor-Based Global Feature Extraction VLSI Processor for Real-Time Object Recognitionと題し,対象画像の相対的に重要な部分より,エッジ情報を優先的に抽出するCMOS特徴抽出イメージセンサーについて述べている.フォトダイオードで得た光強度をピクセル毎にデジタル値に変換してメモリに保持,フィルター演算に必要な画素データをブロック読み出し方式により一括して並列処理回路に転送し,行並列で高速に演算する.またフィルター演算の結果に対し,バイナリー探索でランクオーダー処理を施し,方向性エッジ情報を重要度の高い順に選択的に残す.0.18μm CMOS技術を用いて64×64ピクセルサイズのチップを設計・試作し,チップの計測により60MHzの動作で毎秒4000フレームの処理が可能なことを示した.これは2GHzの汎用プロセッサを用いたソフトウェア処理と比較して約400倍高速である.

第3章は,Design of Advanced Early-Visual-Processing VLSI Processor Using 65-nm Technologyと題し,前章で開発したアーキテクチャをさらに発展させ,静止画のみならず,方向性エッジ情報を用いた動画像認識アルゴリズムにも対応できるよう機能を拡張したCMOSスマートイメージセンサーについて述べている.ランクオーダー処理により抽出した方向性エッジ情報に対し,様々な画素並列演算処理機能を付与し,フレーム間の差分演算,さらにフレーム間差分値を時間軸方向に積分する機能等も加え,動画認識に必要な特徴量抽出機能を実現している.ここで開発したアーキテクチャは,65nm CMOS技術を用いて設計,試作チップの計測によりすべての動作を実証した.これは重要な成果である.

第4章は,A Real-Time Image Recognition System Using a Global Directional-Edge-Feature Extraction VLSI Processorと題し,第2章で開発したイメージセンサーチップを用いた実時間画像認識システムの構築について述べている.イメージセンサーチップによって方向性エッジ情報を抽出した後,FPGA上に構築したシステムによって,特徴ベクトルの生成ならびに学習処理・連想処理等の機能を実現している.ロバストなハンドジェスチャーの認識に加え,システムがより重要な画像情報に自律的に着目することを実験的に示した.これは興味ある結果である.

第5章は,Directional-Edge-Based Object Tracking Employing On-Line Learning and Regeneration of Multiple Candidate Locationsと題し,ハードウェア実装に特化した物体追跡アルゴリズムについて述べている.方向性エッジ情報を用いて追跡物体の特徴を表現し,これにより追跡を行うが,対象物が時々刻々形状を変えてもその変化を随時学習することにより,極めてロバストな追跡が可能であることをシミュレーションにより示している.

第6章は,FPGA Implementation of a Directional-Edge-Based Real-Time Object Tracking Systemと題し,第5章で開発したアルゴリズムをFPGAに実装し,実時間の物体追跡が可能であることを実験的に示している.

第7章は結論である.

以上要するに本論文は,人間のように柔軟な画像認識システム実現を目指し,対象画像より方向性エッジ情報を適応的に抽出し,必要な論理処理を施すデジタル方式CMOS特徴抽出イメージセンサーの開発,並びにVLSIハードウェア化に適合させた物体追跡アルゴリズムの提案を行い,それぞれ静止画認識システム,並びに物体追跡システムを実際に構築することによりその有効性を実証したもので,電子工学の発展に寄与するところが少なくない.

よって本論文は博士(工学)の学位請求論文として合格と認められる.