The energy efficiency of digital circuits for mobile multimedia applications becomes increasingly important as larger numbers of transistors are integrated on a single chip. Multimedia technologies have enhanced the lives of mobile consumers by increasing productivity, enhancing the social networking experience, and delivering improved visual and audio quality for communication links and entertainment.

This work deals with an approach of how video codec processing can be integrated into energy efficient systems for smart society. In order to satisfy the high-performance and low-power requirements for advanced embedded systems with greater flexibility, it is necessary to develop parallel processing on chips by taking advantage of characteristics on video coding processing. This thesis explores high performance and low power technologies on video coding from both hardware architecture and video coding algorithm point of view.

The thesis is organized in the following manner. We start with a discussion on video codec design on semiconductor devices and systems. In particular, we focus on low-power design to show the tradeoffs between high performance and low-power with respect to circuit performance, area, and flexibility to the multi format video.

Chapter 2 describes power-efficient video codec based on parallelism in the design. A two-domain (stream-rate and pixel-rate) processing approach raises the performance of both stream and image processing units for a given operating frequency. In the image-processing unit, a sophisticated dual macroblock-level pipeline processing with a shift-register-based ring bus is introduced. This circuit is simple yet provides high throughput and a reasonable latency for video coding. Stream processor and media processor architecture is also described for flexibility of video signal processing. It discusses the results of implementing circuits from the viewpoints of performance and power consumption.

Chapter 3 focuses on the problem of limited memory bandwidth. An over HD-video solution is still unable to target the today's handset market due to its high memory bandwidth and power consumption. Application specific techniques are proposed to improve the performance and power efficiency of DRAM by analyzing access patterns of video coding processing. With two parallel pipelines for macroblock processing and tile-based address translation circuits; the codec processor chip was implemented in 65nm CMOS. Power evaluation with dynamic frequency selection and changing operating voltage was conducted using 45nm CMOS technology. Proposal architecture even has extensibility for emerging video standards and outperforms homogeneous media processors in terms of performance throughput and power efficiency.

As a result, the first Full-HD SoC available for handsets: 166 MHz mobile application processor is implemented in 65nm to support multistandard video codec at Full-HD resolution. 342 mW was achieved in real-time playback of a Full-HD H.264 High Profile stream from a 64 bit width low-power DDR-SDRAM at an operating frequency of 166 MHz at 1.2 V. This research created a positive economic impact for mobile applications that handset with Full-HD video recording and playback capability could be introduced to market in 2009 ~ 2010.

Chapter 4 looks into the important problem to interfere the parallelism in the video codec design. H.264/AVC is one of the most commonly used video compression formats for high definition video. It provide more flexibility for application than older standards such as MPEG-2 by adopting variable block-size motion compensation, spatial prediction from the edges of neighboring blocks for intra prediction, and context adaptive entropy coding. Foremost, inter and intra prediction scheme is developed separately. From the viewpoint of parallelism on hardware, it is the best solution that hardware can deal with all intra and inter picture prediction in a unified way. We propose a unification of intra and inter prediction based on template matching using repetitive pixel replenishment.

A new intra prediction method based on repetitive pixel replenishment (Intra RPR) is described. The concept of this proposal is unification of directional intra-prediction method and inter-prediction method. This intra-prediction scheme exploit motion vector search in a current picture, thus arithmetic logic unit for template matching processing can share with inter prediction. Intra RPR method reduced the circuits' area by 19% of intra and inters prediction unit, and also Intra RPR outperforms H.264/AVC intra prediction by an average of 5.0% with a maximum of 10.3% BD-Rate improvement.

The methods proposed in this thesis applied in current 65 nm and 45 nm semiconductor technology, and demonstrate power efficiency in Full-HD video processing. The two-domain architecture is always applicable to the prospective semiconductor technologies. Unification of intra/inter prediction is the applied for emerging video coding standard that is based on DCT and motion compensation-based architecture toward a concrete goal to year of 2020 with Super-Hi Vision, i.e. 7,680 x 4,320 pixels, 60 frame/s, progressive scanning, 10/12 bit depth.

本論文は,「Energy Efficient Design of Video Coding Based on Harmonization of Hardware and Algorithm (ハードウエアとアルゴリズム協調に基づくエネルギー効率の高い映像符号化に関する研究と応用)」と題し,英文で書かれており,5章よりなる.マルチメディアの中でもデータ量の膨大な映像は,かならず圧縮して扱われる.TVなどの映像家電はもとよりスマートフォンなどの携帯機器の普及を鑑みても,その圧縮においては,圧縮効率がより高いことは言うまでもなく,さらにその処理にかかる消費電力も,より効率の高いことが求められる.本論文では,この課題に対して,アルゴリズムとハードウエアを協調させた設計による,低消費電力VLSIコーデックの実現,さらには,ハードウエアリソースを考慮した新しいイントラ符号化方式について論じたものである.

第1章は,「Introduction(序論)」であり,研究の背景,映像システムの低消費電力化のトレンドについてまとめるとともに,本論文の構成について述べている.

第2章は,「Energy Efficient Video Codec-Design (エネルギー効率の高いビデオ符号器の設計)」と題し,並列処理に基づくビデオ符号器の設計について論じている.ストリームプロセッサーと複数の画像処理プロセッサーを有する方式を提案し,65nmと45nmのCMOSプロセスで実現し,検証している.ストリームと画像処理を併用し,マクロブロックを2つの並列パイプラインで処理し,さらに,動的なクロック停止を導入し,動作周波数,電圧を可変とすることで,Full HD H.264の映像信号を,動作周波数162MHzで,消費電力172mW(65nm CMOS), 95mW(45nm CMOS)を実現した.さらに,この符号器は,ファームウエアを変更するだけで,H.264だけでなく,MPEG-2, MPEG-4といった複数の標準符号化方式をサポートする柔軟性も有している.

第3章は,「Energy-Efficient Memory Management for HD-Video Processing (HDビデオ処理のためのエネルギー効率の高いメモリ管理)」と題する.高精細なFull HDでは,標準ビデオ(SD)の約6倍の帯域を有しており,携帯電話での利用にはさらなる低消費電力性が求められる.処理に必要な外部メモリへのアクセスの効率改善を図る方策として,Tile-Linear Address Translation(TLAT)という,小分割タイルを線形の物理アドレスにマッピングする手法を提案し,外部SDRAMの44%の消費電力削減を実現した.本方式をもちいて,携帯電話用のFull HDのビデオ符号器のSoCを初めて実現し,166 MHzで稼働するモバイルプロセッサーを用い,H.264を含む複数の標準方式に対応するFull HDの符号器が342mWで稼働することを実証した.

第4章は,「Unification of Intra and Inter Frame Prediction (イントラとインターフレーム予測の統一)」と題し,ハードウエアを考慮したアルゴリズムの設計について論じている.これまでの符号器のイントラ予測とインター予測は,全く異なる処理が用いられ,異なるハードウエアが用意されてきた.前者は,画面内予測,後者は,動き補償予測であり,その計算量とリソースは,後者は前者に比べて大きい.本論文では,イントラ符号化においても,インター符号化の方式を画面内適用することで,ハードウエアの統一化を図るという新しい提案を行った.アルゴリズムにおいても,画面内適用を行うために,非局在な画素補充を行う.これにより,圧縮においても,これまでのイントラ予測に対して,0.5%~5%程度の改善が行えることを示した.

第5章は,「Conclusions(結論)」であり,本論文での成果をまとめるとともに,今後の課題について言及している.

以上これを要するに,本論文では,映像の圧縮,消費電力の両者において,効率の高い圧縮をVLSIとして実現する実践的な取り組みを行い,アルゴリズムとハードウエア双方の観点から,低消費電力プロセッサ,外部メモリへのアクセスの効率化,さらには,イントラ・インター予測のハードウエアの統一による効率化について論じたものであり,その電子情報学上貢献するところが少なくない.

よって本論文は博士(情報理工学)の学位請求論文として合格と認められる.