Abstract

The majority of noise-filtering algorithms available in the literature assume that the nature of the noise and its statistical parameters are known a priori. Whereas in most practical applications, we have no accurate information on the type of noise present in an image. This pre-processing phase, prior to noise filtering, the present work positions itself. The output of this system can then be used to influence or tweak any subsequent noise filtering. Automatic Noise Type Determination essentially means, research into methodologies that automatically analyze an image and then determines what is the type of the most predominant video noise in that image and returns an output such as an 'A, B or C' type of answer. In addition, an estimate of the intensity of the strongest noise is also suggested. Then, armed with this information, the most appropriate noise filter can then be applied to the image to clean it in the best way possible. What's more, using the estimated intensity the noise filter can be adjusted depending on how strongly the image is corrupted. One of the major concerns when identifying the type of noise dominant in an image is that some types of noise are content-dependant, and some are content-independent. In other words, the data of the image itself can in some cases influence the intensity of the noise. The easiest way to imagine this is to compare additive noise with multiplicative noise. In the case of additive noise, the resulting corrupted pixel is simply distorted by an offset,however in the case of multiplicative noise as the pixel's value approaches zero, the same noise signal will have less effect on that pixel whereas as the pixel's value increases towards 255, the same noise signal multiplied by 255 will be greatly increased. Therefore, with these difficulties in mind, a modular strategy was developed using several specialized "expert systems" designed to analyze an image and look for certain features and determine the intensity of one specific noise type and can also provide a certainty factor. So that once the complete system is assembled, a "best guess" with an estimate at the noise level present in the image is produced. The architecture is divided in two sections, one covering content-independent noise types such as Additive White Gaussian Noise (AWGN) and Random Impulse Noise (RIN). The second section covers content-dependant noise types such as Motion Blurring Noise (Blur) and JPEG Compression Artifacts (JPEG). For each of these types of noise a dedicated "expert system" was developed to determine the intensity of that specific type of noise. The first category, considering content-independent noises, a kind of "unity" measure was then developed. In order to consider one "unit" of AWGN as being equal to one "unit" of RIN, an objective Image Quality Metric (IQM) was used to establish this equivalence. This IQM was developed at the University of Texas and is called "SSIM". Thus, using this SSIM an equivalence table was generated using several images, each applied with different intensities of either AWGN or RIN. In this way a certain intensity α of AWGN can be considered to be as corruptive as an intensity β of RIN. Then two algorithms were implemented each specialized in detecting one specific type of feature in an image. The first was developed by J.S. Lee and K. Hoppel titled "Noise Modeling and Estimation" used for determine the parameters of Additive White Gaussian Noise in an image. A software implementation of the algorithm was developed. Then a second expert system was developed, specialized in detecting Random Impulse Noise. The algorithm, developed by myself, consists of sweeping a horizontal edged detector over the image then a vertical edge detector is applied. Then a logical 'OR' is executed using the horizontal and vertical edge maps as inputs. What will appear on this "OrEdgeMap" are odd looking artifacts or donut-shaped rings encircling the pixel location where impulse noise was present in the original image. Then a series of masks are swept over the OrEdgeMap to pinpoint the locations of these donut rings. All of these locations are tabulated into a FlagMap. A final thresholding stage must be done to this FlagMap. It was empirically found that pixels that were flagged 5 times or more are almost certain to be corrupted by impulse noise. Then, using these two algorithms to determine the intensity of first AWGN and then the intensity of RIN in an image, both intensities are then compared to the lookup table, developed offline in the first part. The noise with the highest intensity on the unit-scale is considered, in a first place, to be dominant for content-independent noise. Then a similar process was developed for content-dependant noise types. However for content-dependant noise, the noises couldn't directly be compared to each other by using an equivalence table. In the case of blurring noise, an image is more susceptible when it contains many edges and has complex patterns, whereas a smooth image of a sky will be less susceptible to blurring noise. Conversely for JPEG noise, an image with complex patterns will mask any JPEG artifacts and these will become less visible, however an image with little or no edges will not mask or hide any JPEG artifacts therefore JPEG block artifacts will be much more visible in an image with a large surface of a smooth gradient such as a sky. Therefore using the same Image Quality Metric, the SSIM, as in the content-independent phase to create a "noise equivalence" table however for content-dependent noise such as blurring, five noise intensity equivalence tables are created. The differentiating factor between these tables is the Edge-count of the test image after an edge detector has been passed over the image. So table 1 is used when it is considered to be few edges in an image, and table 5 is used when there are many edges in the test image. As a result a preprocessing phase must be executed on the image to determine the edge count in order to use the appropriate table 1 through 5. A third "expert system", dedicated to determining the intensity of blurring noise is applied to the image. This algorithm I've also developed uses an edge detector. By rotating the image slightly prior to applying the edge detector we obtain an angular edge graph, from this graph we must extract the position with the largest difference in edge count between an angle and its perpendicular angle. That will be considered to the shifting angle. Afterwards, to determine the pixel displacement of the blurring noise, a Richardson-Lucy deconvolution is used. With this algorithm determines the intensity of the motion blurring noise in an image. The fourth specialized modular "expert" developed for the system was a way to detect JPEG noise. The algorithm consisted of sweeping an edge detector over the image and then adding up all the edge information contained at a 'block boundary'. Then doing the same at a pixel row that is just one row above the block boundary or one row below the boundary. Then subsequently repeating this for several threshold values of the edge detector. Then the cumulative statistical data at the block boundary is compared to the cumulative data at the pixel row immediately above or below the block pixel. Finally, the noise intensity equivalent tables of the content-independent noise from the first part is compared to the table of the content-dependant noise that was used respective to the edge count in the second part, and the final winner is then determined.

本論文は,Automatic Image Noise Type Determination Based on Directional Edge Information(方向性エッジ情報を用いた画像ノイズの自動分類)と題し,対象画像から抽出した方向性エッジ情報を用いて,その画像に含まれる様々なノイズの種類を同定するとともに,どの種類のノイズが画質を低下させる主要因と成っているかを定量的に決定するシステムの構築に関する研究成果を纏めたもので,全文8章よりなり英文で書かれている.

第1章は,序論であり,本研究の背景について議論するとともに,本論文の構成について述べている.

第2章は,Additive White Gaussian Noise Detection Algorithmと題し,画像に含まれるwhite Gaussian noiseを定量的に評価する手法について述べている.小領域の画素値の標準偏差値を求め,その画像全体にわたる分布からGaussian noiseの大きさを推定する既存のアルゴリズムを選定し,多くの画像を使った実験によりその推定精度の評価を行い,本研究に十分利用可能であることを示している.さらにGaussian noiseと同時にimpulse noise,あるいはブレに起因するblur noiseが重畳している場合の精度も評価し,前者の影響は少ないが,後者は大きく精度を損なうことを見いだしている.

第3章は,Random-Valued Impulse Noise Detection Algorithmと題し,impulse noiseを画像中より検出しその位置を同定するアルゴリズムを新たに提案している.4方向のエッジフィルタにより検出したエッジ情報に論理処理を施し,これによりimpulse noiseを同定しており,精度が高く且つハードウェア実装に適している.多くの画像を用いてその検出精度を評価し,本研究の目的には十分であることを示すとともに,Gaussian noiseが重畳した場合の精度も評価し,その大きさが極端に大きくない限り精度は保たれることを見いだしている.

第4章は,Performance of Content-Independent Noise Type Determinationと題し,white Gaussian noiseとimpulse noiseの両方が同時に存在するとき,どちらが画質劣化に支配的であるかを決定するシステムの開発について述べている.画質の表現として,従来のPSNRに代わり,人の感性に近い評価指標としてテキサス大より提案されたSSIMを採用し,多くの画像に意図的にノイズを加えた後,2-3章で開発した手法によりノイズを検出するとともに,ノイズの大きさとSSIMとの間の定量的な関係を実験的に導き出した.これを用いて,white Gaussian noiseとimpulse noiseの両方が重畳して存在する画像に対して実験を行い,画質が極端に損なわれていない画像では,支配的なノイズの同定が可能であることを示した.これは重要な成果である.

第5章は,Blurring Noise Detection Algorithmと題し,一方向へのカメラブレによるblur noiseについて,方向性エッジ情報を用いてその方向と大きさを定量的に検出する新たなアルゴリズムを提案するとともに,実験によりその有用性を示している.

第6章は,JPEG Noise Detection Algorithmと題し,JPEG画像に存在するブロックノイズを,方向性エッジ情報を用いて検出し定量化する新たなアルゴリズムを提案するとともに,その有用性を実験により示している.

第7章は, Performance of Content-Dependent Noise Type Determinationと題し,blur noiseとJPEG noiseの両方が同時に存在するとき,どちらが画質劣化に支配的であるかを決定するシステムについて述べている.第4章と同様SSIMをノイズの指標とし,検出されたノイズの大きさとの定量的な関係を実験的に導き出すとともに,多くの画像を用いて,blur noiseとJPEG noiseのどちらが支配的要因であるかを決める実験を行い,Content-Dependent Noiseに関しても,支配的なノイズの同定が可能であることを示した.

第8章は結論である.

以上要するに本論文は,画像に含まれる最も支配的なノイズの種類とその大きさを推定することが,その後のノイズ除去処理において重要であることに着目し,画像のエッジ情報を用いて様々なノイズの種類を同定する新たなアルゴリズムを開発するとともに,人間の感性に近い尺度でその大きさを推定する手法を提案し,実際に複数のノイズが重畳している画像において支配的なノイズ要因が特定できることを示したもので,情報学の基盤に寄与するところが少なくない.

したがって,博士(科学)の学位を授与できると認める.