In this thesis, we study the instanton counting from the string and gauge theory perspective. The basic tools we use in this thesis are the Nekrasov partition function of instanton counting and the topological string theory on the toric Calabi-Yau. The Nekrasov theory of instanton counting describes the low energy dynamics of the N = 2 gauge theory. The topological string theory on the toric Calabi-Yau capture the effective dynamics of the gauge theory as a local model of more complicated compactification of Type II superstrings. One of the main topics of this thesis is the relation of these gauge and string theories.

In the first part of this thesis, we have applied refined topological vertex for SU (N) geometries and we compare the Nekrasov's theory of instanton counting. Then we show that the refined partition functions coincides with the the Nekrasov's partition functions in the non self-dual graviphoton background. The result has several meanings. First we show the validity of the attempt to refine the topological string theory by establishing the geometric engineering in the presence of the non self-dual background. This is a first step toward the formulation of the refined topological string theory in the general framework. It is also interesting that the Nekrasov partition function has the combinatorial interpretation using the Schur and Macdonald functions.

As we discussed in this paper, the refinement of the framing factor is obtained by computing the partition function for the SU (N) geometry. Since we cannot define the factor from the first principle at the present time, we derive it by comparing the refined topological vertex for SU (N) geometry and the Nekrasov's partition function. If the refined topological string theory is formulated, it would reproduce the factor we proposed in this thesis.

We extend the above correspondence to the gauge theories with matter hypermultiplets by studying the refined vertex on the strip geometry. We show that the refined vertex on the strip geometry is also put in the simple equation. This is because that the Schur functions of the partition functions can be summed up as in the case of the topological vertex on strips. Thus we find that the geometric engineering is extended to the gauge theory with the non-self dual graviphoton background. We can interpret this result as the reasonableness of attempt to refine the topological strings.

In the next part, we prove the flop invariance of the reined topological vertex. Since the flop invariance is the property which we expect to the refinement, it is very important to check the non-trivial property. We confirm the flop invariance by evaluating the flop amplitudes using the free fermions. Using the free fermions, we can represent these amplitudes as the Veneziano amplitute. This perspective is the background of the geometric engineering in the previous section. We also show the fundamental formula which is important to caluculate the Schur functions appearing in the refined vertex.

The results we have shown above are the studies of the refine vertex formalism. Then we study the mathematical applications. We compare the open string partition function of the conifold with the sl (N) homological link invariants of the Hopf link. We can see the agreement between them for some small representations. Therefore we propose the superpolynomial as the conifold partition function. We expect that our proposal provides some insights into the study of the homological link invariants.

In the last part of this article, we apply the instanton counting to N = 1 gauge theories. Since the instanton counting in the N = 2 gauge theory is well studied, it is very important to apply this formalism to N = 1 theories. We compute the effective superpotentials from the microscopic superpotential formalism of N = 1 gauge theories which is obtained from the instanton counting technique. The essential point of this approach is same as the strong coupling approach which deals N = 1 theories as a deformation of the strongly coupled N = 2 theories. However the microscopic superpotential for the factorized curves give the useful way to study the vacua and phases of the theory systematically. Then we study the generalized glueball operators using this approach and the connection between the glueball operators, the Konishi anomaly equation, and the Witham hierarchy. We show that the Witham deformation which plays a key role in the Seiberg-Witten theory is also important in N = 1 theories.

瀧雅人氏の学位申請論文は、8つの章と4つの補遺からなり、第1章および第2章は目的と背景の説明および基本事項の解説に当てられ、第3章以降が本研究による成果の叙述である。第3章では、refined vertexを用いたtopological stringによる分配関数の計算手法と、Nekrasov公式を再現するための正しいframing factorの提唱と検証が述べられている。第4章には、この手法の下でのFlop invarianceの証明が、また第5章では、その応用としてのリンク不変量との関係を議論している。第6章では対応するYang-Mills理論におけるsuperpotentialについての議論、第7章でglueball演算子への応用が述べられている。第8章は全体の結果がまとめられている。

本論文が目的とする対象は、Nekrasovによって提唱されたN=2超対称Yang-Mills理論のインスタントンを数え上げる分配関数である。この公式はgraviphoton背景場に対応する二つのパラメータを含んでいる。これに対して、二つのパラメータが特殊な関係を満たすとき(つまりgraviphoton背景場が自己双対のケース)には、トーリック-カラビ-ヤウ空間上のtopological string理論によってこれを計算することが出来て、Nekrasovの公式が再現されることが知られている。このとき、ファイマン則に似たダイアグラマティックな方法が使われ、バーテックスに対応する関数は、"topological vertex"と呼ばれている。

これに対し、元々の2パラメータNekrasov公式を再現するtopological stringがどのようなものかは、これまで知られていなかった。最近"refinedtopological vertex"と呼ばれる2パラメータを含む拡張されたtopological vertexがVafaたちによって提唱されたが、Nekrasov公式が持つ性質を再現しないなど、最終的な解決には至っていなかった。特に擬似的なファイマン則のpropagatorに相当する因子に不定性があった。

瀧氏は、疑似ファイマン則のpropagatorに相当する関数の中の"framing factor"と呼ばれる因子を改良し、それを用いればrefined topological vertexを用いて正しく2パラメータのNerasov公式を再現できることを示すことに本研究において初めて成功した。また、疑似ファイマン図のs-channelt-channel dualityに対応するrefined topological vertexの"flop invariance"についても成立することを証明した。これらの業績は、フルのNekrasov公式とtopological string theoryの関係を理解する上で重要な手掛かりを与える結果であり、今後の発展が期待されるものである。

したがって、博士(理学)の学位を授与できると認める。