arxiv: v2 [cond-mat.str-el] 27 Dec 2018

Transcription

1 Fermi surface instabiities of symmetry-breaking and topoogica types on the surface of a three-dimensiona topoogica insuator Subhajit Sarkar 1, 1 Institute of Physics, P.O.: Sainik Schoo, Bhubaneswar , Odisha, India. (Dated: December 31, 018) arxiv: v cond-mat.str-e] 7 Dec 018 The emergence of the Pomeranchuk instabiity (PI) in a Heica Fermi iquid (HFL) residing on the surface of a three-dimensiona topoogica insuator (3D TI) is addressed at the mean-fied eve. An expression for the PI condition is derived in terms of a few microscopic parameters in each anguar momentum channe corresponding to a centra interaction between the heica eectrons. It is found that because of the presence of strong spin-orbit couping (SOC) the Landau parameter, F corresponding to a particuar anguar momentum channe depends not ony on the eectroneectron interaction in the same channe but aso interactions in ( +1) and ( 1) channes. The formaism automaticay excudes the 1 PI in the HFL where the Gaiean invariance is broken because of the presence of strong SOC. It is aso found that the competing PIs can ony be avoided unti the appearance of PI. In this case, the corresponding nematic instabiity can even be achieved in the 1 anguar momentum channe of interaction between the eectrons. The range of interaction between the eectrons pays a pivota roe in bringing out the PIs. This is estabished by anaysing a few reaistic profies of the interaction. Another cass of instabiity, invoving a change in the topoogy of the Fermi surface without breaking the rotationa symmetry, is found which competes with the PIs. Quantum phase transition originating from this instabiity is quite simiar to the Lifshitz transition but is driven by eectron-eectron interaction. Possibe connections of this instabiity with experiments are aso described briefy. PACS numbers: Nq, Hf, Ay I. INTRODUCTION Topoogica insuators (TIs) are a new cass of materias which possess insuating buk states but conducting boundary states 1 4]. These materias have attracted a great dea of research interest party driven by their appicabiity in technoogy and party because of their fundamenta non-trivia characteristics. In particuar, three-dimensiona topoogica insuators (3D TIs) are non-magnetic insuators that possess metaic surface states with gap-ess spectrum as a consequence of the non-trivia topoogy of the buk eectronic wave functions of the materia 5, 6]. The simpest non-trivia 3D TIs, which are aso caed the strong TI, exhibit inear dispersion reation with the structure of a Dirac cone, and an odd number of such cones are present on the surface of the sampe which is further ensured by Z topoogica invariant of the buk 3 6]. Furthermore, such a phase emerges from the strong spin-orbit couping (SOC), eading to spinmomentum ocking on the surface of 3D TIs 5, 6]. On the surface of a 3D TI, because of the existence of an odd number of Dirac cones, the Fermi sea is a circuar disk containing the Dirac point itsef 3 6]. Surface Fermions constitute a Heica Fermi gas, and with sbhjt7@gmai.com present address: Department of Chemistry, Ben Gurion University of the Negev, Beer Sheba , Israe. Emai: subhajit@post.bgu.ac.i adiabaticay switching on the interaction these surface Fermions ead to a Fermi iquid phase on the surface of the 3D TI which is caed a Heica Fermi iquid (HFL) 7]. Moreover, the existence of a singe non-degenerate circuar Fermi surface makes the HFL an effective spiness Fermi iquid a behaviour whose appearance has its root in the topoogy of the buk 7]. Such a Heica Fermi iquid theory is appicabe to a number of experimentay reaized 3D TIs, for which the Fermi surface has been found to be very neary circuar. These are, Bi Se 3, Bi Te Se, Sb x Bi x Se Te, Bi 1.5 Sb 0.5 Te 1.7 Se 1.3, T 1 x Bi 1+x Se δ, strained α Sn on InSb(001), and strained HgTe, where the observations have been made mosty using ange-resoved photoemission spectroscopy (ARPES) 8 16]. The behaviour of the HFL is described by the projectedlandauparameters F whicharesimiarinspiritto the Landau parameters of a conventiona SU()/Gaiean invariant Fermi iquid (SU()-FL) 7]. However, these Landau parameters are obtained by projecting the Landau parameters of a spin-orbit couped (non-gaiean invariant) FL on the Fermi surface (FS) of a particuar Heicity (originating from strong SOC) for non-zero chemica potentia µ (therefore non-zero doping) 7]. Low energy properties of the HFL are strongy infuenced by the shape of the FS. When the (repusive) interaction between the quasi-partices near the Fermi surface increases sufficienty the system evoves into a new phase where the symmetry of the Fermi surface is broken. The resuting shape deformation instabiity is renowned as Pomeranchuk Instabiity (PI) which occurs in a particuar anguar momentum channe, when corresponding

2 the Landau parameter F becomes sufficienty negative 17 1]. For exampe, in HFL the PI eads to a new phase exhibiting nematic order characterized by eongation (contraction) of the Fermi surface aong k x (k y ) direction and vice versa, and more interestingy a partia breakdown of spin momentum ocking (in the sense that spin and momentum are no-onger orthogona) ]. In the case of SU()-FL such PIs have been studied in a sufficienty genera frame work by taking into account a generic centra Fermion-Fermion(e-e) interaction both in two and three dimensions 3, 4]. This singe framework describes PIs not ony in but aso in other anguar momentum channes, and automaticay excudes 1 PI in a Gaiean invariant Fermi iquid. The absence of 1 PI corresponding to the absence of spontaneous generation of currents has attracted recent attention, and investigations revea that whie in a Gaiean invariant system the Gaiean invariance aone abandons this, in a non-gaiean invariant system such as the HFL, the contributions from the high energy degrees of freedom cance this spontaneous current generation 5, 6]. Furthermore, previous works have focused mosty on superconducting instabiities, time-reversa symmetry breaking instabiities, and nematic instabiities, 7 47]. However, a thorough study of the PIs is sti missing. Motivated by these facts, in this paper, I investigate the PI on the surface of a 3D TI originating from a centra interaction, and findings are quaitativey quite different from those corresponding to the SU()-FL, the reason being the presence of strong SOC. In this paper, I derive a microscopic expression for the Pomeranchuk instabiity corresponding to the HFL residing on the surface of a 3D TI. The nature of the corresponding phase transition is described in terms of a few pertinent parameters of the theory representing the properties (incuding its curvature) of the e-e interaction on the Fermi surface. Most importanty, because of the presence of strong SOC on the surface of a 3D TI, the different anguar momentum channes get couped and the Landau parameter for a particuar anguar momentum channe depends on interaction strengths not ony in the same channe but aso in channes higher and ower than that. This finding has a far-reaching impication in the nematic instabiity corresponding to channe as expained in this paper. Furthermore, over a range of parameters the Pomeranchuk transition driven by the centra interaction becomes first order, thereby signaing discontinuous transition without invoving any quantum critica behaviour. However, nematic instabiity coming from the hypothetica (non-) centra interaction (when compared with the phenomenoogica quadrupoar interaction) fas within the parameter regime where the phase transition is of second order in nature signaing quantum critica nature of the isotropic-nematic transition at zero temperature. This is in agreement with the earier resuts ]. Using a few modes ofthe interaction, I have further shown that apart from the PIs there exists another cass of instabiity invoving a change in the topoogy of the Fermi surface (FS) which is driven by eectron-eectron interaction and competes with the PIs. Moreover, the range of the interaction turns out to be a decisive factor in the appearance of both PI and topoogica-fs instabiity. In this regard, it is worthwhie to mention that the appearance of the Lifshitz transition invoving a change in the topoogy of the Fermi surface has been reported recenty in the doped Topoogica Crystaine Insuators (TCIs) Pb 1 x Sn x Se and Pb 1 x Sn x Te where, by changing the doping an interaction-induced phase transition invoving a change in the FS topoogy has been achieved 48, 49]. The paper is organized as foows. I have described the mode in section II. In section III, I have described the mean-fied theory and derived the microscopic expression of the projected Landau parameters. Section IV describes the microscopic version of the PI condition, and in section V, I expain the nature of the Pomeranchuk phase transition and the corresponding order parameters. Section VI describes the conditions under which the PI can describe the nematic instabiity originating quadrapoar eectron-eectron interaction. In section VII, I have described the importance of range in bringing out the PI and topoogica instabiity. Section VIII concudes the paper and discuss the impications of the resuts. Reevant cacuations are presented in Appendices. II. THE MODEL The non-interacting Hamitonian corresponding to the gap-ess states with a singe Dirac cone on the surface of a three dimensiona topoogica insuator(3d TI) is given by (in the units of k B 1), H 0 d k (π) σ,σ c k,σ v0 Fẑ (τ σ,σ k) µ]c k,σ, (1) where v 0 F is the Fermi veocity the bare partices, τ σ,σ is the vector Paui matrix, (σ,σ ) are the spin indices of the eectrons, and µ is the chemica potentia. The heica Fermi iquid theory corresponding to these surface states in the presence of eectron-eectron interaction is stricty vaid when µ > 0, i.e., in the doped imit 7]. One can diagonaize the above Hamitonian by an unitary transformation, ψ k,s 1 (ie iθ k c k, +s c k, ) () which transform the compex Dirac fermions c k,σ to heica Dirac Fermions ψ k,s with heicity s ±, where θ k tan 1 k y k x 7]. In the aboveequation the heicity + corresponds to a the partices with positive Fermi energy ǫ F above the Dirac point and the heicity - corresponds to a the partices with negative Fermi energy beow the Dirac point. With chemica potentia µ > 0 ony the states above the Dirac point are avaiabe. Therefore, the inverse of the () is given by, c k,σ η k ψ k,+

3 3 where η k 1 ( ie iθ k 1 ). Here the negative heicity Fermion operators are dropped atogether,i.e., a the compex Dirac Fermions avaiabe in the system are projected onto the positive heicity Hibert space. For ease of notation I sha denote the ψ k,+ as ψ k without expicity using the + notation. is given by, H int 1 d k 1 d k Ω D (π) (π) ψ k 1 q ψ k +q ψ k ψ k1, with V(k 1,k,q) V(q) e i(θ k 1 θ k1 q) d q (π) V(k 1,k,q) + e i(θ k θ k +q) +e i(θ k 1 θ k1 q+θ k θ k +q) ]. (5) FIG. 1. (a) Dispersion reation ǫ k. For µ > 0 the ony avaiabe states are corresponding to the Dirac cone for which ǫ k 0. (b) The shaded circuar region on (k x,k y)-pane represents the fied Fermi sea with radius µ v F, undeformed Fermi surface being a circe. The diagonaized non-interacting Hamitonian in the projected space is then given by, H 0 d k (π) (ǫ k µ)ψ k ψ k, (3) where ǫ k vf 0 k is the dispersion reation corresponding to non-interacting Dirac Fermions as shown in FIG. 1 and k kx +k y. The ground state of the non-interacting system is the fied Fermi sea and is given by, F k< ψ k 0, where 0 is the vacuum. Once the interaction is switched on adiabaticay the system evoves into a heica Fermi iquid (HFL) with Hamitonian, The e-e interaction V(q) in the above equation is such that its Fourier transform V( r r ) is a centra potentia, Ω D being the surface area of the system which I sha take as 1 for the rest of the cacuations. The projected Hamitonian corresponding to (4) therefore, represents that of an effective spin-ess Fermi iquid however, at the cost of the eecton-eectron interaction becoming phase dependent 7]. In this paper, I consider a repusive interaction, which with a sufficienty genera dependence on the momentum transfer (k k ), eads to anisotropic state with deformed Fermi surface through Pomeranchuk instabiity characterized by finite interna anguar momentum of the corresponding order parameter. Attractive interaction usuay eads to superconductivity in the BCS channe 7]. The formaism considered in this paper foows cosey that of the Ref. 3], and is sufficienty genera in the sense that it depends ony on some pertinent parameters derived from V( r r ). These parameters are V (n) (k,k ) which are determined from the foowing reation, V(k k ) V (k,k )cosθ k,k, 0 and the ampitudes V are given by, V (k,k ) π(1+δ,0 ) 0 drrv(r)j (kr)j (k r), (6) where k k, θ k,k (θ k θ k ), and J (kr) are the ordinary Besse functions of first kind. The quantity V (k,k ) represent the ampitude of the interaction in th anguar momentum channe. The symbo (n) mentioned above represents the order of the partia derivatives of V (k,k ) with respect to k and k. Such a parametrization is quite common in the iterature. V (k,k ) and their partia derivatives of a orders constitute the microscopic parameters of the theory. H H 0 +H int, (4) where the Fermi veocity vf 0 appearing in (1) and (3) is now repaced by the renormaized Fermi veocity v F of the interacting heica Fermions. The eectron-eectron interactionhamitonianh int, writtenintheheicabasis, III. MEAN-FIELD THEORY AND MICROSCOPIC EXPRESSION FOR PROJECTED LANDAU PARAMETERS To describe a microscopic theory of Pomeranchuk instabiity on the surface of a 3D TI, I first reca the Landau functiona corresponding to the effective spin-ess

4 4 HFL 7], δēδ n d k k] (π) (ǫ k µ)δ n k + 1 d k d k F (π) (π) ρ(ǫ F ) cos(θ k,k ) δ n k δ n k, 0 }{{} f(k,k ) (7) where F are the projected Landau parameters whose microscopic expression in terms of V (k,k ) sha be derived on the Fermi surface, f(k,k ) is the Landau interaction function, and ρ(ǫ F ) kf πv F is the density of states (DOS) at the Fermi surface. In the above equation δ n k n k n 0 k where n k ψ k ψ k is the quasipartice distribution function and the superscript 0 denotes the corresponding ground state distribution of the bare partices. The requirement of δē < 0 eads to the PI condition, F > 1 (8) appied ony to the projected Landau parameters. Since a mean-fied theory is equivaent to doing a one-oop static Hartree-Fock approximation (because of static interaction considered in this case) to the Fermionic sef energy, one can readiy write down the ground state energy, Ē d k (π) n k d k (ǫ k µ)+ 1 (π) ( V(0) 1 (1+cosθ k,k )V(k k ) ) ] n k, (9) where the static sef energy is given by, d k ] 1 Σ(k) (π) (1+cosθ k,k )V(k k ) V(0) n k. (10) The quasi-partice distribution function is given by, n k Θ( E k ) with a renormaized dispersion reation, E k ǫ k µ Σ(k). With this definition of the renormaized dispersion reation the ground state energy takes the form, Ē d k (π) n ke k + 1 Σ(k)]. The variation of the above equation with respect to quasipartice distribution function eads to (7) but with a renormaized dispersion reation, E k and a microscopic expression for the Landau interaction function which is given by, f(k,k ) 0 ( δ,0 ) F ρ(ǫ F ) cos(θ k,k ) 1 ] (1+cosθ k,k )V(k k ) V(0) (11) on the Fermi surface. In the above equation, I have used the conventions and notations used in Ref. 19]. Further, using (6) it is easy to find the microscopic expression for the projected Landau parameters, F ρ(ǫ F ) δ,0 V V 4 (1+δ,0) 1 8 (V +1 +(1+δ,1 )V 1 ) ], (1) within the mean-fied description where the quantity V V(0) d rv(r) is the voume (D) average of the interaction; see Appendix A for detaied derivation. It is due to the presence of strong SOC, the microscopic expression of the projected Landau parameter corresponding to th anguar momentum channe invoves interaction strength corresponding to,( 1), and ( + 1) th anguar momentum channes. However, if a correction to the mean-fied theory is considered then more higher and ower anguar momentum channes are expected to appear in the Landau parameter corresponding to the th anguar momentum channe. The renormaization of Fermi veocity, v F in the HFL is the anaogue of the quasi-partice mass renormaization of the SU() invariant Fermi iquid. In absence of Gaiean invariance the standard way to derive v F is to use the fact that due to adiabatic continuity the tota fux of quasi-partices is equatothat ofthe freeeectrons. Astraightforwardcacuation (see Appendix B) eads to renormaized Fermi veocity expressed in terms of the microscopic parameters of the theory, and is given by, v 0 F v F (1+ F 1 ), (13) where vf 0 is the Fermi veocity of the bare partices and F 1 ρ(ǫf) 8 (V 0 + V 1 + V ) which is (1) with 1. Here V V (, ) which correspondsto (6), evauated at k and k. IV. SMALL FERMI SURFACE DEFORMATION AND INSTABILITY CONDITION In order to quantify the sma deformation of the Fermi surface I first expand the Fermionic sef energy in circuar harmonics, Σ(k) Σ m (k)cos(θ k ), (14) m0 where m Z 0 are the anguar momentum quantum numbers of the heica eectron-hoe pairs. The renormaized dispersion then takes the foowing form, E k E 0 (k) Σ m (k)cos(mθ k ) (15) m1 where E 0 (k) ǫ k µ Σ 0 (k) is the symmetric part of the dispersion reation before the instabiity sets in.

5 5 The symmetric part of the renormaized dispersion reation further determines the Fermi surface (FS) via the reation E 0 ( ) 0, where remains unchanged due to Luttinger s theorem, before the instabiity sets in50, 51]. For any sma deformation of the FS corresponding to + δ (θ), with δ (θ) << the symmetric part of the dispersion reation can be inearised, by Tayor expanding it, to have, E 0 ( +δ (θ)) E 0 ( ) +v F δ (θ), (16) }{{} 0 when in the Tayor expansion in the above equation, for a n 1, the foowing conditions are satisfied: (δ ) n ( n+1 ) E 0 (k) (n+1)! k n+1 v F, and (δ ) n 1 ( n ) Σ (k) n! k n v F, (17) not ony for 0 but aso for a 0. Furthermore, with these conditions being satisfied we can safey assume Σ ( + δ (θ)) Σ ( ) Σ, near the FS for sma deformation of the same. This signifies that the k dependent ampitude of the deformation potentia can be approximated to be a constant near the FS. The above conditions further restricts us to consider ony sma FS deformations. In the symmetry-broken phase with (θ) +δ (θ), thefs isnoongerdetermined by the symmetric part of the dispersion reation, instead it is determined by the fu renormaized dispersion reation, E( +δ ) v F δ (θ k ) 1 Σ cos(θ k ) 0 which impies, δ (θ k ) 0 Σ v F cos(θ k ). (18) The summation in the above equation extends from 0 to, because in the isotropic phase corresponding to 0 the deformation δ (θ) vanishes. This further impies Σ 0 0 from the above equation. Using (6), (10) and (14) it is straight forward to construct a sef consistency equation for Σ for a 0 which is given by, Σ 1 π π 0 dθ k 4π kf+δ(θ k ) k dk π + π 4 {V +1(,k )+(1+δ,1 )V 1 (,k )} V (,k ) ] cos(θ k ). (19) It is worthwhie to note that the above equation is sefconsistent via the dependence of δ (θ k ) on Σ through (18). The sef-consistency equation presented above can further be reduced to a poynomia equation in Σ to a given order in δ (θ k ) if one expands V (,k ) corresponding to the above equation in a Tayor expansion, V (,k ) V (, )+ (k ) j j1 j! V (j) (, ). Here V (j) (, ), being the j th partia derivative of V (,k ) with respect to k at. In the mean-fied theory described here V (, ) V and V (j) (, ) are the microscopic parameters of the system characterizing the nature of the interaction potentia incuding its magnitude and curvature on the Fermi surface. Soving the sef-consistency equation (19) to the owest order in δ (θ), i.e. by taking V (,k ) V (, ), the microscopic form of the instabiity equation for 0 can be derived to be, V V / + V +1 +(1+δ,1 )V 1 ] 1, (0) (see Appendix C for detaied cacuations) where the critica interaction strength required for the instabiity 8 to set-in is ρ(ǫ F), which is determined by the DOS ρ(ǫ F ) at the Fermi surface, and V V + (1 + δ,1 )V 1 + V +1 ]. At this point (0) demands a consideraby detaied anaysis. Instabiity can occur at a particuar anguar momentum channe, not ony if the interaction strength in that channe exceeds a vaue / but aso if the interaction strength in ± 1 channe exceed. Therefore, one may in principe get a Fermi surface instabiity in th anguar momentum channe by a sufficienty strong interaction in the anguar momentum channe one step ower to, i.e. ( 1). However, the interaction in ( 1) th channe must be more than twice of the same in the th channe. This finding has more interesting consequences in the case of nematic instabiity corresponding to, to be discussed in Section VI. As a whoe, for V < 1 the un-deformed FS is a minimum energy configuration. However, for the condition, V > 1 the corresponding deformed FS is the minimum energy configuration, and this condition is the microscopic version of the PI condition (8). This can be easiy seen by rewriting the above equation (0) using (1). Using the expression of the renormaized Fermi veocity it is easy to appear at foowing microscopic form of the PI condition (8), V +V +1 +(1+δ,1 )V 1 ] (V1 +V 0 +V ) 16πv0 F, (1) fora 1,, 3,..., wherev 0 F isthefermiveocityofthe bare partices and the equaity sign represents the critica point where the PI sets in. The above equation represents one of the main resuts of the paper, viz., the microscopic expression of the PI condition. This equation further acks any soution for 1 estabishing the absence of 1 PI, i.e., absence of any spontaneous current generation. It is worthwhie to mention that the critica vaue of the interaction strength 16πv0 F +(V 0 +V 1 +V ) depends not ony on the interaction strength in a of the 0, 1, anguar momentum channes but aso on the properties of the system in absence of interaction, viz., the Fermi momentum and vf 0. The absence of 1 PI thereby becomes intimatey reated to the above mentioned properties of the non-interacting system through

6 6. Within the mean-fied theory, the equation (1) in turn gives a manifestation of the absence of 1 PI ony in terms of properties of the non-interacting system. Furthermore, from (8) and (1) it is easy to recognize thatfor 0the instabiityequationtakesthe form, V 0 V 8 V > 16πv0 F. In this reation, an be evauated by taking the imiting vaue, im k k V(k k ). Using (6) one finds V 0 V (, ) on the Fermi surface. Therefore the 0 instabiity equation takes the form, ] V 0 V 8 V (, ) 1 > 16πv0 F, () which signifies that the instabiity at 0 channe (corresponding to a phase separation instabiity) can be achieved ony via attractive interaction in not ony the 0 channe but aso in any other anguar momentum channes. This is the microscopic manifestation of the fact that unike the SU()-FL, in HFL an instabiity towards the phase separation can be achieved not ony through attractive density-density interaction but aso as a resut of spin-charge or even pure spin-spin interaction 7]. However, it is important to note that the interaction must be attractive in the anguar momentum channes which are participating in the instabiity. This wi be further concretized in section VII with the exampe of a δ-she potentia. strengthifv (1) < 0(otherwisethetermwithinthesquare root woud have been imaginary!), and the corresponding configuration minimizes the energy. This situation is depicted in FIG (a) and is structuray quite simiar to that of Ref. 3], however, bears remarkabe quaitative differences ony because of the presence of strong SOC on the surface of a 3D TI. For V (1) < 0, the above Σ/kFvF Σ/kFvF п Vc (a) п 0. V. NATURE OF THE PHASE TRANSITION In order to investigate the nature of the phase transition emerging through PIs, I now sove the sef consistency equation in next higher order in δ (θ), i.e., use V (,k ) V (, )+V (1) (, )δ (θ k ). Using (18) the foowing sef-consistency equation can be found for a (since 1 instabiity is excuded), ] Σ 1 V Σ + V(1) Σ 3, (3) 8π v F 4v 3 F where V has been defined earier (see discussions near (0), with 1) and V (1) V (1) + V (1) 1 + V (1) +1 (see Appendix C for the detais of the cacuations). This equation has two soutions, one corresponding to Σ 0 which represents a phase corresponding to zero deformation of the Fermi surface which minimizes the energy when the interaction strength is ess than. The other soution is, Σ ± V ( ) 1/ c V +V 1 +V +1 1, v F V (1) (4) which describes the physica Fermi surface deformation ampitude and its variation with respect to interaction (b) FIG.. (a) Pot of the ampitude of the Fermi surface deformation Σ v F as a function of interaction strength. Here V V + V 1 + V +1, and Vc 1 on the Fermi V (1) surface. The back vertica ine indicates the critica interaction strength. (b) Pot of the soution of (6) with + v() η 1/100 and v (3) 1. In this pot, the vertica dashed ine at the critica point represents a first order jump in the order parameter. In the inset the bound for the interaction strength represented by thick vertica dashed ines is potted (see text for expanation). v (1) equation further describes the appearance of a brokensymmetry phase through PI with a mean-fied order parameter Ψ k cos(θ k)θ( +δ (θ k ) k), corresponding to th anguar momentum channe, and a second order quantum phase transition with a mean-fied critica exponent 1/, as expected. This mean-fied order parameter is indeed proportiona to the ampitude of the Fermi surface deformation Σ v F thorough δ (θ k ), and is given by п Vc Ψ Σ. (5) π v F

7 7 This wi further be discussed in detai in the case of the nematic instabiity in the next section. In situations when the interactions in the anguar momentum channes exceed the critica interaction strength in a hierarchica manner from smaer vaues of to arger ones (say a hierarchy assumption ), even a sufficienty strong interaction in ( 1) channe in principe can drive the system towards a PI in the th channe. This situation is described as phase diagrams in FIG 3. In the case when V η Vc, with 0 < η < 1, the PI condition corresponding to (0) takes the form (V +1 + V 1 ) > (1 η), a situation depicted in FIG 3(a) which corresponds to η 0. In the opposite case when η 1 the PI in the th channe appears irrespective of the interaction strength intheotheravaiabechannes(inthiscasev 1 andv +1, within mean-fied theory). The case corresponding to arbitrary η is described in FIG. 3(b) where a the three anguar momentum of interaction compete with each other. When V (1) > 0, the ampitude of the Fermi surface deformation corresponding to (4) turns out to be imaginary and therefore unphysica. In this case the sefconsistency equation is needed to be soved in the next higher order in δ (θ k ) 3]. For the ease of notation I now introduce few dimensioness parameters, λ Σ v (n) V(n) kf n v F,, and v V, foowing Ref. 3]. The sefconsistency equation corresponding to (4) now takes the form (see Appendix C for detais), ( v (3) 48 λ5 + 1 v (1) 4 ) + v() λ 3 +(v 1)λ 0, (6) which has one soution, λ 0 corresponding to the isotropic phase with no deformation. As a characteristic of the mean-fied theory, this equation is structuray the same as that obtained in the case of SU() invariant Fermi iquid and therefore a the concusions corresponding to the atter hod true 3]. However, the quaitative differences originate from the strong SOC present in the system. For competeness the situations are described here too. When v (3) < 0, one gets two types of soutions, one corresponding to v (1) + v() < 0, and the other corresponding to v (1) + v() > 0. Forv (1) + v() ξ, the soutionofthe aboveequation grows in a second-order fashion as shown in the FIG (a) however, with a shifted quantum critica point given by, V / (1 3ξ /4). Therefore, the higher order correction in δ has its effect in shifting the quantum critica point with the shift being controed by the nature of the centra interaction and its curvature on the Fermi surface. In the other case, for v (1) + v() ξ the behaviour of the soution is potted in FIG (b), which shows a jump in the order parameter corresponding to a first order phase transition, before the PI sets in. Anaysing the soution it is easy to find a bound in the interaction FIG. 3. The white portion of the diagram represents the isotropic phase; symmetry broken phase is indicated in the diagram. Density represents the ampitude of the deformation corresponding to (4). (a) This phase diagram corresponds to the situation when the interaction in th anguar momentum channe is zero. In this case, the critica strength required for instabiity to occur is the same for both ( 1) th and ( + 1) th channes. This further represents the situation when PI corresponding to th channe can be achieved through interaction in the th channe exceeds the critica strength. (b) This phase diagram corresponds to the situation when the interaction in th anguar momentum channe and (±1) th channe compete with each other. The horizonta back arrow aong the V /-axis represents the situation when interaction in (±1) th channe is zero. In this case the critica strength required is /. The vertica back arrow represents the situation when V 0 and the instabiity can be achieved in either ( 1) th or ( + 1) th channe. In the case when a the interactions are present, the arrow woud be in the (V / V ±1 /)- pane (corresponding arrow is shown in the diagram).

8 8 strength which is given by, ) (1 3ξ 4 V jump < 1. (7) The quaitative difference from the SU()-FL once again appears in the above equation in the form of the fact that the interaction corresponding to the jump depends on the interactions in, ( 1), and ( + 1) th anguar momentum channes. Therefore the concusions corresponding to the hierarchy assumption introduced earier remain vaid. From the inset of FIG (b), it is easy to see that initiay the order parameter deveops even if the interaction strength decreases however, such a decrease is bounded from beow as indicated in the equation above. On reaching the ower bound the order parameter starts growing in a second order fashion and PI sets in when the interaction strength exceeds. Therefore, in this case the system goes through a weak first order phase transition before it utimatey enters in to the brokensymmetry phase. First order transition here is argued to be weak because it appears ony in a very narrow parameter regime. For v (3) < 0, one need to consider soving the sefconsistency equation to even higher order in δ (θ) to resove the issue of unphysica soutions. However, since ony the sma Fermi surface deformations are considered here, appearance of such parameter ranges corresponding to v (3) < 0 are restricted. It is worthwhie to point out that, in the expression of the Pomeranchuk order parameter corresponding to (5) the renormaized Fermi veocity v F appears in the denominator. The renormaized Fermi veocity depends on the interaction strengths in 0, 1, anguar momentumchannes. However,since 1PIisruedoutv F can not diverge even if the interaction strengths in these anguar momentum channes exceeds the corresponding critica vaue, but interestingy enough, in this situation the PI sets in, as expained in the next section. VI. QUADRUPOLAR INSTABILITY With sufficienty strong e-e interaction in the anguar momentum channe the the ground state energy of an HFL is further owered by spontaneous quadrupoar deformation of the corresponding Fermi surface, eading to a nematic phase ]. Such spontaneous rotationa symmetry breaking in the surface states of 3D-TI has been studied considering the foowing effective Hamitonian, H H drdr F(r r )TrQ (r)q(r )] (8) where Q(r) is a matrix of rank two which is rea, symmetric, and traceess ]. In the projected space corresponding to µ > 0, the eements of the matrix Q(r) are given by, Q ab (r) ψ (r)ˆ aˆ b 1 δ abˆ ]ψ(r). These matrix eements determine the order parameters in the nematic phase of the HFL on the surface of a 3D-TI, and written in the origina un-projected basis such order parameters contain both spin and charge degrees of freedom and inear in eectrons momentum, unike the case corresponding to SU() FL, 5]. This is because, in the presence of a strong SOC the rotationa symmetry invoves simutaneous rotations in momentum and spin spaces generated by the tota anguar momentum ]. In order to understand the connection between theories based on the above Hamitonian and the Hamitonian (4) with centra interaction, foowing Ref. 3] I need to construct a Hamitonian whose interaction term is constrained to contain ony terms of the form of (8). To demonstrate expicity I first make a Fourier transform of the above Hamitonian (8) so that, H d k (π) (v Fk µ)ψ k ψ k + (π)4 d k d k d ] q 8kF 4 (π) (π) (π) F(q) k q k+ q cos(θ k q/ θ k+q/ ) ψ k + q ψ k qψ k ψ q k+ q, (9) where the quantity F(q) (π) F δ(q 0) and for nematic instabiity the vaue of F is usuay taken to be negative]. With this choiceinow comparethe sef energy of the above Hamitonian with the coefficient of the cos(θ kk ) of the Hartree-Fock sef energy corresponding to (10) by using the parametrization (6). Simpe cacuation the eads to, V (k,k ) V (k,k )+V 3 (k,k )+V 1 (k,k )] (π)4 kf 4 k k F (30) for attractive interaction, i.e., F F. Therefore, athough nematic instabiity originates from a quadrapoar interaction which is not a centra interaction, it can be considered to be originated from a hypothetica interaction whose reevant term containing cos(θ kk ) has the from (30). This equation further suggests that the nematic instabiity originating from such hypothetica interaction can be achieved through any one or a of the anguar momentum channes of interaction with 1,, and 3. Moreover,onthe Fermisurfacethe above

9 9 equation(30) describesthe PIcondition, (π) 4 F, i.e., with sufficienty arge but negative vaues of F the system enters into the nematic phase where the equaity represents the quantum critica point corresponding to the onset of PI. However, not a the anguar momentum channes require equay arge interaction strength to achieve the instabiity for exampe, strength required in channe is haf of that required in 1 channe. Now if I consider a situation, which is quite natura, when the anguar momentum channes are activated in a hierarchica manner from the ower to higher vaues of (the hierarchy assumption as mentioned earier), it is very much possibe to achieve the instabiity ony through 1 channe of interaction athough it is easier in channe. Furthermore, since 1 PI is absent, the nematic instabiity is competed ony by the instabiity in 3 anguar momentum channe, and the hierarchy assumption therefore, has its impication in this case in avoiding competing instabiity coming from 3 anguar momentum channe. From (30) it is easy to recognize that V (, ) (π)4 F < 0 which impies that the isotropic to nematic transition is a second order phase transition, in agreement with the Ref. ]. It is worthwhie to point out that in the case of the conventiona Landau Fermi iquid, it has been essentia to incorporate non-inear terms in the non-interacting dispersion reation to stabiize the nematic quantum critica point 5]. However, in case of HFL which is stricty vaid for finite doping ony, µ > 0, the inear dispersion reation corresponding to a singe Dirac cone has been sufficient to stabiize the quantum critica point and phase transition becomes continuous in nature even at T 0 ]. To find the reation between the nematic order parameters deveoped from the quadrupoar interaction corresponding to (8) and the Pomeranchuk order parameter deveoped from the centra interaction corresponding to (4), I now derive the nematic order parameters corresponding to the quadrupoar interaction which are defined by, Q xx (q 0) Q yy (q 0) d k k (π) kf cos(θ k )Θ( +δ (θ k ) k) and Q xy (q 0) Q yx (q 0) d k k (π) kf sin(θ k )Θ( +δ (θ k ) k) (31) at zero temperature. In the isotropic phase the above order parameters vanish. Evauating the integras by using the imit of k-integrations from 0 to + δ (θ k ) and using (18) it is easy to find, Q xx (q 0) Q yy (q 0) Σ Ψ 8π v F 4 (3) and Q xy (q 0) Q yx (q 0) 0, where Σ is given by, Σ v F ± (π) 4 F ( V +V 1 +V 3 ) 1/ 1, (33) and (5) has aso been used. The above equations therefore, express the nematic order parameter of a nematic heica fermi iquid on the surface of a 3D TI in terms of the Pomeranchuk order parameter obtained in the HFL description with centra interaction. Nematic order parameter aso grows in a second-order fashion with universa mean-fied critica exponent 1/. VII. δ-shell MODEL AND POSSIBILITY OF TOPOLOGICAL PHASE TRANSITIONS The phase separation instabiity corresponding to 0 anguar momentum channe can not occur when the interaction is repusive, as expain in section IV. In this section this fact is further expained by studying a specific form of the interaction, V(r) gδ (1) ( r r 0 ), (34) where g is the strength of the e-e interaction. This interaction profie exhibits a very sharp peak at a particuar inter-partice distance r 0. This further becomes a contact interaction when r 0 0, whie finite vaues of the r 0 represent a finite range interaction. Such an interaction has been studied in the case of conventiona Fermi iquid in three dimensions to estabish the fact that in order to have PI in isotropic systems the range of the interaction must be finite and r 0 k 1 F 4]. Furthermore, study of such an interaction aso have reveaed that apart from PI, the three dimensiona isotropic Fermi iquid aso exhibit an interaction driven Lifshitz-ike transition which does not invove any symmetry breaking 4]. Motivated by this, I study the fate of PIs in different channes and possibiity of such interaction driven topoogica Lifshitz transition in the HFL by considering the above mentioned form of the interaction. Both repusive (g > 0) and attractive (g < 0) interactions are considered keeping in mind the fact that ony repusive interaction eads to PI. The PI equation corresponding to 0 anguar momentum channe is given by, V 0 V 8 V > 16πv0 F, (35) where V 0 πgr 0 J 0 ( r)], V πgr 0 J ( r)], and V πr 0 g. In FIG. 4 the corresponding phase is potted by soving the above equation and it can be easiy seen that the phase separation instabiity corresponding to 0 channe can ony appear when the interaction is

10 10 g v 0 F attractive, i.e., < 0. The dimensioness couping constant g and the dimensioness range of interaction k vf 0 F r 0 are the ony two parameters avaiabe in the theory. Sim- g vf r type FIG. 5(a) type FIG. 5(b) from the corresponding overapping PI domes in FIG. 4. Moreover, PI corresponding to 3 for repusive interaction can ony appear if the interaction possesses a finite range, and PI domes appear ony within certain vaues of interaction range, for exampe the 4 PI appears for 5 r However, in the case of contactinteraction corresponding to r 0 0, no PI can appear whatever may be the vaue of the strength of the repusive interaction (g > 0), as can be seen from FIG. 4 which doesn t exhibit any PI dome near r 0 0. E - FIG. 4. Phase diagram showing zero-temperature instabiities for both repusive and attractive δ-she mode. is g the v 0 F dimensioness couping constant and r 0 is the dimensioness range of interaction. Attractive interaction corresponds g to the negative vaues of and repusive interaction corresponds to the positive vaues of the same. Instabiities at different anguar momentum channes are indicated in the figure where the shaded regions are the symmetry broken phases. Regions bounded by ong-dashed ine (grey-shaded), dotted ine, and soid ine are symmetry-broken phase achieved in, 3, 4 channes respectivey. Regions bounded by bue short-dashed ine and purpe dotted-dashed ine are instabiities corresponding to FIG. 5(a) and FIG. 5(b) respectivey. v 0 F E (a) iary, the instabiity equation for channes, given by (1), can be soved and the corresponding phases are potted too in FIG. 4. It turns out that there is no PI when the interaction is repusive as can be seen from FIG. 4. Therefore, even within the mode of the δ-she interaction the nematic instabiity in anguar momentum channe can appear ony for attractive interaction which is consistent with the resuts obtained in section VI. Surprisingy enough, the dome corresponding to 3 PI is fuy contained within the dome corresponding to 4 PI as can be seen from FIG. 4. Furthermore, the strength of interaction needed to achieve 3 instabiity is much arger than that needed to achieve the 4 instabiity which indicates that 4 instabiity is more prone to appear. Because of the presence of strong SOC, interaction in different anguar momentum channes contribute quite distincty when instabiities in 3 and 4 channes are compared. This can be seen from equation (1). Therefore, it turns out that the strong SOC prefers the 4 instabiity over 3 instabiity. However, in certain parameter regimes, mosty when the interaction is much stronger (for exampe when g v 0 F 100, and r 0 5 corresponding to FIG. 4), the 3 and 4 PI do indeed compete, as is evident (b) FIG. 5. Renormaized dispersion reation for the δ-she mode for fixed r 0 in the Hartree-Fock ground state, (a) r 0 5, (b) r 0 8, where in the inset the corresponding Fermi surface configurations are drawn schematicay and in the both the cases the dispersion reations are indicated in dotteddashed curves. Apart from the PI, the δ-she potentia aso exhibits another type of Fermi surface instabiity which does not invove any symmetry breaking resuting from the shape deformation of the Fermi surface. This type of instabiity is purey an interaction driven one as expained beow. To anayse such an instabiity, the renormaized dispersion reation is needed to be evauated for the interaction under consideration. The renormaized dispersion reation is given by, E k v F (k ) Σ(k), (36) where the mean-fied sef energy is given by (10). In the state n k Θ( k ), it is easy to evauate the renormaized dispersion for the above δ-she interaction

11 11 by using (6). The renormaized dispersion reation can be evauated to be, E k v F (k )+ 0 drv(r) J 0 ( r)j 1 ( r) 1 J 0(kr)J 1 ( r) π 16 J 1(kr)J 1 ( r)h 0 ( r) + π ] 16 J 1(kr)J 0 ( r)h 1 ( r), (37) correspondingtoaninteractionv(r) whereh n (kr) isthe Struve function order n which is the soution of the nonhomogeneous Besse differentia equation54]. Using(13) and (34), the above mentioned renormaized dispersion reation is potted as a function of dimensioness ( ) range kr 0 in FIG. 5, where ǫ 0 v0 F r0, and g r 0ǫ 0 g is the vf 0 dimensioness couping as mentioned earier. From FIG. 5 it is easy to recognize that two situations appear, (1) the dispersion reation goes above the Fermi eve (corresponding to E k 0) at the centre of the Fermi surface (FIG. 5(a)), and () the dispersion reation peaks above the Fermi eve at some intermediate point 0 < k < (FIG. 5(b)). In the former, there appears a region in the k-space within the circuar Fermi surface which is vacated. In the atter case, there exists a thin she of vacated states for some k-vaues within the Fermi surface. The associated instabiities are competey different from the PI, as it invoves a change in the occupation number away from the Fermi surface and do not invove any symmetry breaking. However, the topoogy of the Fermi surface changes. Therefore, the phase transition occurring in these cases can be considered to be interaction driven topoogica phase transition simiar to the Lifshitz transtion, in anaogy with the corresponding SU()-FL in three dimensions (3D) 4, 55, 56]. It is worthwhie to mention that the phase transition corresponding to FIG. 5(a) and (b) appear ony when r 0 5 and r 0 8 respectivey, not at any other vaues of the range of the interaction as can be seen from FIG. 4. Furthermore, this type of topoogica phase transition compete with the Pomeranchuk type phase transitions too, which is easiy recognized from the overapping of the corresponding domes (see FIG. 4). However, such a competition is exhibited ony between the topoogica FS instabiity and 3,4 PIs. When the range of interaction is considered to be onger (the vaue of r 0 > 10 in FIG. 5), the PI domes reappear but are not competed by the topoogica instabiities. It is worthwhie to point out that a more reaistic hardcore potentia of the form V(r) g Θ(r 0 r ) does not exhibit any PI, however, exhibit ony one type of interaction-driven Lifshitz transition corresponding to the appearance of an annuar sheet of hoe Fermi sea within the. This can be obtained by soving the PI equation and equation (37) numericay. The anaysis performed in this section suggests that the interactiondriven topoogica Fermi surface instabiity is as ikey to occur as the PI. A ongrangeinteraction of Yukawatype, V(r) g r e r r 0 on the other hand, does neither revea any PI nor any topoogica Fermi surface instabiity. VIII. CONCLUSIONS AND DISCUSSIONS In this paper, I have investigated the emergence and the nature of Pomeranchuk and topoogica transitions originating from a centra eectron-eectron interaction on the surface of a 3D TI. The description is at the mean-fied eve and is in terms of a few microscopic parameters which are the anguar momentum components of the interaction potentia and their momentum space derivatives on the Fermi surface. This theory is appicabe ony to those 3D TIs for which the Fermi surface is neary circuar, and therefore such a continuum description remains vaid. The framework is very simiar to that corresponding to the conventiona SU() invariant Fermi iquids. However, quaitative as we as quantitative resuts are quite distinct owing to the presence of strong SOC in 3D TIs. The ony simiarity between the case of 3D TIs considered here and the conventiona Fermi iquids is that the critica exponent in both the cases is 1/, which is a characteristic of the mean-fied approach. A microscopic expression of the PI equation(0) corresponding to the interacting surface state of a 3D TI has been derived in terms of the parameters mentioned above. This equation has a few important impications which are markedy different from its SU() invariant counterpart occurring in conventiona Fermi iquid. The PI equation signifies that the Fermi surface instabiity at a particuar anguar momentum channe can appear not ony when the interaction strength in that channe exceeds the critica vaue / but aso when the interaction strength in any one of the ( 1) and (+1) channes exceeds a vaue of. Therefore, athough it is easier to achieve a Pomeranchuk phase transition in th channe when the interaction in the same is sufficienty strong, a combined effect of a the three anguar momentum channes can drive the system into a symmetry broken phase without having interaction strength in any one of the channes arger than the critica vaue, as expained in FIG 3. However, the situation is indeed quite different when one considers the case of PI. In this case, even if the interaction strength in 1 channe exceeds /, the instabiity is forbidden, and once the interaction exceeds in the same channe, the PI appears even if interaction in channe is absent. This is indeed quite remarkabe and is a consequence of theabsenceof 1PIeveninaSOcoupednon-Gaiean invariant Fermi iquid. For a the other anguar momentum channes this is, in genera, not true. For a other channes corresponding to 3 the Pomeranchuk order parameter corresponding to a particuar channe, in genera, depends on the competing interactions in both higher and ower anguar momentum channes. Within the hierarchy assumption proposed here, any competing instabiities can be avoided up to the appearance of ne-

12 1 matic instabiity in anguar momentum channe since the 1 PI is absent. The Pomeranchuk order parameter grows in a second order fashion with critica exponent 1/. Furthermore, it is shown that the phase separation instabiity corresponding to the 0 channe can ony appear for an attractive interaction of not ony the density-density type but aso a spin-charge or a pure spinspin type. In particuar, this is microscopicay manifested by the finding that the interactions in any one or a of the anguar momentum channes corresponding to 0 can drive the system towards instabiity. It is further shown that within the mean-fied theory a hypothetica non-centra interaction, which mimics the quadrupoar interaction, can produce nematic instabiity even with a sufficienty arge but negative vaue of the interaction strength in 1 anguar momentum channe ony. The hierarchy assumption introduced here has its effect in avoiding the ony avaiabe competing PI coming from 3 anguar momentum channe, in absence of 1 PI. The nematic order parameter emerging from the quadrupoar interaction is indeed proportiona to the Pomeranchuk order parameter. Moreover, using a few reaistic forms of e-e interaction it is shown that the PI can emerge from a repusive interaction if and ony if the interactionhas a finite range r 0. However, the or the nematic instabiity can never occur when the interaction is repusive. For repusive interaction, on the other hand, instabiities in a the anguar momentum channes with 3 can appear for finite range interactions. Surprisingy enough, the 4 instabiity is more prone to appear than 3 instabiity owing to the strong SOC. In addition to the PI, the system exhibits another cass of competing Fermi surface instabiity invoving a change in the topoogy of the Fermi surface when the centra e-e interaction is stricty repusive. This is more of a reminiscent of the Lifshitz transition as expained in the section VII. It is further shown that there exist two types of topoogica FS instabiities at the most, viz., the appearance of vacated states at the centre, and a thin she of vacated states within. The topoogica phase transition reported here is driven by eectron-eectron interactionand turns out to be quite genericto the systems which supportthe emergenceofthe PI, andhas asobeen obtained in the case of 3D Gaiean invariant Fermi iquid 4]. Significanty enough, the topoogica FS instabiities appearing here for repusive and centra e-e interaction compete with the 3, and 4 PIs ony. The nematic instabiity, on the other hand, is not competed by any of these topoogica FS instabiities. The interaction induced Lifshitz/topoogica phase transition obtained in thispaperfromthetheoryofhflisstrictyvaidonyfor strong TIs having a singe disk type FS. However, recent findings of the appearance of such a topoogica phase transition in the doped Topoogica Crystaine Insuators strongy suggest that in strong TIs simiar behaviours are expected to show up 48, 49]. On the physica ground, with the increase of eectron number density (by doping), the eectron-eectron interaction strength increases which further is expected to ead to such interaction induced Lifshitz transition. Furthermore, this Lifshitz ike transition must have some intriguing consequences in the surface transport properties which woud be interesting to investigate in the future. The detection of entropy spikes can aso serve as a signature of interaction driven Lifshitz transition 57]. Moreover, signatures of the appearance of an annuar FS, quite simiar to what has been found here, has recenty been discovered in a GaAs quantum we structure 58]. Simiar studies on the surface of 3D TIs are expected to revea such topoogica FS instabiity from the topoogy of a disk to the topoogy of an annuus. ACKNOWLEDGEMENTS Author acknowedges Yonatan Dubi for usefu comments and suggestions. The author woud ike to express his appreciation for the vauae suggestions and criticisms of the anonymous referees in preparing the revised manuscript. Appendix A: Detaied cacuations of Landau Parameters ( dθ π One can cacuate the expression for the projected Landau parameters (1) by mutipying both sides of (11) by ) and integrating from 0 to π, and by using the parameterization (6). Evauating the integras it is easy to find, 1 F ρ(ǫ F ) δ,0 V 4 (1+δ,0)V 1 ] 8 (V 1 +V 1 +V +1 ). (A1) However, there are few facts corresponding to the above equation, viz., 1. when 1, both V 1 and V 1 are non-zero,