arxiv: v1 [hep-ph] 11 Jan 2019

Transcription

1 Prepared for submission to JHEP EFI-19-1 Hunting for Vectorlike Quarks arxiv: v1 [hep-ph 11 Jan 019 adovan Dermíšek 1, Enrico unghi 1 and Seodong Shin,3 1 Physics Department, Indiana University, Bloomington, IN 47405, USA Enrico Fermi Institute, University of Chicago, Chicago, I 60637, USA 3 Department of Physics and IPAP, Yonsei University, Seoul 037, Korea dermisek@indiana.edu, elunghi@indiana.edu, shinseod@indiana.edu Abstract: We analyze decays of vectorlike quarks in extensions of the standard model and a two Higgs doublet model. We identify several typical patterns of branching ratios of the lightest new up-type quark, t 4, and down-type quark, b 4, depending on the structure of Yukawa couplings that mix the vectorlike and standard model quarks we assume only mixing with the third generation and also on their doublet or singlet nature. We find that decays into heavy neutral or charged Higgs bosons, when kinematically open, can easily dominate and even be close to 100%: b 4 Hb at medium to large tan β, t 4 Ht at small tan β and b 4 H ± t, t 4 H ± b at both large and small tan β. The pair production of vectorlike quarks leads to 6t, 4tb, t4b and 6b final states. The decay modes into W, Z and h follow the pattern expected from the Goldstone boson equivalence limit that we generalize to scenarios with all possible couplings. We also discuss in detail the structure of Yukawa couplings required to significantly deviate from the pattern characteristic of the Goldstone boson equivalence limit that can result in essentially arbitrary branching ratios.

2 Contents 1 Introduction 1 Model 3 Parameter space scan and experimental constraints 4 4 esults Decays into heavy Higgses 6 4. Decays into W, Z and h 7 5 Conclusions 13 A Details of the Model 14 A.1 Mass eigenstates 14 A. Couplings of the W boson 16 A.3 Couplings of the Z boson 17 A.4 Couplings of the Higgs bosons 18 A.5 Partial decay widths of vectorlike quarks into the W, Z, h, H and H ± 1 A.6 Off-diagonal couplings in special cases A.7 On ellipses 5 1 Introduction Models with extra Higgs bosons or vectorlike quarks and leptons are among the simplest extensions of the standard model SM. Consequently many strategies have been designed to search for them at collider experiments [1 14. The success of this effort depends on understanding the decay patterns of new Higgs bosons or vectorlike matter. It is often assumed that only one particle or a specific coupling of a new particle is present. However, if two new particles are present, or more than one coupling of a new particle is sizable, the decay patterns can be dramatically altered. We analyze decays of new quarks in extensions of the standard model and a two Higgs doublet model type-ii by vectorlike pairs of new quarks VQ, corresponding to a copy of SM quark SU doublets and singlets and their vectorlike partners. We identify several typical patterns of branching ratios of the lightest new up-type quark, t 4, and down-type quark, b 4, depending on the structure of Yukawa couplings that mix the vectorlike and standard model quarks and also on their doublet or singlet nature. We assume only mixing with the third generation of SM quarks, nevertheless the results can be straightforwardly generalized for cases of mixing with the first or second generation. 1

3 We find that decays into heavy neutral H or charged H ± Higgs bosons, when kinematically open, can easily dominate and even be close to 100%: b 4 Hb at medium to large tan β, t 4 Ht at small tan β and b 4 H ± t, t 4 H ± b at both large and small tan β. Thus, the pair production of vectorlike quarks leads to 6t, 4tb, t4b and 6b final states and similar final states for single production. The SM backgrounds for these final states at large invariant mass are very small and thus searching for these processes could lead to the simultaneous discovery of a new Higgs boson and a new quark. The usual decay modes into W, Z and the SM Higgs boson, h, cluster around the pattern expected from the Goldstone boson equivalence limit GBE, corresponding to sending all vectorlike quark masses to infinity, that we generalize to scenarios with all possible couplings. For singlet-like new quarks this leads to :1:1 branching ratios into W, Z and h. For doublet-like new quarks this leads to a one parameter family of branching ratios characterized by an arbitrary branching ratio to W and equal branching ratios to Z and h. We also discuss in detail the structure of Yukawa couplings required to significantly deviate from the pattern characteristic of Goldstone boson equivalence limit that can result in essentially arbitrary branching ratios. Extensions of the SM, two Higgs doublet models or the minimal supersymmetric model MSSM with vectorlike matter were previously explored in a variety of contexts. Examples include studies of their effects on gauge and Yukawa couplings in the framework of grand unification [15 3 and on electroweak symmetry breaking and the Higgs boson mass [4 6. The supersymmetric extension with a complete vectorlike family provides a very sharp prediction for the weak mixing angle [, 7 and also a possibility to understand the values of all large couplings in the SM from the I fixed point structure of the renormalization group equations [8. In addition, vectorlike fermions are often introduced on purely phenomenological grounds to explain various anomalies. Examples include discrepancies in precision Z-pole observables [9 3 and the muon g- anomaly [33 35 among many others. In this paper we focus on vectorlike quarks with the same quantum numbers as the quarks in the SM. Examples of signatures of related scenarios with vectorlike leptons can be found in efs. [ For related studies and especially for studies of decay modes and signatures in scenarios with different quantum numbers of vectorlike matter see also efs. [41 47 and references therein. This paper is organized as follows. In Sec., we outline the model and assumptions. Details of the analysis and experimental constraints are discussed in Sec. 3. The main results and their discussion are contained in Sec. 4 and we conclude in Sec. 5. The appendix contains details of the model, formulas for couplings and relevant partial widths, and approximate formulas that are useful to understand the results. Model We consider an extension of a two Higgs doublet model by vectorlike pairs of new quarks: SU doublets Q, and SU singlets T, and B,. The quantum numbers of new particles are summarized in table 1. The Q, T and B have the same quantum numbers

4 as the SM quark doublet q and the right-handed quark singlets u and d, respectively. We further assume that quarks couple to the two Higgs doublets as in the type-ii model, namely the down sector couples to H d and the up sector couples to H u. This can be achieved by the Z symmetry specified in table 1. The generalization to the whole vectorlike family of new fermions, including the lepton sector which has been studied in ref. [37, is straightforward. The most general renormalizable agrangian consistent with q i u i d i Q, T, B, H d H u SU U1 Y Z Table 1. Quantum numbers of standard model quarks q i, ui, di for i = 1,, 3, extra vectorlike quarks and the two Higgs doublets. The electric charge is given by Q = T 3 + Y, where T 3 is the weak isospin, which is +1/ for the first component of a doublet and -1/ for the second component. our assumptions contains the following Yukawa and mass terms for the SM and vectorlike quarks: y ij d qi d j H d λ i B q i B H d λ j Q Q d j H d λ Q B H d λh d B Q y ij u q i u j H u κ i T q i T H u κ j Q Q u j H u κ Q T H u κh u T Q M Q Q Q M T T T M B B B + h.c.,.1 where the first term represents the Yukawa couplings of the SM down-type quarks, followed by Yukawa couplings of vectorlike quarks to H d denoted by various λs, Yukawa couplings of the SM up-type quarks, Yukawa couplings of vectorlike quarks to H u denoted by various κs, and finally by mass terms for vectorlike quarks. The components of doublets are labeled as follows: q i = u i d i, Q, = T Q, B Q,, H d = H + d Hu 0 Hd 0, H u = Hu.. We assume that the neutral Higgs components develop real and positive vacuum expectation values, Hu 0 = vu and Hd 0 = vd, as in the CP conserving two Higgs doublet model with vu + vd = v = 174 GeV and we define tan β v u/v d. For simplicity, we further assume that the new quarks mix only with one family of SM quarks and we consider the mixing with the third family as an example; the mixing of new quarks with more than one SM family simultaneously is strongly constrained by various flavor changing processes and we will not pursue this direction here. In the basis in which the SM quark Yukawas are diagonal, the mass matrices describing the mixing between the third generation and the vectorlike quarks are see the appendix: t Q T T t M t T Q = t Q T T y t v u 0 κ T v u t κ Q v u M Q κv u T Q,.3 T 0 κv u M T T 3

5 b BQ B M b b B Q B = b BQ B y b v d 0 λ B v d λ Q v d M Q λv d 0 λvd M B b B Q B..4 Note that the corresponding mass matrices in the case of a single Higgs doublet can be obtained by setting v u = v d = 174 GeV. A complete discussion of the mass eigenstates and of their couplings to the W, Z, and Higgs bosons can be found in the appendix. 3 Parameter space scan and experimental constraints We study the branching ratio patterns that can be obtained in the model by varying the relevant parameters as follows: M Q,T,B [900, 4000GeV, 3.1 κ T, κ Q, κ, κ [ 1.0, 1.0 if mixing exists, 3. λ B, λ Q, λ, λ [ 1.0, 1.0 if mixing exists, 3.3 tan β [0.3, Note that the upper range of the couplings has no impact on presented results as long as it is common for all couplings. We impose the experimental constraints from precision electroweak measurements [48 1, h γγ [50 and direct searches for vectorlike quarks pair produced at the HC [1 3, 54. For the latter constraints, we directly use the data points from hepdata.net [55 obtained from ef. [ where the limits in terms of mass of vectorlike quarks and the branching ratios into W, Z and the SM Higgs boson are similar to the other search results. Note that searches for the single production of VQ via t-channel quark gluon interactions also exist but the experimental constraints are not stronger than those for the pair production of VQ [8, 56 although the production cross section can be larger than that of the pair production for VQ heavier than 800 GeV [43. Hence, we do not consider the corresponding experimental bounds here but leave it to a future work [57. In scenarios where the lightest VQ can decay into heavy neutral or charged Higgs boson, the currently existing searches for VQ, focussed on decay modes into W, Z, and h, do not directly apply. In the model we are considering we expect potentially large contributions to B X s γ. While the charged Higgs loop is suppressed for Higgs masses above 1 TeV, the b W t vertex given in eq. A.6 yields a diagram which is chirally enhanced. Note that this vertex requires couplings to both H u and H d. This diagram yields a contribution to the coefficient of the magnetic moment operator C 7 see, for instance, ref. [58 which is enhanced by a factor mt v uv d m b κ MQ Q λ Q 1 sin β κ 1TeV Qλ Q. This contribution is unacceptably large for MQ moderate tan β and κ Q λ Q See also ef. [49 for a detailed discussion about imposing constraints from precision electroweak measurements in models with vectorlike quarks. Constraints from related experimental results [51 53 are not very different from constraints resulting from ef. [50. 4

6 64π m t4 Γ doublet singlet t 4 W + b λ Q c β κ T s β t 4 Zt κ Q s β κ T s β t 4 ht κ Q s β κ T s β t 4 Ht κ Q c β κ T c β t 4 H + b λ Q s β κ T c β 64π m b4 Γ doublet singlet b 4 W + t κ Q s β λ B c β b 4 Zb λ Q c β λ B c β b 4 hb λ Q c β λ B c β b 4 Hb λ Q s β λ B s β b 4 H + t κ Q c β λ B s β Table. eading dependence of the t 4 and b 4 decay widths on the agrangian parameters in the large mass limit. We use the shorthand notation s β = sin β and c β = cos β. Contributions to other flavor transitions are much smaller because, in absence of chiral enhancements, experiments constrain various combinations of the entries of the effective CKM matrix given in eq. A.19. Any deviation from the SM expectation is related to the non-unitarity of this matrix, which is of order vuκ T /M T or v d λ B /M B, yielding contributions which can be at most 1.5% for vectorlike quarks heavier than 1 TeV. Finally we note that in our framework there are no flavor changing neutral interactions in the Higgs nor Z sector. Therefore, tan β enhanced tree-level Higgs contributions to B d,s l + l, which are present in supersymmetric models with loop-induced nonholomorphic Higgs couplings, are absent. 4 esults The main features of the decays of vectorlike quarks can be understood from the dominant couplings that appear in table, which can be easily read from the approximate expressions presented in the appendix. An important quantity that controls the decay of the lightest vectorlike quark is its doublet or singlet fraction, which add up to unity and, for up-type vectorlike quarks t a a = 4, 5, are defined respectively as d ta = 1 { [V u a3 + [V u a4 + [V u a4 }, 4.1 s ta = 1 { [V u a3 + [V u a5 + [V u a5 }. 4. The doublet and singlet fractions for down-type vectorlike quarks b a a = 4, 5 are obtained by replacement V u, V d,. Note that in the absence of couplings to H u H d, i.e., in the case when all λ s κ s in.1 vanish, the doublet fraction of t 4 b 4 defined in eq. 4.1 is exactly 1 or 0. The latter case corresponds to a vectorlike singlet quark which is completely uncoupled and thus of no interest for our study. An important point is that decays into heavy Higgses can easily dominate when kinematically open. A second point is that, when these decay channels are not open, vectorlike quarks branching ratios into W, Z and h follow a very simple pattern which we summarize in table 3. If only couplings to H u H d are present, the t 4 b 4 branching ratios into W, Z and h are 0, 1/, 1/ and 1/, 1/4, 1/4 for the doublet and singlet case respectively. 5

7 Couplings to H u only Couplings to H d only Couplings to H u and H d doublet singlet doublet singlet doublet singlet Bt 4 W + b 0 1/ x 1/ Bt 4 Zt 1/ 1/4 0 0 x/ 1/4 Bt 4 ht 1/ 1/4 0 0 x/ 1/4 Bb 4 W + t / 1 x 1/ Bb 4 Zb 0 0 1/ 1/4 x/ 1/4 Bb 4 hb 0 0 1/ 1/4 x/ 1/4 Table 3. Vectorlike quark branching ratios in the Goldstone boson equivalence limit. x varies between 0 and 1. If couplings to H u and H d are present simultaneously, the doublet branching ratios into Z and h are still equal but the W channel can be as large as 100%. In the next two subsections we discuss decays into heavy Higgses and then dwell into the intricacies of standard decays into W, Z and h in situations in which subdominant couplings become important. 4.1 Decays into heavy Higgses In figure 1 we present the branching ratios of vectorlike quarks into heavy CP even neutral and charged Higgses that we obtain from the parameter space scan described in section 3. For simplicity, we assume that only one either charged or neutral heavy Higgs decay channel is kinematically open. 3 In all panels orange green points correspond to allowing only couplings to H u H d ; gray points show the additional branching ratio values that we obtain by allowing simultaneous couplings to H u and H d. The interpretation of these plots follows immediately from the couplings presented in table which yield the branching ratios collected in tables 4-6. The channel t 4 Ht is allowed only if couplings to H u are present and is sizable only at small tan β independently of the t 4 doublet fraction. Conversely, the channel b 4 Hb requires couplings to H d and can easily dominate at medium-to-large tan β independently of the b 4 doublet fraction. For the decay t 4 H ± b, couplings to H u H d only, result in a mostly singlet doublet t 4 with large branching ratio at small medium-to-large tan β. Similarly, for the decay b 4 H ± t, couplings to H u H d only, result in a mostly doublet singlet b 4 with large branching ratio at small medium-to-large tan β. These features can be inferred directly from tables and 6. Inspection of the bottom panels of figure 1 reveals also that, in presence of coupling to H d only and at large tan β, the t 4 H + b branching ratio is typically close to unity while the b 4 H t one can be small. This happens because in presence of couplings to H d only, t 4 is either 100% doublet or 100% singlet the doublet fraction in eq. 4.1 is unity but it is possible to have a singlet t 4 by lowering M T while the b 4 doublet fraction varies continuously between 0 and 100%. This implies that the t 4 H + b branching ratio is either close to 100% doublet t 4 or close to 0 singlet t 4, while the b 4 H t one covers 3 Note that the results for the decay mode into CP odd neutral Higgs would be similar. 6

8 Figure 1. Allowed branching ratios of t 4 and b 4 into heavy charged and neutral Higgses. In all panels, orange green points correspond to couplings to H u H d only; gray points are the additional possibilities allowed by a full parameter space scan. smoothly the entire range, with large small branching ratios corresponding to a mostly singlet doublet b 4. The presence of couplings to both H u and H d gray points allows the t 4 H ± b branching ratio to acquire any value by tuning the contributions of the two sets of couplings. 4. Decays into W, Z and h The results of the parameter space scan for decays into W, Z and h are summarized in Fig., where we show branching ratios of the lightest up-type and down-type vectorlike quarks t 4 and b 4. In the top two panels we consider the limiting cases of couplings to H u and H d only for which we consider only the vectrolike quark with non-trivial branching ratios; in fact, in presence of couplings to H u H d only, the lightest vectorlike quark b 4 t 4 decays exclusively to W t, t 4 W b, b Note that, in presence of couplings to H u H d only, the decay mode t 4 b 4W b 4 t 4W is not allowed. Considering the t 4 case the b 4 one is completely analogue this fact can be understood as follows. If the b 4 is a singlet, the absence of mixing in the down sector causes the W t 4b 4 vertex to vanish. If the b 4 is a doublet, in absence of mixing its mass is exactly M Q. The mass matrix for t 4 and t 5 has M Q and 7

9 Bt 4 W + b 0 Bt 4 Zt Bt 4 ht Bt 4 Ht Couplings to H u only Couplings to H d only Couplings to H u and H d doublet singlet doublet singlet doublet singlet t β t β +1 t β t β +1 t β 4t β x + Ot β t β 4t β +1 t β t β 4t β +1 t β 4t β +1 4t β xt β t β 4t β xt β 1 1 t β +1 4t β x + Ot β 1 4t β +1 Bb 4 W + t t β +4 1 x + O1/t β t β Bb 4 Zb 0 0 t β + t β +4 x/t 1 β t β +4 Bb 4 hb t β + 1 Bb 4 Hb 0 0 t β t β + x/t β 1 t β +4 t β +4 t β t β +4 x + O1/t β t β t β +4 Table 4. Branching ratios of the vectorlike quarks t 4 and b 4 into W, Z, h and H as a function of t β = tan β. These expressions are exact for heavy vectorlike masses and in the limit in which the couplings in table dominate. Bt 4 W + b 0 Bt 4 Zt 1/ Bt 4 ht 1/ Couplings to H u only Couplings to H d only Couplings to H u and H d doublet singlet doublet singlet doublet singlet Bt 4 H + b 0 1 4t β +1 Bb 4 W + t t β 4t β +1 t β + 0 t β 4t β x t β 4t β x t β t β + 0 t β t β Bb 4 Zb 0 0 1/ Bb 4 hb 0 0 1/ Bb 4 H + t 1 t β t β +4 1 t β +4 1 t β +4 t β t β +4 1+t β 1 x 1+t β +t β t β 4t β +1 t β t β 1+t β 4t β +1 t β t β 1+t β 4t β +1 t β 1+t β 1 x 1+t β +t β 4t β +1 t β 1+t β x 1+t β 1+t β t β +4 x 1 1+t β t β +4 x 1 1+t β t β +4 1+t β x t β 1+t β 1+t β t β +4 Table 5. Branching ratios of the vectorlike quarks t 4 and b 4 into W, Z, h and H ± as a function of t β = tan β. These expressions are exact for heavy vectorlike masses and in the limit in which the couplings in table dominate. In the bottom panels we consider the general case. In presence of couplings to both H u and H d the decays t 4 b 4 W and b 4 t 4 W can be present. In this case, we rescale M T on the diagonal with some small mixing, implying that m t4 < minm Q, M T. In both cases the decay t 4 b 4W is kinematically not allowed: for doublet t 4 M Q < M T we have m t4 < M Q = m b4 and for singlet t 4 M T < M Q we have m t4 < M T < M Q = m b4. 8

10 Couplings to H u only Couplings to H d only doublet singlet doublet singlet Bt 4 Ht/Bt 4 SM 1/t β 1/4t β 0 0 Bb 4 Hb/Bb 4 SM 0 0 t β / t β /4 Bt 4 H + b/bt 4 SM 0 1/4t β t β / 0 Bb 4 H + t/bb 4 SM 1/t β 0 0 t β /4 Table 6. Branching ratios of the vectorlike quarks t 4, b 4 into H, H ± normalized to the total branching ratio into SM particles W, Z and h as a function of t β = tan β. These expressions are exact for heavy vectorlike masses and in the limit in which the couplings in table dominate. Figure. The allowed branching ratios of the lightest vectorlike quarks in the scenario with couplings to H u only top-left, couplings to H d only top-right, and the general scenario bottom panels. ed: 95% or more singlet-like, purple: 50%-95% singlet-like, cyan: 50%-95% doublet-like, blue: 95% or more doublet-like. the branching ratios by the sums, Bt 4 ht + Bt 4 W b + Bt 4 Zt and Bb 4 hb + Bb 4 W t + Bb 4 Zb, respectively. The rescaled branching ratios into h, W and Z shown in Fig. add to one the values read from the figure are the upper limits for the actual branching ratios. ed, purple, cyan, and blue points correspond to d t4,b 4 < 0.05 s t4,b 4 > 0.95, 0.05 < d t4,b 4 < < s t4,b 4 < 0.95, 0.5 < d t4,b 4 <

11 Figure 3. The left right panel shows the allowed branching ratios of a mostly doublet singlet t4. The green ellipse line is obtained for κq = 0 κt = 0. The red, blue and orange ellipses show three cases of how a κq = 0 κt = 0 point moves towards the Goldstone boson equivalence limit as κq κt is increased. Solid and dashed lines correspond to MT = 1, 4 TeV MQ = 1, 4 TeV. Purple points correspond to a coarse scan of the whole parameter space. For the doublet case, yellow green points correspond to κq MT /vu > 10 1 κq MT /vu < 10 1 ; for the singlet case, /MT v. the shaded green regions show the effect of decreasing κt MQ and dt4,b4 > 0.95, respectively. Note that if the decay modes into heavy Higgses are kinematically open all plots in Fig. correspond to rescaled branching ratios. The main features of these plots can be understood using the approximate formulas presented in eqs. A.77-A.88, which we use to generate the two plots in figure 3. The following very simplified expressions for the h, W and Z couplings for a mostly doublet t4 yield the correct branching ratios up to terms suppressed by κ0 s, λ0 sv/mq,t,b : vu κt κ sin β, MT ' κq sin β, g v vu vd ' κt κ sin β λb λ cot β, MB MQ MT g v ' [ λq cos β, MQ λhtt4 ' 4.3 λht4 t 4.4 gw t4 b W t4 b g gzt4 t ' 0, Zt4 t g v g ' [κq sin β. cos θw MQ 4.8 /M Note that the v/mq factor in the W and Z couplings is compensated by the MQ W,Z 10

12 factors that appear in the formulae for the decay widths, see the discussion below A.88. We start by considering the case of a mostly doublet t 4 with couplings to H u only i.e. all λ s set to zero. In the Goldstone boson equivalence limit we see that only couplings to Z and h are non-zero and that they are controlled by the parameter κ Q, implying Bt 4 Zt = Bt 4 ht = 1/, that corresponds to the point [1/, 0 in the [Bt 4 Zt, Bt 4 W b plane. Corrections to this limiting case are suppressed by at least one mixing parameter κ, κ and by additional powers of the heavy singlet mass M T which for mostly doublet t 4 is larger than M Q. ooking at the subleading contributions in Eqs , we see that the widths into h, W and Z are controlled by two combinations of parameters κ Q sin β and vu M T κ T κ sin β, implying that as κ Q decreases while all other κ s remain sizable, the branching ratios move into an ellipse in the [Bt 4 Zt, Bt 4 W b plane red in the left panel of figure 3 as discussed in appendix A.7. For κ Q = 0 the dominant couplings are λ h tt 4 and g W t 4t = g v λ h tt 4 M Q, which imply Bt 4 W b = 1/3 and Bt 4 ht = /3 See the note at the end of appendix A.6, corresponding to the point [0, 1/3 in the figure the end point of the red ellipse. Including all the subleading dependence on the κ s i.e. terms suppressed by at least two powers of the heavy vectorlike mass M T one obtains the formulas in eqs. A.77-A.88. For κ Q similar in size to v u /M T κ κ a scan deviates mildly from the red ellipse yellow points; if κ Q is allowed to be very small, the subleading terms mentioned above green points fill completely the green ellipse. For κ Q = 0 the six couplings depend on the two combinations v u /M T κ T κ sin β and v u M Q /MT κ T κ sin β, implying that the branching ratios lie exactly on the green ellipse. 5 In particular, note that unless κ κm Q /M T only the region of the ellipse near the point 0, 1/3 is allowed; this is reflected in the high density of points near this limit at small κ Q. This feature can be seen also in the full scan presented in the upper panels of figure, in which the right-hand side of the ellipse is sparsely populated. Finally, allowing also couplings to H d all branching ratios on the segment between [1/, 0 and [0, 1 are allowed. In fact, branching ratios into h and Z are controlled by κ Q and are identical in the limit in which all other κ s are neglected while the branching ratio into W is controlled by λ Q. A limited parameter space scan yields points purple clustered around this line. If only κ Q λ Q is present the points cluster around [1/, 0 [0, 1. On the other hand, if κ Q = λ Q = 0, there are only sizable contributions to λ h tt 4 and g W t 4b see eqs. 4.3 and 4.5 which are now controlled by two independent combinations of parameters, implying that decay into Z is suppressed and the remaining two channels can have any branching ratios. This can be seen by the purple points clustered on the vertical line corresponding to Bt 4 Zt = 0 in figure 3. In the high-density scan presented in 5 Starting from a point on the ellipse one quickly moves towards the GBE point [1/, 0 as κ Q increases. The resulting path in the plane for three representative points is described by the red, blue and orange ellipses. The minor axes of the ellipses decreases for heavier vectorlike quark masses. Since the points on the right side of ellipses are considerably fine-tuned, as soon as we change the vectorlike quark mass without changing the other parameters, the points move towards the left-hand side as is the case for the orange curves. 11

13 the bottom two panels of figure we see that every branching ratio is possible, but that the region in between the two purple lines in figure 3 is significantly more populated. Note that, if there are couplings only to H d, t 4 decays exclusively into the channels W b and W b 4. In the case of a mostly singlet t 4 with couplings to H u only i.e. all λ s set to zero the situation is very different. Keeping only the leading terms in 1/M Q,T,B for each coupling we obtain the following very simplified expressions for the h, W and Z couplings for a mostly singlet t 4 : λ h tt 4 κ T sin β, 4.9 λ h v u t 4 t sin β κκ Q, M Q 4.10 g W t 4b g v [κ T sin β, M T 4.11 g W t 4b g v g Zt 4t g Zt 4t M T [ g v cos θ W g cos θ W v M T κ v dm T M Q λ Q sin β, 4.1 [κ T sin β, 4.13 M T [ κκ Q sin β v um T M Q In the GBE the h, W and Z couplings are controlled by the parameter κ T, implying Bt 4 ht = Bt 4 Zt = 1 Bt 4 W b = 1/4, that corresponds to the point [1/4, 1/ in the [Bt 4 Zt, Bt 4 W b plane. Corrections to this limiting case are suppressed by at least one mixing parameter κ, κ and by additional powers of the heavy doublet mass M Q which for mostly singlet t 4 is much larger than M T. As κ T decreases below v/m Q, the largest coupling becomes λ h t 4 t, implying that the branching ratio into h increases and we move along a line towards the origin in the [Bt 4 Zt, Bt 4 W b plane see the red line in the right panel of figure 3. As κ T further decreases below vm T /MQ, the right-handed Z coupling becomes sizable and a large t 4 Zt branching ratio becomes possible see green shaded triangle in the right panel of figure 3. For κ T = 0, the W channel disappears and the points collapse on the line at Bt 4 W b = 0 green line in the right panel of figure 3; note that points with large Bt 4 Zt require some degree of fine tuning. 6 Finally, allowing also couplings to H d, we see that the t 4 W b branching ratio becomes effectively independent of the others for κ T < vm T /MQ as we see from eq Under these conditions we start moving along a line towards the point [0,1 in the [Bt 4 Zt, Bt 4 W b plane see purple points in the right panel of figure 3. 6 Starting from a point on the κ T = 0 line one quickly moves towards the GBE point [1/4, 1/ as κ T increases. The resulting path in the plane for three representative points is described by the red, blue and orange ellipses. The minor axes of the ellipses decreases for heavier vectorlike quark masses. Since the points at large Bt 4 Zt are considerably fine-tuned, as soon as we change the vectorlike quark mass without changing the other parameters, the points move considerably, as is the case for the orange and blue curves. 1

14 The features described here help understanding the results of the full scan presented in the figure. Note that the b 4 case is almost identical to the t 4 one, the only difference being the replacement of the top Yukawa and mass with the bottom ones. 5 Conclusions We have analyzed decays of new quarks in extensions of the standard model and a two Higgs doublet model type-ii by vectorlike pairs of new quarks, corresponding to a copy of SM quark SU doublets and singlets and their vectorlike partners. We assumed only mixing with the third generation of SM quarks, nevertheless the results can be straightforwardly generalized for cases of mixing with the first or second generation. We have identified several typical patterns of branching ratios of the lightest new uptype quark, t 4, and down-type quark, b 4, depending on the structure of Yukawa couplings that mix the vectorlike and standard model quarks and also on their doublet or singlet nature. Among the most striking results is the finding that decays into heavy neutral or charged Higgs bosons, when kinematically open, can easily dominate and even be close to 100%: b 4 Hb at medium to large tan β, t 4 Ht at small tan β and b 4 H ± t, t 4 H ± b at both large and small tan β. We found that the conventional decay modes into W, Z and the SM Higgs boson, h, follow the pattern expected from the Goldstone boson equivalence limit that we have generalized to scenarios with all possible couplings. For doublet-like new quarks this leads to a one parameter family of branching ratios characterized by an arbitrary branching ratio to W and equal branching ratios to Z and h. We have also discussed in very detail the structure of Yukawa couplings required to significantly deviate from the pattern characteristic of Goldstone boson equivalence limit that can result in essentially arbitrary branching ratios. For vectorlike quark masses within the reach of the HC and near future colliders this does not require very special choices of model parameters. We found that large deviations from the GBE are possible for κ Q, λ Q < v/m T and κ T < vm T /MQ for doublet-like and singlet-like t 4, respectively similar relations hold for b 4 with M T M B and κ T λ B. The new decay modes of vectorlike quarks through heavy Higgs bosons and the fact that they easily dominate imply that the usual search strategies are not sufficient and the current exclusion limits might not necessarily apply. Among the smoking gun signatures are the 6t, 4tb, t4b and 6b final states resulting from the pair production of vectorlike quarks and similar final states for single production. The SM backgrounds for these final states at large invariant mass are very small and thus searching for these processes could lead to the simultaneous discovery of a new Higgs boson and a new quark. Acknowledgments SS thanks Bogdan Dobrescu and Yuval Grossman for insightful discussion. The work of D was supported in part by the U.S. Department of Energy under grant number DE- SC This work was performed in part at the Aspen Center for Physics, which 13

15 is supported by National Science Foundation grant PHY SS appreciates the hospitality of Fermi National Accelerator aboratory. SS is supported by the National esearch Foundation of Korea NF-0171D1A1B A Details of the Model A.1 Mass eigenstates From the agrangian in eq..1 we read the 5 5 mass matrices in the up and down sectors in the gauge eigenstate basis: Q ū i T T yu ij v u 0 κ i T v u u j κ j Q v u M Q κv u T Q, A.1 0 κv u M T T di BQ B y ij d v d 0 λ i B v d d j λ j Q v d M Q λv d B Q, A. 0 λvd M B B where i, j = 1,, 3. It is convenient to define the following vectors of flavor eigenstates: u a = u 1, u, u 3, T Q, T and d a = d 1, d, d 3, B Q, B, and similarly for u a and d a. We then use four unitary matrices to diagonalize the Yukawa matrices y ij u The resulting up and down mass matrices are: yu diag v u 0 U il κ l T v u κ l Q U lj v u M Q κv u 0 κv u M T y diag d v d 0 D il λ l B v d λ l Q D lj v d M Q λv d 0 λvd M B and y ij d : yu diag = U y uu, A.3 y diag d = D y ud. A.4 m u m c y t v u 0 κ T v u 0 0 κ Q v u M Q κv u κv u M T m d m s y b v d 0 λ B v d 0 0 λ Q v d M Q λv d λvd M B m u m c M t, A.5 m d m s 0, 0 0 M b A.6 where the requirement of mixing with the third generation only has been imposed by setting U il κ l T = κ T δ i3, κ l Q U lj = κ Q δ j3, D il λ l B = λ Bδ i3 and λ l Q D lj = λ Q δ j3. The basis vectors corresponding to the lower-right 3 3 block are t, T Q, T and b, B Q, B and similarly for right-handed components, see eqs..3 and.4. 14

16 Finally we diagonalize the 3 3 mass matrices M t and M b : m t 0 0 V u M tv u = 0 m t4 0, 0 0 m t5 V d M bv d = m b m b m b5. A.7 A.8 The vectors û a and ˆd a of mass eigenstates and similarly for right-handed fields are explicitly given by: û i U u i û a = ˆt 4 = 0 3 V ˆt u T Q, A T ˆd a = ˆd i ˆb4 ˆb5 = V d D d i B Q B. A.10 It is useful to have approximate expressions in the limit κ T v u, κ Q v u, κv u, κv u M Q, M T, M Q M T and λ B v d, λ Q v d, λv d, λv d M Q, M B, M Q M B. If the lightest vectorlike quarks t 4, b 4 are mostly SU doublets, we find: V u = V u = V d = v u v u v d 1 v u κ T MT κ T κm Q y tκ Q M T M Q M T v u κ T M Q κm Q + κm T M T M Q 1 v u v u κ T M T v u M Q κ+m T κ M T M Q 1 v u κ Q M Q v u κ Q M Q κ Q κm T y tκ T M Q M Q M T 1 v d λ B MB λ B λmq y b λ Q M B M Q M B v d 1 v u ytκ Q M Q v u κ T M T M Q κ+m T κ M MT M v Q κ+m T κ Q u MT M Q κ T v u κ Q M Q 1 v u v u M T v u M Q κ+m T κ M T M Q κ Q M Q κ+m T κ M T MT M + ytκ Q T MT κ Q v M Q κ+m T κ M MQ u MT M v Q κ+m T κ Q u v u M Q κ+m T κ M T M Q λ B λm Q + λm B M Q MB M y bλ Q Q MQ 1 v d v d λ B M B v d M Q λ+mb λ M B M Q 1 v u MT M Q M Q κ+m T κ MT M Q v d λ B M B M Q λ+mb λ M MB M v Q λ+mb λ Q d MB M Q λ B 1 v d M B v d, A.11 M Q λ+mb λ M B M Q, A.1, A.13 15

17 V d = v d 1 v d λ Q M Q v d λ Q M Q λ Q λmb y b λ B M Q M Q M B 1 v d v d λ Q M Q λ Q v M Q λ+m B λ MQ d v d M Q λ+m B λ M B M Q v d λ Q M Q λ+mb λ M B MB M + y bλ B Q MB M MB M v Q λ+m B λ Q d MB M Q 1 v d M Q λ+m B λ M B M Q, A.14 up to corrections of Oɛ 3 where ɛ = κ Q, κ T, κ, κv u /M Q, M T or λ Q, λ B, λ, λv d /M Q, M B. The corresponding formulas for the lightest vectorlike quarks being mostly SU singlets can be obtained by swapping the second and third column in each mixing matrix. Note that in the numerical analysis we use the exact expressions. A. Couplings of the W boson The agrangian in eq..1 yields the following W boson interactions for left-handed and right-handed fermions: T W = g ū i d i T Q /W B Q + h.c. A.15 T B T = g û i ˆVCKM ˆdi ˆt 4 /W 0 3 V ˆt u V d ˆb4 + h.c ˆb5 A.16 W = g = g û i T Q T û i ˆt 4 ˆt 5 T T /W /W V u d i B Q B + h.c V d A.17 ˆdi ˆb4 + h.c.. ˆb5 A.18 Note that the matrix ˆV CKM U D that appears in eq. A.16 is not the standard CKM matrix. In fact, the matrix that controls the SM quark interactions with the W boson is given by: V CKM = ˆV td V u 33 ˆV ud ˆVus ˆVub V d 33 ˆV cd ˆVcs ˆVcb V d 33 ˆV ts V u 33 ˆV tb V u 33 V d 33 + V u 34 V d 43 ˆV ud ˆVus ˆVub V d 33 ˆV cd ˆVcs ˆVcb V d 33 ˆV ts V u 33 ˆV tb V u 33 V d 33 ˆV td V u 33, A.19 where in the last step we used the fact that V u 33 1 v uκ T /M T, V d 33 1 v d λ B /M B, while V u 34 V d 43 Oɛ 4 and thus negligible. 16

18 From eqs. A.16 and A.18 we can immediately read out the complete set of left and right handed W couplings: where W = ûa γ µ g W uad b ˆd b + û a γ µ g W uad b ˆd b W + µ + h.c., A.0 g W u id j = g ˆV CKM ij for i, j = 1,, A.1 g W u id k = g ˆV CKM i3 V d 3k for i = 1, & k = 3, 4, 5, A. g W u kd i = g ˆV CKM 3i V u k3 for i = 1, & k = 3, 4, 5, A.3 g W u kd l = g [ ˆV CKM 33 V u k3v d 3l + V u k4v d 4l for k, l = 3, 4, 5, A.4 g W uad b = 0 for a = 1, or b = 1,, A.5 g W u kd l = g V u k4v d 4l for k, l = 3, 4, 5. A.6 A.3 Couplings of the Z boson The couplings of the Z boson to the quarks can be obtained from the kinetic terms: kin ū a i /D a u a + ū a i /D a u a + d a i /D a d a + d a i /D a d a, where the covariant derivative is g D µa = µ i T 3 cos θ a sin θ W Q Z µ, W A.7 A.8 and the weak isospin T 3 a is +1/ -1/ for up down components of SU doublets and 0 for singlets. Following the same steps as in the previous section with the simplification that flavor changing couplings are absent after the rotations in eqs. A.3 and A.4 we obtain the following Z interactions: Z = fa γ µ g Zfaf b f b + f a γ µ g Zfaf b f b Z µ, A.9 where f a = û a or ˆd a and g Zd id j g = 1 cos θ + 13 sin θ W δ ij, W g Zd kd l g = cos θ W g Zd id j = 1 g sin θ W δ ij, 3 cos θ W [ g Zd kd l g 1 = cos θ W 3 sin θ W δ kl 1 V d k4v d 4l g Zu iu j g 1 = cos θ W 3 sin θ W δ ij, [ sin θ W δ kl + 1 V d k5v d 5l g Zd 3d j = 0 for i, j = 1,, A.30 for k, l = 3, 4, 5, A.31 g Zd 3d j = 0 for i, j = 1,, A.3 for k, l = 3, 4, 5, A.33 g Zu 3u j = 0 for i, j = 1,, A.34 17

19 [ g Zu ku l g 1 = cos θ W 3 sin θ W g Zu iu j = g sin θ W δ ij, 3 cos θ W g Zu ku l g = cos θ W δ kl 1 V u k5v u 5l for k, l = 3, 4, 5, A.35 g Zu 3u j = 0 for i, j = 1,, A.36 [ 3 sin θ W δ kl + 1 V u k4v u 4l for k, l = 3, 4, 5. As expected there are no FCNC among the SM quark generations. A.4 Couplings of the Higgs bosons A.37 The couplings of neutral Higgs bosons to up and down quarks follow from the Yukawa interactions in the agrangian. The couplings of the first two generations to neutral Higgs bosons are not affected by mixing with the vectorlike fermions. In the basis in which the SM quark Yukawas are diagonal and keeping only the heaviest SM quarks and the vectorlike fermions we have: H 0 = u,d t Q T T b BQ B y t Hu 0 0 κ T Hu 0 κ Q Hu 0 0 κhu 0 0 κhu 0 0 y b Hd 0 0 λ B Hd 0 λ Q Hd 0 0 λhd λh d 0 t T Q T b B Q B + h.c.. A.38 The CP even h and H, CP odd A and Goldstone boson G mass eigenstates are: H cos α sin α eh 0 = d v d h sin α cos α eh 0, A.39 u v u G cos β sin β ImH 0 = d A sin β cos β ImH 0. A.40 u We require that in the CP conserving two Higgs doublet model we consider, the light Higgs h couplings to gauge bosons are identical to those in the SM. This implies that the heavy CP even Higgs H has no couplings to gauge bosons. In this limit, α = β π/ and the mass eigenstates h and H read: h cos β sin β = H sin β cos β eh 0 d v d eh 0 u v u In terms of the mass eigenstates the agrangian for h and H reads:. A.41 h,h = 1 û V u Y uv u û h sin β H cos β 1 ˆd V d Y dv d ˆd h cos β + H sin β + h.c., A.4 18

20 where the 3 3 matrices Y u and Y d are given by y t 0 κ T y b 0 λ B Y u = κ Q 0 κ and Y d = λ Q 0 λ. 0 κ 0 0 λ 0 A.43 Since these matrices are not proportional to the corresponding 3 3 minors in the mass matrices given in eqs. A.5 and A.6, the Higgs couplings are in general flavor violating. The resulting agrangian is: where h,h = 1 ûa λ h u au b û b h 1 ˆda λ h d ad b ˆdb h A.44 1 ûa λ H u au b û b H 1 ˆda λ H d ad b ˆdb H + h.c., A.45 λ h u au b = sin β V u Y uv u ab, A.46 λ h d ad b = cos β V d Y dv d ab, A.47 λ H u au b = cos β V u Y uv u ab, A.48 λ H d ad b = sin β V d Y dv d ab. A.49 Since Y u v u = M t diag0, M Q, M T, the Higgs boson couplings in the up and down sectors can be written as: m t λ h u au b v = 0 m t4 0 V u 0 M Q 0 V u, A m t5 0 0 M T m t λ H u au b v tan β = 0 m t4 0 V u 0 M Q 0 V u, A m t5 0 0 M T m b λ h d ad b v = 0 m b4 0 V d 0 M Q 0 V d, A m b5 0 0 M B m b λ H v d ad b tan β = 0 m b4 0 V d 0 M Q 0 V d, A m b5 0 0 M B where we used v u = v sin β and v d = v cos β. These expressions show explicitly that in the absence of vectorlike fermions the lightest Higgs h couplings are SM-like, while the heavier scalar Higgs H couplings in the up down sector are suppressed enhanced by tan β. The agrangian for the CP odd Higgs A reads: A = i û V u Y A u V u û A cos β 19

21 where the 3 3 matrices Yu A and Yd A y t 0 κ T Yu A = κ Q 0 κ 0 κ 0 i ˆd V d Y d A V d ˆd A sin β + h.c., are given by and Y A d = y b 0 λ B λ Q 0 λ 0 λ 0. A.54 A.55 Thus we get: A = 1 ûa λ A u au b û b A 1 ˆda λ A d ad b ˆdb A + h.c., A.56 where λ A u au b = i cos β V u Y A u V u ab, A.57 λ A d ad b = i sin β V d Y d A V d ab. A.58 Note that in the limit in which there are no CP violating phases, the left and right components of the diagonal pseudoscalar interactions combine to yield a single coupling proportional to γ 5. Charged Higgs interactions involving SM quarks are controlled by the CKM and can be obtained following the procedure detailed in section A.. We focus only on interactions involving the third generation of SM quarks and the heavy vectorlike fermions for which CKM effects are negligible. In the basis in which the SM quark Yukawas are diagonal and keeping only the heaviest SM quarks and the vectorlike fermions we have: H ± = t Q T T y b H + d 0 λ B H + d u,d λ Q H + d 0 λh + d 0 κh u + 0 b BQ B y t Hu 0 κ T Hu κ Q Hu 0 κhu 0 λh d 0 b B Q B t T Q T + h.c., A.59 where we adopt the conventional notation H d H+ d and H u + Hu. The charged Higgs H ± and Goldstone bosons G ± mass eigenstates are given by: G ± cos β sin β H ± H ± = d sin β cos β H u ±. A.60 In terms of the mass eigenstates the agrangian for H ± reads: H ± = 1 û V u Y H± d V d ˆd H + 1 ˆd V d Y H± u V u û H + h.c., A.61 0

22 where the 3 3 matrices Yu H± and Yd H± are given by y t 0 κ T y b 0 λ B Yu H± = cos β κ Q 0 κ and Yd H± = sin β λ Q 0 λ.a.6 0 λ tan β 0 0 κ/ tan β 0 Thus we get: where H ± = 1 ûa λ H± u ad b ˆdb H + 1 ˆda λ H± d au b û b H + h.c., A.63 λ H± u ad b = V u Y H± d V d ab, A.64 λ H± d au b = V d Y H± u V u ab. A.65 Explicit expressions for the couplings relevant to our analysis are: [ λ H± t 4 b = sin β y b V u 43 V d 33 + λ B V u 43 V d 53 + λ Q V u 44 V d 33 + λv u 44 V d 53 + cos β κv u 45 V d 43, A.66 λ H± bt 4 = cos β [y t V d 33 V u 34 + κ T V d 33 V u 54 + κ Q V d 34 V u 34 + κv d 34 V u 54 sin β λv d 35 V u 44, A.67 λ H± b 4 t = cos β [y t V d 43 V u 33 + κ T V d 43 V u 53 + κ Q V d 44 V u 33 + κv d 44 V u 53 sin β λv d 45 V u 43, A.68 λ H± tb 4 = sin β [y b V u 33 V d 34 + λ B V u 33 V d 54 + λ Q V u 34 V d 34 + λv u 34 V d 54 + cos β κv u 35 V d 44. A.69 A.5 Partial decay widths of vectorlike quarks into the W, Z, h, H and H ± In this section, we present explicit expressions for the partial widths of the vectorlike quarks. The partial width for t i W b j for i, j = 3, 4, 5, if kinematically allowed, is given by: Γt i W b j = m t i λ 1, M W 3π m, m b j t i m t i { g W t i b j + g W t ib j 1 + m t i m b j + m b j MW m + m4 b j t i 6 g W t ib j g W t ib j + g W t ib j M W g W t ib j m bj m ti } M W m t i. A.70 The decay width for b i W t j can be obtained by replacing t i b i and b j t j. Similarly, the partial width for t i Zt j for i, j = 3, 4, 5 is given by: Γt i Zt j = m t i λ 1, M Z 3π m, m t j t i m t i 1

23 { g Zt i t j + 6 g Zt it j g Zt it j 1 + m t i m t j + m t j MZ m + m4 t j t i g Zt it j + g Zt it j M Z g Zt it j m t m t4 } M Z m t i. A.71 The decay width for b i Zb j can be obtained by replacing t i b i and t j b j. The partial width for t i ht j for i, j = 3, 4, 5 is given by: Γt i ht j = m t i 64π λ 1, m h m t i, m t j m t i + λ h t i t j λ h t j t i + λ h t i t j λ h t j t i m tj m ti { λ h t i t j + λ h tj t i 1 + m t j m h } m t i. A.7 The decay width for b i hb j can be obtained by replacing t i b i and t j b j. The formulas presented above are consistent with those in ref. [59. Finally, the decay widths into heavy neutral and charged Higgs bosons are given by: Γt i Ht j = m { λ t i λ 1, m H 64π m, m t j H λ t i m t i t j + H tj t i 1 + m t j m H t i m t i + λ H t i t j λ H t j t i + λ H t i t j λ H t j t i } m tj, A.73 m ti Γt i H ± b j = m { λ t i λ 1, m H ± 64π m, m b j H ± λ t i m t i b j + H ± b j t i 1 + m b j m H ± t i m t i + λ H± t i b j λ H± b j t i + λ H± t i b j λ H± b j t i } m bj. A.74 m ti A.6 Off-diagonal couplings in special cases In this subsection, we present approximated formulas for the off-diagonal Yukawa and gauge couplings between a vectorlike heavy quark t j where j = 4, 5 and the third generation SM quarks in special cases where t j is almost a singlet or a doublet. We assume that all κ s and λ s are small parameters of the same order of magnitude. The mixing matrices given in eqs. A.11 A.14 depend on the four combinations: v u ɛ u Q,T κ Q, κ T, κ, κ, M Q,T ɛ d Q,B v d λ Q, λ B, λ, M λ, Q,B A.75 A.76 where ɛ u,d Q ɛu,d T,B ɛu,d Q ɛu,d T,B for a SU doublet singlet vectorlike quark this is simply due to the fact that m t4,b 4 M Q M T for a SU doublet and m t4,b 4 M T M Q for a SU singlet. We remind the reader that eqs. A.11 A.14 have been derived under the assumption that the lightest vectorlike quark is a doublet; the singlet case is obtained by swapping the second and third column in each matrix.

24 For almost doublet t j ɛ u Q ɛu T the off-diagonal Yukawa and gauge couplings read: λ h tt j = M Q v V u 34 V u 4j M T v V u 35 V u 5j y t κ Q v u sin β κ T κ κ T κm Q + κ T κm T M Q M T MT M Q M Q M T [ v u M T κ T κ sin β + v u M Q y t κ Q sin β v um Q λ h t j t = M Q v V u j4 V u 43 M T v V u j5 V u 53 κ Q sin β + vu κm Q + κm T sin β MT M Q M Q M T g W t jb g = g v u M T κq κ M Q y tκ T M T [ κκq κ κ Q sin β + vu sin β + κ κ Q y t κκ T M Q M T MT [V CKM 33 V u j3 V d 33 + V u j4 V d 43 { κ T M Q M Q M T g v M Q [ } κm Q + κm T MT M Q y tκ Q MQ + g vd κ T κ sin β, A.77 y t κκ T M Q MT 3, A.78 λ B λ M Q M B y bλ Q M Q v u κ T κ sin β v u y t κ Q sin β + v d y b λ Q cot β M T M Q M Q v d v u M Q λ B λ cot β + κκ T sin β M B g W t jb = g V u j4 V d 43 M T M Q M T g v [ λ Q cos β, M Q g Zt jt g = V u j5 cos θ V u 53 W g κm Q + κm T κ T v u cos θ W MT M Q v u M T [ g v v u M Q cos θ W M Q MT M Q M T g Zt jt = M Q M T g cos θ W V u j4 V u 43 g v [κ Q sin β, cos θ W M Q, A.79 κκ T sin β + v umq MT 3 κκ T sin β A.80, A.81 up to Oɛ u Q 3. For almost singlet t j ɛ u T ɛu Q the off-diagonal Yukawa and gauge couplings read: λ h tt j = M Q v V u 34 V u 4j M T v V u 35 V u 5j A.8 3

25 κ T sin β vu κt κ sin β M T M Q [ M T κ T sin β + v u sin β y tκ Q κm Q + κm T M Q MT M Q λ h t j t = M Q v V u j4 V u 43 M T v V u j5 V u 53 κ Q κm Q + κ Q κm T v u sin β MT M Q + y tκ T M T M T M Q [ g W t jb g = [ g M T M Q v u sin β y t κ T M T κκ T κ + κ T κ y t κκ Q M Q M T MQ y tκ Q κm T MQ 3 κκ Q κκ QM T M Q M Q κ Q κ M Q [V CKM 33 V u j3 V d 33 + V u j4 V d 43 κ T v u + v u M T g v [κ T sin β, M T κm Q + κm T MT M Q vd g W t jb = g V u j4 V d 43 g κm Q + κm T λ Q v u MT M Q v d M Q [ g v g Zt jt g Zt jt = M T M Q M T κ v dm T M Q + κ v dm T M 3 Q λ Q sin β, A.83, A.84 λ B λ M Q M B y bλ Q M Q g v [κ T sin β, cos θ W M T g = V u j5 cos θ V u 53 W g V u j4 cos θ V u 43 W g κm Q + κm T v u cos θ W MT M Q v uκ Q M Q [ g v cos θ W M T κκ Q sin β v um T M Q + κκ Q sin β v um T M 3 Q A.85, A.86 A.87, A.88 up to Oɛ u T 3. Couplings of b j are obtained by replacing κ s λ s, y t y b, and β β + π/. Note that in the case of a singlet heavy vectorlike quark in absence of any mixing with a doublet, there are only left-handed couplings to W, Z and h and we have g W g v M λh and g Z g v cos θ W M λh. Then eqs. A.70-A.7 directly lead to Γ W / = Γ Z = Γ h = M/64π λ h, implying that the W, Z and h branching ratios are 50%, 5% and 5%. In the case of a heavy SU doublet vectorlike quark in absence of any mixing with a singlet, there are two relevant parameters which control the couplings to W, Z and h. While the latter two are related as in the singlet case, g W is independent; implying that the branching ratio into W is arbitrary while the Z and h channels are identical. 4

26 A.7 On ellipses Here we show that if a vectorlike quark decays into three channels Z, W, h whose effective couplings are linear combinations of two fundamental parameters a and b, any two branching ratios lie on an ellipse. By assumption the widths Γ W,Z,h are linear combinations of a, b and ab. A generic ellipse in the [B Z, B W plane is given by the equation: 0 = ɛ 1 B Z + ɛ B W + ɛ 3 B Z B W + ɛ 4 B Z + ɛ 5 B W + 1 A.89 = linear combination of a4, b 4, a b, a 3 b and ab 3 Γ Z + Γ W + Γ h, A.90 which admits a unique solution for the five coefficients ɛ i. eferences [1 M. Aaboud et al. [ATAS Collaboration, arxiv: [hep-ex. [ M. Aaboud et al. [ATAS Collaboration, JHEP 1710, [arxiv: [hep-ex. [3 M. Aaboud et al. [ATAS Collaboration, Phys. ev. ett. 11, no. 1, [arxiv: [hep-ex. [4 A. M. Sirunyan et al. [CMS Collaboration, JHEP 1806, [arxiv: [hep-ex. [5 A. M. Sirunyan et al. [CMS Collaboration, Phys. ett. B 779, [arxiv: [hep-ex. [6 A. M. Sirunyan et al. [CMS Collaboration, JHEP 1808, [arxiv: [hep-ex. [7 M. Aaboud et al. [ATAS Collaboration, arxiv: [hep-ex. [8 The ATAS collaboration [ATAS Collaboration, ATAS-CONF [9 A. M. Sirunyan et al. [CMS Collaboration, Phys. ett. B 781, [arxiv: [hep-ex. [10 A. M. Sirunyan et al. [CMS Collaboration, arxiv: [hep-ex. [11 A. M. Sirunyan et al. [CMS Collaboration, arxiv: [hep-ex. [1 M. Aaboud et al. [ATAS Collaboration, JHEP 1801, [arxiv: [hep-ex. [13 M. Aaboud et al. [ATAS Collaboration, JHEP 1809, [arxiv: [hep-ex. [14 A. M. Sirunyan et al. [CMS Collaboration, JHEP 1809, [arxiv: [hep-ex. [15 K. S. Babu and J. C. Pati, Phys. ett. B 384, [hep-ph/ [16 C. F. Kolda and J. March-ussell, Phys. ev. D 55, [hep-ph/ [17 D. Ghilencea, M. anzagorta and G. G. oss, Phys. ett. B 415, [hep-ph/