1 Introduction

With the discovery of the Higgs particle at the LHC in 2012, a very important revolution was marked in the field of particle physics [1]. This discovery, apart from providing the missing piece to the Standard Model (SM), paved the way for a better understanding of the fundamental particles and through Higgs mechanism the process responsible for generating the masses of quarks and leptons could be explained. But this grand success was not devoid of shortcomings. There are many challenging issues in cosmology, astrophysics, particle physics etc. which are still not addressed and the answers to these phenomena are yet to be found. Some of these prominent challenges are neutrino oscillations and generation of their tiny masses, lepton number violating processes (LNV), baryon asymmetry of the universe (BAU), dark matter, dark energy etc. The Standard Model of particle physics, recognised as one of the most successful theories in the field, fails to provide a proper explanation of these challenges. As a result, the concept of beyond the Standard Model (BSM) frameworks have come into the picture. And people believe that a proper understanding with a suitable explanation of these challenging issues can be found in this BSM paradigm.

One of the prime ingredients that motivates physicists to go for BSM theories is associated with mass of the neutrinos. With the discovery of neutrino oscillation [2], it became obvious that neutrinos should have tiny mass and mixing between them. The experimental findings made by Super-Kamiokande [3, 4] and Sudbury Neutrino Observatories [5] gave the final confirmation to this idea. But due to the absence of right-handed neutrinos it remained massless within the Standard Model. As a result, to generate this tiny mass of neutrinos an extension of SM was inevitable. Consequently a new theory that could incorporate and provide an elegant explanation to these questions came in the form of Seesaw mechanism. A detailed review about seesaw mechanism can be found in the literature [6,7,8]. This mechanism is broadly classified into Type I [9,10,11], Type II [12], Type III [13], Inverse seesaw etc. In recent times, Inverse seesaw has become more popular in model building. In this framework the mass of right-handed neutrinos can be brought down to TeV scale. This peculiar property enhances the possibility of detecting these particles at the LHC and future experiments. For a detailed study on Inverse seesaw one can refer to [14,15,16].

It is now an established fact that there is an asymmetry between matter and anti-matter in the universe. Cosmological observations indicate that the number of baryons in the universe is unequal to the number of anti-baryons. This difference in number between baryons and anti-baryons is termed as Baryon Asymmetry of the Universe (BAU) [17,18,19]. At the beginning, as evident from various considerations, the universe started with an equal number of both the types of particles i.e. baryons and anti-baryons. As such the asymmetry observed in the universe occured in much later times and must have been generated through a dynamical process called baryogenesis. As per Planck data, the value of this asymmetry, denoted by \(\eta _{B}\) is found to be [20]

$$\begin{aligned} \eta _{B}=(6.04\pm 0.08)\times 10^{-10} \end{aligned}$$
(1)

As proposed by Sakharov, three important conditions are necessary for baryogenesis: Baryon number violation (B), C and CP violation, interactions out of thermal equilibrium [21]. In the last couple of decades several attempts have been made to address the phenomena of BAU. One of the popular and successful theoretical process is baryogenesis via leptogenesis. This mechanism was first proposed by Fukugita and Yanagida [22]. According to this mechanism, the L-violating out of equilibrium decays of singlet neutrino creates an asymmetry in the leptonic sector. This excess in lepton number can be converted into the observed baryon asymmetry through B+L violating sphaleron processes [23, 24]. In this regard, Inverse seesaw contains gauge singlet right-handed neutrinos and sterile fermions. As a result, the asymmetry generated by decay of one of the quasi-Dirac pairs formed by these particles can be converted into baryon asymmetry of the universe.

The presence of dark matter (DM), an inevitable mystery of the universe, has been well-established through various observations in astrophysics and cosmology. Some of the strong observations in this regard are galaxy cluster observations by Zwicky [25], galaxy rotation curves [26], recent observation of the Bullet clusters [27] and cosmological data from the Planck collaboration [28]. All of these remarks suggest the existence of an unknown, non-luminous, non-baryonic dark matter which constitutes about 26% of the energy density of the universe and is approximately five times more than luminous matter. Currently the amount of dark matter in the universe as found from the Planck data is [29]

$$\begin{aligned} \Omega h^2=0.1199\pm 0.0027 \end{aligned}$$
(2)

This is called the relic density of dark matter. The properties that a candidate must have to qualify as dark matter candidate has been highlighted in [30, 31]. Unfortunately, the SM particles do not possess these required criteria and so none of them can be considered to be a viable dark matter candidate. Therefore, from the particle physics view point, one has to extend the SM particle content by incorporating new fields to find a suitable candidate that could produce the correct relic abundance.

Another serious conundrum in particle physics is the nature of neutrinos [32,33,34]: whether they are Dirac or Majorana particle. A possible solution to this question lies in the discovery of neutrinoless double beta decay (0\(\nu \beta \beta \)/NDBD). It is a SM forbidden second order lepton number violating interaction, which if discovered can shade light about the Majorana nature of the neutrinos. The decay channel of this reaction can be expressed as:

$$\begin{aligned} (A,Z)\rightarrow (A,Z+2) + 2e^- \end{aligned}$$

Currently there are many ongoing experiments which actively look for this decay. Some of these popular experiments are KamLANDZen [35], GERDA [36], CUROE etc. These experiments use Xenon-136 and Germanium-76 nuclei for possible detection of lepton number violating decays. The recent results published from these experiments predict the allowed lower bound for half-life (\(T_{\frac{1}{2}}\)) to be \(\ge 10^{26}\) years. Moreover the effective neutrino mass is found to be \(\le \) 0.165 eV.

Symmetry plays a very important role in model building in particle physics. Numerous work that has been done based on discrete flavor symmetry can be found in [37,38,39,40]. In this type of symmetry a large number of flavons, along with their VEV alignments, are used in the models.The non-Abelian discrete symmetry groups like \(A_4\), \(S_4\), \(Z_N\) etc. play a very crucial role in developing these models. In our work we have used modular symmetry in the framework of minimal Inverse seesaw [ISS(2,3)]. In this type of symmetry the Yukawa couplings are not free. Instead they are functions of a complex variable \(\tau \), called the modulus [41]. As a result this helps in reducing the number of flavons that are used in the model. In addition to this, one does not have to deal with their VEV alignments. The modular group, \(\Gamma (N)\), where N is the level of the group, acts on the upper half of the complex plane and transforms \(\tau \) in the following way:

$$\begin{aligned} \tau \rightarrow \frac{a\tau +b}{c\tau +d} \end{aligned}$$

where a, b, c, d are the elements of a 2\(\times \)2 matrix, such that \(ad-bc=1\). The literatures [42,43,44,45] highlight some of the recent works done in inverse seesaw using modular symmetry. Some of the finite modular groups ( \(N\le 5\)) are isomorphic to non-Abelian discrete symmetry groups (\(A_4, S_4, A^\prime _5\) etc). In our work we have used \(\Gamma (3)\) modular group. This group has three Yukawa modular forms of weight 2. One can obtain the higher weight modular forms by using these weight 2 modular functions. Moreover their values can be found with the help of their q-expansions [46]. A brief description on \(\Gamma (3)\) is given in Appendix A. The authors in [47, 48] have studied dark matter phenomena using discrete flavor symmetry. But in our work, along with dark matter, we have also studied leptogenesis and neutrinoless double beta decay by using modular symmetry.

This paper is organised as follows: in Sect. 2 we discsuss the basic structure of the model, charge assignments of particles under various groups used in the work and then construct its lagrangian. In Sects. 3 and 4 we give a brief description about neutrinoless double beta decay and leptogenesis in the framework of ISS(2,3). We have given a brief explanation about dark matter in Sect. 5. In Sect. 6 we provide the analysis and results of our work. We have shown the different plots and their descriptions in this part of the paper. Finally in Sect. 7 we conclude by giving an overview of our entire work.

2 The framework of inverse seesaw

Inverse seesaw is a popular BSM framework which is used in model building in neutrino physics. It is an extension of the Standard Model with three right-handed neutrinos and gauge singlet sterile fermions. Unlike the conventional seesaw mechanism, the Majorana mass of right-handed neutrinos in Inverse seesaw can be reduced to TeV scale. As a result, apart from generating tiny neutrino masses, it also enhances the possibility of detecting these right-handed particles in the ongoing and near future experiments.These possibilities of inverse seesaw have been highlighted in [49,50,51,52,53]. For the basis \(n_L=(\nu _{L,\alpha },N^c_{R,i},S_j)^T\), the Yukawa lagrangian for neutrino in this framework can be written as:

$$\begin{aligned} {{\mathcal {L}}}=-\frac{1}{2}n^T_LCMn_L+h.c. \end{aligned}$$
(3)

where \(C\equiv \gamma ^2 \gamma ^0\) is the charge conjugation matrix. The component \(\nu _{L,\alpha }\) for \(\alpha =e,\mu ,\tau \) are left-handed Standard Model neutrinos. The complete mass matrix for neutral fermion arising from the lagrangian can be written as [54]:

$$\begin{aligned} {\mathcal {M}}= \begin{pmatrix} 0&{}\quad M^T_D&{}\quad 0\\ M_D&{}\quad 0&{}\quad M_{NS}\\ 0&{}\quad M^T_{NS}&{}\quad M_S \end{pmatrix}_{9\times 9} \end{aligned}$$
(4)

The elements of the matrix in Eq. (4) represent the different mass matrices involved in ISS mechanism. The Dirac mass matrix (\(M_D\)) results from the interaction between left and right-handed components of neutrinos. \(M_{NS}\) is the mixing matrix that occurs due to right-handed neutrinos and sterile singlet fermions interactions. The interaction between the singlet sterile fermions forms the Majorana mass term, \(M_S\). The dimensions of the matrices are related to the number of generations of the particles that have been considered in the model. Accordingly in ISS the dimensions of these matrices can be defined in the following way (# represents the number of generations of particles.):

$$\begin{aligned}{} & {} Dimension \hspace{1.5mm} of\hspace{1.5mm} M_D=(\#\nu _L\times \#N_R)\nonumber \\{} & {} Dimension \hspace{1.5mm} of\hspace{1.5mm} M_{NS}=(\#N_R\times \#S)\nonumber \\{} & {} Dimension \hspace{1.5mm} of\hspace{1.5mm} M_S=(\#S\times \#S) \end{aligned}$$
(5)

From Eq. (5) one can find that the three matrices, \(M_D\), \(M_{NS}\) and \(M_S\), have the same dimension i.e. \(3\times 3\). The effective neutrino mass matrix for the active light neutrinos can be written as:

$$\begin{aligned} m_\nu =M^T_D(M^T_{NS})^{-1}M_SM^{-1}_{NS}M_D \end{aligned}$$
(6)

The light neutrino mass matrix in Eq. (6) can be diagonalised with the help of an unitary matrix. The corresponding eigenvalues will be the mass of the active neutrinos. In order to produce sub-eV Standard Model neutrinos, the energy scales of these matrices should be in the order \(M_S<<M_D<M_{NS}.\)

2.1 The model

In this work we have used minimal inverse seesaw [ISS(2,3)] framework. Compared to ISS, this mechanism of minimal inverse seesaw constitutes of two right-handed neutrinos (\(N_1,N_2\)) and three singlet sterile fermions (\(S_1,S_2,S_3\)). As a result, the order of \(M_D\), \(M_{NS}\) changes to \(3\times 2\) and \(2\times 3\); whereas for \(M_S\) it remains unchanged. Along with these particles we have used a flavon (\(\phi \)) whose VEV alignments facilitate to get a diagonal charged lepton mass matrix. As such the role of this flavon is restricted only to the charged lepton sector without affecting the neutrino sector. We have extended the minimal inverse seesaw by a Higgs-type scalar triplet \(\eta =(\eta _1,\eta _2,\eta _3)\) whose neutral component is our dark matter candidate [55, 56]. Proceeding ahead, we have used \(A_4\) modular symmetry to construct the desired lagrangian of the model. This group contains three Yukawa modular forms (\(y_1,y_2,y_3\)) of weight 2 which have been considered as triplet under \(A_4\) charge assignment. The right-handed neutrinos (\(N_1\) and \(N_2\)) in the model are taken to be singlets under \(A_4\) and they transform as \(1'\) and \(1''\), respectively; while the sterile neutrinos (\(S_i\)) and lepton doublets (L) are considered as triplets. The modular weight of the right-handed neutrinos is taken as \(-3\), the lepton doublets (L) is +1 where as for sterile neutrinos and \(\eta \) it is taken to be zero. In order to restrict certain interaction terms in the Lagrangian, we have used \(Z_3\) symmetry group. Moreover we introduce two weighton fields, \(\beta _1\) and \(\beta _2\), which are associated with the right-handed neutrinos. These are Standard Model singlet fields with non-zero modular weights [57]. In our work we have taken the weightons to be singlet under \(A_4\) and their modular weight is +1. These charge assignments of the particles under various groups in the model have been highlighted in Table 1.

Table 1 Charge assignments of the particles under the various groups considered in the model
Full size table

As we have considered a DM candidate in our model, it is necessary to introduce a discrete symmetry \(Z_2\) in order to maintain stability of the DM candidate. The SM particles remain \(Z_2\) even whereas the right handed neutrino and the newly added field eta are odd under this symmetry. The scalar potential of the Higgs sector can be found in [55]. After electroweak symmetry breaking, one of the \(\eta \)’s acquire VEV and their form can be written as:

$$\begin{aligned} \eta _1=\begin{pmatrix} \eta _1^+\\ \frac{v_\eta +h_1+iA_1}{\sqrt{2}} \end{pmatrix}, \hspace{1mm} \eta _2\!=\!\begin{pmatrix} \eta _2^+\\ \frac{h_2\!+\!iA_2}{\sqrt{2}} \end{pmatrix}, \hspace{1mm} \eta _3\!=\!\begin{pmatrix} \eta _3^+\\ \frac{h_3\!+\!iA_3}{\sqrt{2}} \end{pmatrix} \nonumber \\ \end{aligned}$$
(7)

As mentioned in [56], the VEV alignment of \(\eta \) can be written as \(\eta =v_\eta (1,0,0)\) and \(\eta _2,\eta _3\) will be the dark matter candidates. Moreover we get a diagonal charge lepton mass matrix when the VEV of \(\phi \) is taken as \(\phi =(u,0,0)\) [41]. The Lagrangian for the charged leptons can be written as:

$$\begin{aligned} L_L=\alpha _1E^c_1H_d(L\phi )_1+ \alpha _2 E^c_2H_d(L\phi )_{1'}+ \alpha _3E^c_3H_d(L\phi )_{1''}\nonumber \\ \end{aligned}$$
(8)

In the above equation, the parameters \(\alpha _1,\alpha _2,\alpha _3\) can be adjusted to get the desired masses of the charged leptons. Accordingly the mass matrix is found to be: \( M_L=diag(\alpha _1,\alpha _2,\alpha _3)uv.\) Here v is the vaccum expectation value of the Higgs field.

Now based on the above discussions and relevant charge assignmets, the Yukawa Lagrangian for the neutrino sector can be written as:

$$\begin{aligned}{} & {} {\mathcal {L}}=N_1(LY)_3\eta + N_2(LY)_3\eta \nonumber \\{} & {} \quad + \beta _1N_1(SY)_{1''} + \beta _2N_2(SY)_{1'}+\mu _0(SS)_1 \end{aligned}$$
(9)

The first two terms in the lagrangian denotes the interaction between left-handed and right-handed neutrinos. The next two terms represents the interaction between N’s and S’s while the last term denotes the interaction between the sterile fermions. As a result, from Eq. (9) we can write down the corresponding neutrino mass matrices in the following way. The Dirac mass matrix, which is a matrix of order 3\(\times \)2, for the neutrinos can be written as:

(10)

Similarly, following the \(A_4\) multiplication rules, the Majorana mass matrix for right-handed neutrino and sterile fermions, and the lepton number violating mass term for the sterile fermions can be written as:

$$\begin{aligned} M_{NS}=\begin{pmatrix} \beta _1y_3 &{}\quad \beta _1y_2 &{}\quad \beta _1y_1\\ \beta _2y_2 &{}\quad \beta _2y_1 &{}\quad \beta _2y_3 \end{pmatrix}, \hspace{7mm} M_S=\mu _0 \begin{pmatrix} 1 &{}\quad 0 &{}\quad 0\\ 0 &{}\quad 0 &{}\quad 1\\ 0 &{}\quad 1 &{}\quad 0 \end{pmatrix} \end{aligned}$$
(11)

Following the methods as mentioned in the literature, the full \(8\times 8\) neutrino mass matrix for ISS(2,3) can be written as:

$$\begin{aligned} {\mathcal {M}}= \begin{pmatrix} 0&{}M^T_D&{}0\\ M_D&{}0&{}M_{NS}\\ 0&{}M^T_{NS}&{}M_S \end{pmatrix}_{8\times 8} \end{aligned}$$
(12)

This \(8\times 8\) matrix M in Eq. (12) can be diagonalised with the help of an Unitary matrix, \({\mathcal {U}}\) as

$$\begin{aligned} \mathcal {U^T}M{\mathcal {U}}=M_{diag}=diag(m_1,m_2,m_3,....,m_8) \end{aligned}$$
(13)

where \(m_i\)’s in the above equation are masses of the particles of the model. Since the matrix in Eq. (11) is a rectangular matrix, so we cannot find the light neutrino mass matrix by the conventional approach [58]. Because of this reason, the formula to find the effective light neutrino mass matrix in Eq. (6) changes and takes the form [59]:

$$\begin{aligned} m_\nu =M_D \cdot d \cdot M^T_D \end{aligned}$$
(14)

In the above equation, d is a \(2\times 2\) matrix which can be derived from the \(5\times 5\) heavy neutrino mass matrix \(M_H\). The form of d can be obtained in the following way:

$$\begin{aligned} M_H^{-1}=\begin{pmatrix} 0 &{} M_{NS} \\ M_{NS}^T &{} M_S \end{pmatrix}^{-1}=\begin{pmatrix} d_{2\times 2} &{}.....\\ ...... &{}..... \end{pmatrix} \end{aligned}$$
(15)

Thus we have constructed a model using \(A_4\) modular symmetry in the framework of ISS(2,3). We can diagonalise Eq. (14) to obtain the masses of light neutrinos which will facilitate to study the related neutrino phenomenology.

3 Leptogenesis

In order to validate our model considering the cosmological constraints, we try to generate the observed baryon asymmetry of the universe through leptogenesis. There are five heavy neutrinos in ISS(2,3). As mentioned above, two of them are right handed neutrinos (\(N_1,N_2\)) and the other three are gauge singlet neutral sterile fermions (\(S_i\)). Four of these heavy particles form two pairs, called quasi-Dirac pairs, and one of them gets decoupled. Interestingly, the mass splitting between these pairs is comparable to their decay width. The out-of-equilibrium decay of the lightest pair to any lepton flavor creates an asymmetry in the leptonic sector. This asymmetry created by decay of the lightest heavy neutrinos can be converted into baryon asymmetry through sphaleron processes [52, 60, 61]. On the other hand, asymmetry generated by decay of the heavier pair is washed out by lepton number violating scatterings of the lightest pair, thereby, it does not contribute to the asymmetry produced.

3.1 Computation of CP asymmetry

To calculate the CP-asymmetry we need the mass matrix for the heavy neutrinos. This matrix can be written as:

$$\begin{aligned} M_H=\begin{pmatrix} 0 &{} M_{NS} \\ M_{NS}^T &{} M_S \end{pmatrix} \end{aligned}$$
(16)

Now on diagonalising the above matrix, one can get masses of the five heavy neutrinos. Also the mass splitting between the degenerate pairs is proportional to \(M_S\).

$$\begin{aligned} M_{diag}=V^TM_HV=diag(m_1,m_2,m_3,m_4,m_5) \end{aligned}$$
(17)

To diagonalise this \(5\times 5\) matrix analytically is a challenging and formidable task. So we opt for numerical diagonalisation so as to simplify our analysis. Again for the calculation of CP-asymmetry, a particular basis is preferred in which the matrix \(M_H\) becomes diagonal. In this basis the Lagrangian takes the form:

$$\begin{aligned} \mathcal {L_S}= h_{i\alpha }N_i\eta L_\alpha + M_iN_i^TC^{-1}N_i+h.c. \end{aligned}$$
(18)

where \(h_{i\alpha }\) corresponds to the couplings in diagonal mass basis and \(N_i\) are the mass eignenstates of the four non-decoupled heavy fermions. These are related to the couplings in the flavor basis as represented by the following relations [62, 63]:

$$\begin{aligned} h_{1\alpha }= & {} V_{11}^*\hspace{1mm}y_{1\alpha }+V_{12}^*\hspace{1mm}y_{2\alpha }\nonumber \\ h_{2\alpha }= & {} V_{21}^*\hspace{1mm}y_{1\alpha }+V_{22}^*\hspace{1mm}y_{2\alpha }\nonumber \\ h_{3\alpha }= & {} V_{13}^*\hspace{1mm}y_{1\alpha }+V_{23}^*\hspace{1mm}y_{2\alpha }\nonumber \\ h_{4\alpha }= & {} V_{14}^*\hspace{1mm}y_{1\alpha }+V_{24}^*\hspace{1mm} y_{2\alpha } \end{aligned}$$
(19)

For the decay \(N_i\rightarrow l_\alpha \phi \hspace{2mm} (\bar{l_\alpha }\phi )\), the formula for calculating the CP asymmetry \(\epsilon _i\) by summing over the SM flavor \(\alpha \) is given by [64]

$$\begin{aligned} \epsilon _i= & {} \frac{\sum _\alpha [\Gamma ({\tilde{\psi }}\rightarrow l_\alpha \phi )-\Gamma ({\tilde{\psi }}\rightarrow \bar{l_\alpha }\phi ^\dagger )]}{\sum _\alpha [\Gamma ({\tilde{\psi }}\rightarrow l_\alpha \phi )+\Gamma ({\tilde{\psi }}\rightarrow \bar{l_\alpha }\phi ^\dagger )]}\nonumber \\= & {} \frac{1}{8\pi }\sum _{i\ne j}\frac{Im[(hh^\dagger )^2_{ij}]}{(hh^\dagger )_{ii}}f_{ij} \end{aligned}$$
(20)

For the case of resonant leptogenesis, the self energy correction, \(f_{ij}=\frac{(M_i^2-M_j^2)M_iM_j}{(M_i^2-M_j^2)^2+(M_i\Gamma _i+M_j\Gamma _j)^2}\). \(M_i\) and \(M_j\) are the real and positive eigenvalues of the heavy neutrino mass matrix. \(\Gamma _i\) is the decay width of one of the quasi-Dirac pair which is expressed as \(\Gamma _i=\frac{M_i}{8\pi }(hh)^\dagger _{ii}\). Thus the explicit form of CP parameter for the decay of a quasi-Dirac pair, say \((N_1,S_1)\), can be expressed as:

$$\begin{aligned} \begin{aligned} \epsilon _1&=\frac{1}{8\pi (hh^\dagger )_{11}}\text {Im}[(hh^\dagger )^2_{12}f_{12}+(hh^\dagger )^2_{13}f_{13}+(hh^\dagger )^2_{14}f_{14}]\\ \epsilon _2&=\frac{1}{8\pi (hh^\dagger )_{22}}\text {Im}[(hh^\dagger )^2_{21}f_{21}+(hh^\dagger )^2_{23}f_{23}+(hh^\dagger )^2_{24}f_{24}] \end{aligned}\nonumber \\ \end{aligned}$$
(21)

As already mentioned, the asymmetry produced by decay of the heavier pair is washed out. Thus the wash out parameter for such decays, in terms of the Hubble parameter H, is written as:

$$\begin{aligned} K_i=\frac{\Gamma _i}{H}=\frac{M_i}{8\pi }(hh)^\dagger _{ii}\times \frac{M_{pl}}{1.66\sqrt{g*}M_i^2} \end{aligned}$$
(22)

In the above equation, \(M_{pl}\) is the Planck mass and \(g*\) denotes the effective number of relativistic degrees of freedom. The final expression for BAU can be written as:

$$\begin{aligned} Y_{B}=10^{-2}\sum \kappa _i\epsilon _i \end{aligned}$$
(23)

where \(\kappa _i\) is the dilution factor responsible for washout out of the asymmetry associated with the heavy pair. \(\epsilon _i\) is the CP asymmetry generated in the leptonic sector. The expressions for \(\kappa _i\) depends on the values of washout factor in eq (22). These relations can be summarised as [65]:

$$\begin{aligned} \begin{aligned} -\kappa&\approx \sqrt{0.1K}\hspace{2mm}exp\Bigg [\frac{-4}{3(0.1K)^{0.25}}\Bigg ], \hspace{6mm} \text {for } K\ge 10^6\\&\approx \frac{0.3}{K(\text {ln}K)^{0.6}},\hspace{2.5cm} \text {for} \hspace{2mm} 10\le K\le 10^6\\&\approx \frac{1}{2\sqrt{K^2+9}}, \hspace{2.5cm} \text {for} \hspace{2mm} 0\le K\le 10 \end{aligned} \nonumber \\ \end{aligned}$$
(24)

Thus, we use Eq. (23) to calculate the value of BAU in the framework of ISS(2,3). In our work we have computed the values of this asymmetry for both the hierarchies. We have highlighted our findings in the numerical analysis section of the paper. We find that for both the hierarchies there are a large number of points that satisfy the Planck value of BAU.

4 Neutrinoless double beta decay

Neutrinoless double beta decay \((0\beta \beta \nu /\text {NDBD})\) is an important phenomena in particle physics. It is a lepton number violating decay which if discovered can give an answer to many of the open questions in particle physics. In our work, we have studied the effective electron neutrino Majorana mass which characterizes \(0\beta \beta \nu \). As mentioned in [66], the experiments like KamLAND-Zen [35], GERDA [36], CUORE [67] etc. provide a stringent bound on \(m_{ee}\). In absence of any sterile neutrino, this effective mass is expressed as given below:

$$\begin{aligned} m_{ee}=\Bigg |\sum ^3_iU_{ei}^2m_i\Bigg | \end{aligned}$$
(25)

The presence of sterile neutrinos in ISS(2,3) changes the expression for effective mass in Eq. (25). As a result, the modified expression become [14, 68]:

$$\begin{aligned} m_{ee}=\Bigg |\sum ^3_{i=1}U_{ei}^2m_i\Bigg |+ \Bigg |\sum _{j=1}^5U_{ej}^2\frac{M_j}{k^2+M_j^2}\big |\big <k\big >\big |^2\Bigg | \end{aligned}$$
(26)

where \(U_{ej}\) represents the coupling of the heavy neutrinos to the electron neutrino. \(M_j\) is the mass of the heavy neutrinos. The parameter k is known as the virtuality momentum and its value is \(\big |\big <k\big >\big |\approx \) 190 MeV.

5 Dark matter

At the beginning of the universe the particles present in the thermal pool were in thermal equilibrium with each other. This implies that the rate at which lighter particles combined to form heavy particles and vice-versa was the same. During the course of evolution, the conditions that were required to maintain this equilibrium state were disturbed. As a result, after a certain temperature the density of some particle species became too low. Once this is achieved, the abundance of those particle remains the same and their density becomes constant. This phase of the particle species is called freeze-out and the density hereafter is referred to as relic density. For a particle \(\chi \), which was in thermal equilibrium, the relic density can be obtained from the Boltzman equations [69, 70]:

$$\begin{aligned} \frac{dn_\chi }{dt}+3{\mathcal {H}}n_\chi =-<\sigma v\>\left( n_\chi ^2-\left( n_\chi ^{eqb}\right) ^2\right) \end{aligned}$$
(27)

In the above equation \(n_\chi \) is the number density of the dark matter particle whereas \(n_\chi ^{eqb}\) is the density of \(\chi \) when it was in equilibrium with the thermal bath. Here \({\mathcal {H}}\) represents the Hubble constant and \(<\sigma v\>\) is the thermally averaged annihilation cross-section of the dark matter candidate. For the interactions in Eq. (9), the cross-section formula can be writtern as [71]:

$$\begin{aligned} <\sigma v\>= \frac{v^2 y^4 m_\chi ^2}{48\pi (m_\chi ^2+m_\psi ^2)^2} \end{aligned}$$
(28)

where the parameters \(m_\chi , m_\psi \hspace{2mm}\text {and}\hspace{2mm} y\) represent the mass of the relic particle, mass of the Majorana fermion and the interaction between the dark matter and fermions in the model. v in the above expression represents the relative velocity of the relic particles whose value at the time of freeze-out is taken to be 0.3c. The solution of Eq. (27) in terms of reduced Hubble constant (h), as found in [72], can be written as:

$$\begin{aligned} \Omega _\chi h^2=\frac{3\times 10^{-27} \text {cm}^3 \text {s}^{-1}}{<\sigma v\>} \end{aligned}$$
(29)

The above expression of \(\Omega _\chi h^2\) gives the relic density of dark matter particle. It is found that the self annihilation between dark matter and next to lightest neutral component of \(\eta \) contributes to the annihilation cross-section. For low mass region i.e. \(m_{DM}<M_W\), the cross-section for the self annihilation of either \(\eta _2\) or \(\eta _3\) into Standard Model particles via the Higgs boson can be written as [73]:

$$\begin{aligned} \sigma _{xx}=\frac{|Y_f|^2|\lambda _x|^2}{16\pi s}\frac{(s-4m_f^2)^{\frac{3}{2}}}{\sqrt{s-4m_x^2}((s-m_h^2)^2+m_h^2\Gamma _h^2)} \end{aligned}$$
(30)

where x is the dark matter particle (\(\eta _2,\eta _3\)) and \(\lambda _x\) is the coupling of x with the SM Higgs boson h. In Eq. (30) \(Y_f\) represents the Yukawa couplings of the fermions. \(\Gamma _h\) represents the decay width of the SM Higgs and \(m_h\) is equal to 125 GeV. And s in the expression represents the thermally averaged center of mass squared energy and is given by \(s=4m_x^2+m_x^2v^2\), \(m_x\) is mass of the relic. In our work the neutral component of the scalar triplet \(\eta \) is the dark matter candidate. Moreover for the low mass region, the mass of the DM particle should be less than the mass of W boson, \(M_W\) [47]. The results that we have found in this work and their respective analysis are shown in the next section.

6 Numerical analysis

In this section we discuss the processes that we followed to arrive at the results of our work. In our calculations we have used the latest nu-fit data for the oscillation parameters. This set of data has been shown in the Table 2.

Table 2 The latest \(3 \sigma \) nu-fit values of oscillation parameters [74]
Full size table
Fig. 1
figure 1

Corelation between real and imaginary parts of \(\tau \) with the mixing angles for both the orderings

Full size image
Fig. 2
figure 2

Contour plots of the Yukawa modular forms with the solar mixing angle (\(\theta _{12}\))

Full size image
Fig. 3
figure 3

Contour plots of the Yukawa modular forms with the reactor mixing angle (\(\theta _{13}\))

Full size image
Fig. 4
figure 4

Contour plots of the Yukawa modular forms with the atmospheric mixing angle (\(\theta _{23}\))

Full size image

In order to find the model parameters, we diagonalise the light neutrino mass matrix using the standard relation \(m_\nu =U^T \text {diag}(m_1,m_2,m_3)U\), where U is the PMNS matrix. In normal hierarchy the diagonal matrix can be written as \(\text {diag}(0,\sqrt{m_1^2+\Delta m^2_{solar}},\sqrt{m_1^2+\Delta m^2_{atm}})\), whereas in the inverted ordering it takes the form \(\text {diag}(\sqrt{m_3^2+\Delta m^2_{atm}}), \sqrt{\Delta m_{atm}^2+\Delta m^2_{solar}},0)\) [75]. Through this process we get the masses of light active neutrinos. Moreover, the solar and atmospheric mass squared differences, along with the sum of neutrino masses, provide additional constraints on the model. In order to evaluate the values of the mixing angles from the model we use the following relations [42]:

$$\begin{aligned}&\text {sin}^2\theta _{13}=|U_{e3}|^2, \hspace{6mm} \text {sin}^2\theta _{23}=\frac{|U_{\mu 3}|^2}{1-|U_{e3}|^2}, \nonumber \\&\text {sin}^2\theta _{12}=\frac{|U_{e 2}|^2}{1-|U_{e3}|^2}, \end{aligned}$$
(31)

The real and imaginary parts of the complex modulus, \(\tau \) is found to lie within the fundamental domain [41]. Here in this work the ranges for the real and imaginary parts are found to be: Re(\(\tau \))\(\rightarrow \) [− 0.9, 0.9] and Im(\(\tau \))\(\rightarrow \)[0.2, 6]. With these values of \(\tau \) we can compute the Yukawa modular forms for both normal and inverted ordering. For further calculations we have taken \(\mu _0\) in the range [10, 20] KeV and \(v_\eta \) is considered in between (\(30-50\)) GeV. In a similar way, we have taken the values of \(\beta _1\) and \(\beta _2\) in the ranges [\(10^5, 10^6\)] GeV and [\(10^2, 10^3\)] GeV, respectively, so as to obtain the values of different parameters in the allowed ranges. In the following parts of this section we will show the different plots and then discuss the respective results that are obtained in this work.

The above Fig. 1 demonstrate the relation between Re\((\tau )\), Im\((\tau )\) of the complex modulus \(\tau \) and the neutrino mixing angles for both the orderings. It can be seen from the figures that the range of Re\((\tau )\) is same for all the cases, which lies around the range (\(-\) 0.9 to 0.9). The allowed values of the atmospheric mixing angle (\(\theta _{23}\)) is found to lie in between (0.2–1.2) of Im(\(\tau \)) for normal ordering, whereas for inverted ordering this range is found to be (0.4–1.0). This is depicted by the above two figures at the top. The two figures in the middle is for solar mixing angle (\(\theta _{12}\)). Interestingly for both the orderings the range of Im\((\tau )\) which accomodates the allowed values of this mixing angle is found to be almost similar i.e. (0.3–6). From the two figures at the bottom we can find that the reactor mixing angle \((\theta _{13})\) is confined to the region (0.8–1) of Im\((\tau )\). Consequently we can find a common space of Re\((\tau )\) and Im\((\tau )\) for the mixing angles: \(-0.9\le \text {Re}(\tau )\le 0.9\) and \(0.7\le \text {Im}(\tau )\le 1.0\).

In the above Fig. 2 we show the corelation between the Yukawa modular forms \((y_1,y_2,y_3)\) and solar mixing angle (\(\theta _{12}\)). The two figures on the left are the contour plots for (\(y_1,y_2\)) with the mixing angle \(\theta _{12}\), the middle images are for (\(y_1,y_3\)) and the figures on the right are for (\(y_2,y_3\)). From these figures the ranges of the Yukawa modular forms can be found as: \(0.9\le |y_1|\le 1.2, 0.1\le |y_2|\le 2 \hspace{1mm}\), \(0.1\le |y_3|\le 1.5\) for normal ordering and \(0.9\le |y_1|\le 1.2, \hspace{1mm} 0.1\le |y_2|\le 1.8, 0.1\le |y_3|\le 1.5\) for inverted ordering.

Similarly in Figs. 3 and 4 we show the contour plots of the Yukawa modular forms, reactor mixing angle (\(\theta _{13}\)) and atmospheric mixing angle (\(\theta _{23}\)). Accordingly we can find the ranges of \(y_1,y_2,y_3\) corresponding to the mixing angles from these figures. For the reactor mixing angle these ranges are: \(1.024\le |y_1|\le 1.045, \hspace{1mm} 0.6\le |y_2|\le 0.9, \hspace{1mm} 0.2\le |y_3|\le 0.4\) for both normal and inverted ordering, whereas for the atmospheric mixing angle these ranges are found to be: \(0.94\le |y_1|\le 1.02, \hspace{1mm} 0.5\le |y_2|\le 1.2, \hspace{1mm} 0.2\le |y_3|\le 0.9\) for normal ordering and \(0.94\le |y_1|\le 1.02, \hspace{1mm} 0.5\le |y_2|\le 1.1, \hspace{1mm} 0.2\le |y_3|\le 0.7\) for inverted ordering. These values of the modular forms have been summarised in the Table 3.

Table 3 This table shows the range of the modular forms corresponding to the mixing angles for both normal and inverted hierarchy
Full size table
Fig. 5
figure 5

Variation between \(<\eta \>\) and Baryon asymmetry of the universe

Full size image
Fig. 6
figure 6

The above figures illustrate the corelation between \(\sum m_\nu \) and \(m_{eff}\). The horizontal and vertical lines represent the upper bounds of the parameters

Full size image
Fig. 7
figure 7

The figures above show the corelation between BAU and \(M_1\) for both the hierarchies. In these figures, the horizontal lines represent the Planck value of BAU

Full size image
Fig. 8
figure 8

These figures show the corelation between BAU and sum of neutrino masees \(\sum m_{\nu }\). The horizontal and vertical lines represent the Planck value of BAU and upper bound of \(\sum m_\nu \), respectively

Full size image
Fig. 9
figure 9

The above figure shows the variation between DM relic density and lightest right-handed neutrino \(M_1\). The horizontal line represents the current dark matter abundance in the universe

Full size image
Fig. 10
figure 10

The above figure shows the corelation between cross-section \(<\sigma v\>\) and \(M_1\)

Full size image

In Fig. 5 we show the relation between the VEV of \(\eta \) and BAU. For normal ordering it can be seen that the Planck value of BAU does not have any correspondence to the mass of the scalar \(\eta \). In other words, we can say that it does not contribute to the study of leptogenesis. But for the case of inverted ordering, there are sufficient values of BAU within the Planck limit that corresponds to the mass of the scalar. This implies that for inverted ordering the scalar has some contribution to leptogenesis.

Figure 6 shows the relation between sum of neutrino mass (\(\sum m_\nu \)) and effecive mass (\(m_{eff}\)). The recent cosmological findings from KamLANDZen provide the upper bound of \(\sum m_\nu \) \(\le \) 0.12 eV, whereas the allowed range for the effective neutrino mass for NDBD is found to be \(\le \) 0.165 eV. The horizontal line in the figure represents the upper bound of effective electron neutrino mass, whereas the vertical line represents the upper bound for sum of neutrino mass. It is quite evident from both the graphs that there are sufficient parameter space within the allowed range for both the normal and inverted orderings.

In Fig. 7 we show the relation between BAU and lightest right-handed neutrino mass, \(M_1\). We performed the calculations for both the hierarchies. From these graphs it is clear that for both the type of hierarchies there are sufficient values of BAU which satisfy the Planck limit. This value of BAU for normal ordering is mainly concentrated in between (100–1000) GeV of \(M_1\). Beyond this mass range the values of asymmetry is very less. But for inverted ordering this mass range is found to lie in between (100–5000) GeV. From this discussion it is clear that the values of baryon asymmetry of the universe can be obtained for both the hierarchies from this model. Also in Fig. 8, we show the relation between BAU and sum of neutrino masses.

Finally in Figs. 9 and 10 we show the variation of dark matter relic density and thermally averaged annihilation cross-section with respect to lightest heavy neutrino \(M_1\). It can be seen that for \(M_1\) in between (100–450) GeV, the values of relic density is more prominent for both the hierarchies. However, in case of inverted hierarchy, there are certain areas of mass that produce the observed relic density. As for the thermally averaged cross-section, we see that for almost the entire mass range of the \(M_1\), the values of the scattering cross section is consistent with the indirect detection limits [76].

7 Conclusion

In this work, we have extended the minimal inverse seesaw by adding an extra Higgs-type scalar triplet (\(\eta \)) to its particle content. As mentioned in the earlier sections, this is a very popular mechanism in model building because of its ability to reduce the mass scale of right-handed neutrinos to TeV. We have used \(A_4\) modular symmetry and \(Z_3\) symmetry groups in our work. In \(A_4\) modular group there are three Yukawa modular forms which are functions of the complex modulus \(\tau \). These modular forms play a vital role in our calculations. In this paper, we have studied neutrino masses and mixings, neutrinoless double beta decay, baryogenesis via leptogenesis and dark matter. We have considered the results and bounds related to these phenomena which are published by different ongoing experiments and observations. We find that the values of the parameters calculated from the model are within the allowed ranges. Moreover, we have performed these calculations for both the normal and inverted hierarchies. In case of the Yukawa couplings, most of the values are mainly found to lie in the region (0.1–1.8). Also there is a sufficient parameter space within the allowed range for sum of neutrino masses and effective electron neutrino mass for NDBD.

Furthermore, on studying BAU in our model, we have obtained a satisfying parameter space corresponding to \(M_1\) which abide by the Planck limit for both NO/IO. Again, we have extended our investigation to dark matter sector as well by calculating the relic abundance and scattering cross section of the DM candidate. From plots in Figs. 9 and 10, we have obtained a certain parameter space for DM mass which generates the observed relic abundance. Also, we have sufficient points which showcase very small scattering cross section. Thus, from the study of various phenomena that we have carried out in this work, we can have a conclusive idea that this model is a viable one.