Consider the problem Here $\mathsf{\Omega }$ is an open bounded domain with smooth boundary $\partial \mathsf{\Omega }$ in ${\mathbb{R}}^N\left(N\ge 3\right)$, and $0\in \mathsf{\Omega }.0\le \mu <\overline{\mu }:={\left(\frac{N-2}{2}\right)}^2,2^{*}\left(s\right)=2\left(N-s\right)/\left(N-2\right)(0<s<2)$ is the Hardy-Sobolev critical exponent and $2^{*}=2^{*}\left(0\right)$ is the Sobolev critical exponent. $f\in C\left(\mathsf{\Omega }\times \mathbb{R},\mathbb{R}\right)$, $F\left(x,t\right)={\int }_0^{t}f\left(x,s\right)ds$. We point out that (1) is related to the application of hydrodynamics and glaciology ([1]). And it is also used in some physical or mathematical problems, such as the theory of gas combustion in thermodynamics ([2]), quantum field theory and statistical mechanics ([3,4,5]), as well as gravity balance problems in galaxies ([2,6]). For more investigations on solutions for nonlinear equations with Hardy potential, one can see [7,8,9] etc.

The modern variational method ([10,11,12,13]) plays a significant role in studying PDEs (see [14,15,16,17,18]). In 1973, the mountain pass lemma was proposed by A. Ambrosetti and P. Rabinwitz in [14], it is a milestone in the history of the development of critical point theory. However, in the process of studying the properties for certain equations, there are a lot of phenomena that lose compactness conditions, such as semilinear elliptic equations that involving Sobolev critical exponent or Hardy-Sobolev critical exponent on bounded domain. In 1983, H. Brezis and L. Nirenberg first chose special mountain pass and selected energy estimates to prove the existence of a critical point if the energy functional satisfies the local $\left(PS\right)$ condition (see [15]), they investigated the problem and obtained that there exists a ${\lambda }_{*}\in \left(0,{\lambda }_1\right)$ such that for any $\lambda \in \left({\lambda }_{*},{\lambda }_1\right),$ problem (2) admits a positive solution. It is a special case of equation (1) ($s=0,\mu =0$ and $f\left(x,u\right)=\lambda u$). Since then, many excellent results based on the above methods (see [11,19,20,21]) appeared.

In the past decades, the semilinear elliptic equation with Hardy term and Sobolev critical exponent (i.e. when $s=0$ and $\mu \ne 0$) has been investigated by many mathematicians, one can refer to [22,23,24,25] etc. For example, the following elliptic problem is considered in [22,23,24].

For simplicity, in the following, we denote the condition (H1) and (H2) as follows:

(H1) $0<\lambda <{\lambda }_1\left(\mu \right)$ and $0\le \mu \le \overline{\mu }-1;$

(H2) $\overline{\mu }-1<\mu <\overline{\mu }$ and ${\lambda }_{*}\left(\mu \right)<\lambda <{\lambda }_1\left(\mu \right).$

In [24], by the variational method, E. Jannelli proved that: If (H1) or (H2) holds, then (3) has at least one positive solution in $H_0^1\left(\mathsf{\Omega }\right)$. Later in [26], the authors investigated problem (1) with $f\left(x,u\right)=\lambda {\left|u\right|}^{q-2}u$ or $f\left(x,u\right)=\lambda u$. And obtained the following conclusion.

Also there are some results dealing with the case $\mu \ne 0,$ $s\ne 0$ and the general form $f\left(x,u\right)$ (see [27,28]). In [27], M.C. Wang and Q. Zhang showed that problem (1) has at least one nonnegative solution. In [28], L. Ding and C.L. Tang also investigated problem (1) and obtained the existence result. Inspired by [26,27,28], we study the existence of nontrivial solutions for problem (1). Our main conclusions are

Obviously, in Theorem 1, the values of $f\left(x,t\right)$ are irrelevant for $t<0$, therefore, we define

By Hardy inequality and Hardy-Sobolev inequality (see [29]), we define equivalent norm and inner product in $H_0^1\left(\mathsf{\Omega }\right)$

Let The energy functional $J:H_0^1\left(\mathsf{\Omega }\right)\to \mathbb{R}$ to (1) is given by We can easily obtain that $J\left(u\right)$ is well defined with $J\in C^1\left(H_0^1\left(\mathsf{\Omega }\right),\mathbb{R}\right)$ and

When $0\le \mu <\overline{\mu }$, the best constant can be defined as follows (see [30])

From (8), we know for ${\alpha }_2\in \left(1,2^{*}\left(s\right)-1\right)$, if we take $\xi \ge max\left\{0,\left(\lambda -{\epsilon }_1\right)M_1^{1-{\alpha }_1}\right\}$, then for any$t\in \left[M_1,+\infty \right)$, we have $\left(\lambda -{\epsilon }_1\right)t-\xi t^{{\alpha }_1}\le 0$, thus

When $t\in \left[\delta ,M_1\right]$, taking $\xi \ge max\left\{0,{\displaystyle \underset{t\in \left[\delta ,M_1\right]}{max}}\{\left(\lambda -{\epsilon }_1\right)t^{1-{\alpha }_1}+\left(\eta -{\epsilon }_1\right)t^{2^{*}\left(s\right)-1-{\alpha }_1}\}\right\},$ we have

As mentioned above, if we take then

Similarly, we may obtain that there exists $\xi >0$ such that

The conclusion is proved.

Now we introduced the extremal functions. Let define a cut-off function $\phi \in C_0^{\infty }\left(\mathsf{\Omega }\right)$ such that where $B_{2R}\left(0\right)\subset \mathsf{\Omega },0\le \phi \left(x\right)\le 1$, for $R<\left|x\right|<2R$, set

Assume that $u\equiv 0$ in $\mathsf{\Omega }$, from $〈J^{\prime }\left(u_n\right),u_n〉=o\left(1\right)$ and (15) we know By (4), From (16) and (17), we have If $∥u_n∥\to 0$, then (16) implies that $J\left(u_n\right)\to 0$, while $J\left(u_n\right)\to c$, which contradicts $c>0$. Hence By (12), (16) and (18), we obtain which contradicts $c<\frac{2-s}{2\left(N-s\right)}{A}_{\mu ,s}^{\frac{N-s}{2-s}}$. Thus, u is not constantly equal to 0 and u is a nontrivial solution of problem (1).

On the other hand, $\overline{g}\left(t\right)\le$ $\overline{g}\left({\overline{t}}_{\epsilon }\right)$ for any $t\in \left[0,{\overline{t}}_{\epsilon }\right]$. From (5), (11), (12), (21), (22) and Lemma 2, we get

If (H1) holds, notice that $2\ge \frac{N}{\sqrt{\overline{\mu }}+\sqrt{\overline{\mu }-\mu }}$, then when $\epsilon$ is sufficiently small, the sign of $-\frac{\lambda -{\epsilon }_1}{2}{t}_{\epsilon }^2{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{2}dx+\frac{\xi }{\alpha +1}{\left|t_{\epsilon }\right|}^{{\alpha }_1+1}{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{{\alpha }_1+1}dx-\frac{\eta -{\epsilon }_1}{2^{*}\left(s\right)}{\left|t_{\epsilon }\right|}^{2^{*}\left(s\right)}{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{2^{*}\left(s\right)}dx$ is decided by the sign of $-{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{2^{*}\left(s\right)}dx.$Thus, when $\epsilon$ is small enough, (20) holds true.

If (H2) holds, since ${\alpha }_1>1$ is arbitrary, we can choose ${\alpha }_1>\frac{N}{\sqrt{\overline{\mu }}+\sqrt{\overline{\mu }-\mu }},$ then by Lemma 2, we know that when $\epsilon$ is sufficiently small, the sign of $\frac{\xi }{\alpha +1}{\left|t_{\epsilon }\right|}^{{\alpha }_1+1}{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{{\alpha }_1+1}dx-\frac{\eta -{\epsilon }_1}{2^{*}\left(s\right)}{\left|t_{\epsilon }\right|}^{2^{*}\left(s\right)}{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{2^{*}\left(s\right)}dx$ is decided by the sign of $-{\int }_{\mathsf{\Omega }}{\left|v_{\epsilon }\right|}^{2^{*}\left(s\right)}dx.$ Thus from (23), when $\epsilon ,$ ${\epsilon }_1$ and ${\epsilon }_2$ are small enough, From (13) we know where $C={\left(\frac{2\left(\overline{\mu }-\mu \right)\left(N-s\right)}{\sqrt{\overline{\mu }}}\right)}^{\frac{\sqrt{\mu }}{2-s}},$ ${\epsilon }_0={\epsilon }^{\frac{\sqrt{\overline{\mu }}}{2-s}},$ $P={\left(C^{2}D{\epsilon }_0+A_{\mu ,s}^{\frac{N-s}{2-s}}\right)}^{\frac{N-2}{2-s}},$ $Q={\left(E{C}^{2^{*}\left(s\right)}{\epsilon }_0^{\frac{N-s}{N-2}}+A_{\mu ,s}^{\frac{N-s}{2-s}}\right)}^{\frac{N-4+s}{2-s}}.$ By (14), (13) and (24), we have so, if $\epsilon$ is small enough, then $c\le g\left(t_{\epsilon }\right)<\frac{2-s}{2\left(N-s\right)}{A}_{\mu ,s}^{\frac{N-s}{2-s}}.$