It is known that convolutional neural network provides various models and algorithms to process data model in many fields such as computer vision(see e.g. [1]), natural lagrange processing (see e.g. [2]), and sequence analysis in bio-informatics(see e.g. [3]). The regularized neural network learning has thus become an attractive research topic (see e.g.[4,5,6,7,8,9]). In the present paper, we shall give theory analysis for the convergence rate of regularized regression associated with zonal translation network on the unit sphere.

Let X be a compact subset in the d-dimensional Euclidean space $R^d$ with the usual norm $\parallel x\parallel =\sqrt{{\displaystyle \sum _{k=1}^d}{x}_k^2}$ for $x=(x_1,x_2,\cdots ,x_d)\in R^d$ and Y be a nonempty closed subset contained in $[-M,M]$ for a given $M>0$. The aim of the regression learning problem is to learn the target function which describes the relationship between the input $x\in X$ and the output $y\in Y$ from a hypothesis function space. In most of the cases, the target function is offered as a set of observations $z={\left\{z_i\right\}}_{i=1}^m={\left\{(x_i,y_i)\right\}}_{i=1}^m\in Z^m$ which has been drawn independently and identically distributed (i.i.d.) according to a probability joint distribution (measure) $\rho (x,y)={\rho }_X\left(x\right)\rho \left(y\right|x)$ on $Z=X\times Y$, where $\rho \left(y\right|x\left)\right(x\in X)$ is the conditional probability of y for a given x and ${\rho }_X\left(x\right)$ is the marginal probability about x, i.e.,for every integrable function $\phi :X\times Y\to R$ there holds

For a given normed space $(B,\parallel ·{\parallel }_B)$ consisting of real functions on X we define the regularized learning framework with B as where $\lambda >0$ is the regularization parameter,${\mathcal{E}}_z\left(f\right)$ is the empirical mean The optimal target function is the regression function satisfying where ${\mathcal{E}}_{\rho }\left(f\right)={\int }_Z{(y-f\left(x\right))}^{2}d\rho$ and the inf is taken over all the ${\rho }_X$-measurable functions f. Moreover, there holds the famous equality (see e.g.[10])

The choices for the hypothesis space B in (1) are riches. For example, C.P.An et al choose the algebraic polynomial class as B (see [11,12,13],). In [14], C. De Mol et al chose the dictionary as B. Recently some papers chose the Sobolev space as the hypothesis space B (see [15,16]). By kernel method we mean traditionally replacing B with a reproducing kernel Hilbert space (RKHS) $(H_K,{\langle ·,·\rangle }_K)$ which is a Hilbert space consisting of real functions defined on X and there is a Mercer kernel $K_x\left(y\right)=K(x,y)$ on $X\times X$ (i.e., $K_x\left(y\right)$ is a continuous and symmetric function on $X\times X$ and for any $n\ge 1$ and any $\{x_1,x_2,\cdots ,x_n\}\subset X$ the Mercer matrices ${\left(K(x_i,x_j)\right)}_{i,j=1,2,\cdots ,n}$ are positive semi-definite) such that and there hold the embeded inequality where c is a constant independent of f and x. There are two results about the optimal solution $f_{z,\lambda }$. The reproducing property (3) yields the representation The embeded inequality (4) yields the inequality

Representation (5) is the theory basis for kernel regularized regression (see e.g.[17,18]). Inequality (6) is the key inequality for bounding the learning rate with covering number method (see e.g.[19,20,21]). For other skills of the kernel method one can cite [10,22,23,24] et al.

It is particularly important to mention here that translation networks have recently been used for the hypothesis space of regularized learning (see e.g. [25,26]). From the view of approximation theory, a simple single layer translation network with m neurons is a function space produced by translating a given function $\varphi$ which can be written as where $\overline{X}={\left\{x_i\right\}}_{i=1}^m\subset X$ is a given node set, and for a given $x\in X$$T_x(\varphi ,y)$ is a translation operator corresponding to X. For example, when $X=R^d$ and $X={[-\pi ,\pi ]}^d$, we choose $T_x(\varphi ,y)$ as the usual convolution translation operator $\varphi (x-y)$ for the $\varphi$ defined on $R^d$ or an $2\pi$-periodic function $\varphi$ (see [27,28]. When $X=S^{d-1}=\{x\in R^d:\parallel x\parallel =1\}$ is the unit sphere in $R^d$, one can choose $T_x(\varphi ,y)$ as the zonal translation operator $\varphi \left(xy\right)$ for a given $\varphi$ defined on the interval $[-1,1]$ (see [29]). In [30], we defined a kind of translation operator $T_x(\varphi ,y)$ for $X=[-1,1]$.

It is easy to see that the approximation ability and the construction of a translation network depend upon the nodes set $\overline{X}={\left\{x_i\right\}}_{i=1}^m$ (see e.g. [31,32,33]). On the other hand, according to the view of [34], the quadrature rule and the Marcinkiewicz-Zygmund (M-Z) inequality associated with $\overline{X}$ also influence the construction of the translation network ${\Delta }_{\varphi ,\overline{X}}$. For a bounded closed set $\Omega \subset R^d$ with measure $d\omega$ satisfying ${\int }_{\Omega }d\omega =V<+\infty$. We denote by $P_n\subset L^2(\Omega )$ the linear space of polynomials on $\Omega$ of degree at most n, equipped with the $L^2$-inner product $\langle v,z\rangle ={\int }_{\Omega }vzd\omega .$ The m-point quadrature rule (QR) is where $\overline{X}={\left\{x_j\right\}}_{j=1}^m\subset \Omega$ and weights $w_j$ are all positive for $j=1,2,\cdots ,m.$ We say the QR (7) has polynomial exactness n if The Marcinkiewicz-Zygmund (MZ) inequality based on a set $\overline{X}={\left\{x_j\right\}}_{j=1}^m\subset \Omega$ is where the weights $w_j$ in (9) may be not the same as the $w_j$ in (7) and (8).

Accord to the view of [34], the quadrature rule (QR) follows automatically from the Marcinkiewicz-Zygmund (MZ) inequality. H.N.Mhaskar et al gave a method of transition from MZ inequality to polynomial exact GR in [35]. So the MZ inequality (9) is an important features for describing the nodes set $\overline{X}$. Since this reason the node set $\overline{X}={\left\{x_j\right\}}_{j=1}^m\subset \Omega$ which yields an MZ inequality is given a specially terminology called Marcinkiewicz-Zygmynd Family (MZF) (see [34,36,37,38]). However, from these literatures we know that the MZFs do not totally coincide with the Lagrange interpolation nodes in the case of $d>1$. The hyperinterpolations are then developed with the help of exact QR (see [39,40,41,42,43]) and are applied to approximation theory and regularized learning (see e.g.[11,12,13,44]). On the other hand, we find the problem of polynomial exact QR is investigated individually (see e,g, [45,46]). The concept of spherical t-design was first defined in [47] and is given investigations by many papers subsequently, one can see the classical references [48,49]. We say $T_t={\left\{x_i\right\}}_{i=1}^{|T_t|}\subset S^{d-1}$ is a spherical t-design if where ${\omega }_{d-1}$ is the volume of $S^{d-1}$ and $\pi \left(x\right)$ is a spherical polynomial with degree t. Moreover, in many applications the polynomial exact QR and the MZFS has been used as assumptions. For example, C.P.An et al gave approximation order for the hyperinterpolation approximation under the assumptions that (8), (10) and the MZ inequality (9) hold (see [50,51]). Also, in [25], Lin et al gave investigations on regularized regression associated with zonal translation network by assuming the nodes set $\overline{X}={\left\{x_j\right\}}_{j=1}^m\subset S^{d-1}$ is a type of spherical t-design.

Polynomial exact QR is also a good tool in approximation theory. For example, we had ever used it to bound the norm for some Mercer matrices (see [52,53,54,55]). In particular, H.N.Mhaskar et al use polynomial exact QR to construct the first periodic translation operators (see [27]) and the zonal translation network operators (see [29]). Along this line, the translation operators defined on the unit ball, on the Euclidean space $R^d$ and on the interval $[-1,1]$ are constructed (see [28,30,56]).

Above investigations encourage us to use (8), (10) and (9) as hypothetical conditions to describe the approximation ability of zonal translation networks ${\Delta }_{\varphi ,\overline{X}}^X$. To ensure the single layer translation network ${\Delta }_{\varphi ,\overline{X}}^X$ can approximation the constant function, ${\Delta }_{\varphi ,\overline{X}}^X$ is modified as In the case of $T_y(\varphi ,y)=\sigma (w^{\top }x+b)$ and $w,x\in R^d,b\in R$, R.D.Nowak et al used (11) to design regularized learning frameworks (see [57]). An algorithm is provided by S.B.Lin et al [26] for designing such kind of network and is applied to construct regularized learning algorithms. In [5]$N_{\varphi ,\overline{X}}^X$ is used to construct deep neural network learning frameworks. The same type of investigations are given in [58,59] and [60].

In the present paper, we shall design the translation networks $N_{\varphi ,\overline{X}}^X$ by taking $X=S^{d-1}$, assuming $\overline{X}={\Omega }^{\left(m\right)}={\left\{x_j\right\}}_{j=1}^m\subset S^{d-1}$ satisfies equalities (8) and (9), and choosing the zonal translation $T_x(\varphi ,y)=\varphi \left(xy\right)$ with $\varphi$ being a given integrable function $\varphi$ on $[-1,1]$. Under these assumptions we shall provide a learning framework with $N_{\varphi ,{\Omega }^{\left(m\right)}}^{S^{d-1}}$ as the hypothesis space and give error analysis.

The contributions of the present paper are two folds. First, after absorbing the ideas of [34,36,37,38] and the successful experience of [11,25,27,29,50,51,61,62], we propose the concept of Marcinkiewicz-Zygmund Inequality Setting (MZIS) for the scattered nodes on the sphere unit, based on this as an assumption we show the reproducing property for the convolutional zonal translation network associated with the scattered nodes ${\Omega }^{\left(m\right)}$. Second, we give investigation on the kernel regularized neural network learning by combining classical the kernel approach and the convex analysis method, according to this method, the convergence rate given can be dimensional independent and capacity independent. Since the translation networks are produced by the zonal translations of convolutional kernels, we call them the convolutional zonal translation networks.

The paper is organized as follows. In Section 2 we first show the density for the zonal translations class and then show the reproducing property for the translation network ${\Delta }_{\varphi ,{\Omega }^{\left(m\right)}}^{S^{d-1}}$. In Section 3, we shall provide some results in the present paper, for example, a new regression learning framework and a learning setting, the error decomposition for the error analysis,and an estimate for the convergence rate. In Section 4, we shall give some lemmas which are used to prove the main results. The proofs for all the theorems and propositions are given in Section 5.

Throughout the paper, we write $A=O\left(B\right)$ if there is a positive constant C independent of A and B such that $A\le CB$.In particular, by $A=O\left(1\right)$ we show A is a bounded quantity. We write $A\sim B$ if both $A=O\left(B\right)$ and $B=O\left(A\right)$.

Let $\varphi \in L_{W_{\eta }}^1={\{\varphi :\parallel \varphi \parallel }_{1,W_{\eta }}={\int }_{-1}^1\left|\varphi \left(x\right)\right|W_{\eta }\left(x\right)dx<+\infty \},W_{\eta }\left(x\right)={(1-x^2)}^{\eta -{\displaystyle \frac{1}{2}}},\eta >-{\displaystyle \frac{1}{2}}$. Then H.N.Mhaskar et al constructed in [29] a sequence of approximation operators to show that the zonal translation class is density in $L^p\left(S^{d-1}\right)(1\le p\le +\infty )$ if $\widehat{a_l^{\eta }\left(\varphi \right)}\ne 0$ for all $l=0,1,2,\cdots ,$ where and $C_n^{\eta }\left(x\right)=p_n^{(\eta -{\displaystyle \frac{1}{2}},\eta -{\displaystyle \frac{1}{2}})}\left(x\right)$ is the n-th Legendre polynomial satisfies the orthogonal relation with $c_{\eta }={\displaystyle \frac{\Gamma (\eta +1)}{\Gamma \left(\frac{1}{2}\right)\Gamma (\eta +\frac{1}{2})}}$, $h_n^{\eta }={\displaystyle \frac{\eta }{n+\eta }}{C}_n^{\eta }\left(1\right)$ and it is known that (see it from (B.2.1),(B.2.2)and (B.5.1) of [63])$C_n^{\eta }\left(x\right)\le C_n^{\eta }\left(1\right)=n^{2\eta -1},x\in [-1,1].$ It follows that where $Z_l^{\eta }\left(t\right)={\displaystyle \frac{l+\eta }{\eta }}{C}_l^{\eta }\left(t\right),\eta ={\displaystyle \frac{d-2}{2}}$.

For a given real number $p\ge 1$,we denote by $L^p\left({\rho }_X\right)$ the class of ${\rho }_X$-measurable functions f satisfying ${\parallel f\parallel }_{L^p\left({\rho }_X\right)}=({\int }_X{\left|f\left(x\right)\right|}^{p}d{\rho }_X)<+\infty$.

Let $P_n^d$ denote the space of all homogeneous polynomials of degree n in d variables. We denote by $L^p\left(S^{d-1}\right)$ the class of all measurable functions defined on $S^{d-1}$ with the finite norm and for $p=+\infty$ we assume that $L^{+\infty }\left(S^{d-1}\right)$ is the space $C\left(S^{d-1}\right)$ of continuous functions on $S^{d-1}$ with the uniform norm.

For a given integer $n\ge 0$, the restriction to $S^{d-1}$ of a homogeneous harmonic polynomial of degree n is called a spherical harmonics of degree n. If $Y\in P_n^d$, then $Y\left(x\right)={\parallel x\parallel }^{n}Y\left(x^{\prime }\right),x^{\prime }={\displaystyle \frac{x}{\parallel x\parallel }}$, so that Y is determined by its restriction on the unit sphere. Let $H_n\left(S^{d-1}\right)$ denote the space of spherical harmonics of degree n. Then Spherical harmonics of different degrees are orthogonal on the unit sphere. For further properties about harmonics one can refer to [64].

For $n=0,1,2,\cdots ,$ let $\{Y_l^n:1\le l\le dim{H}_n\left(S^{d-1}\right)\}$ be an orthonormal basis of $H_n\left(S^{d-1}\right)$. Then where ${\omega }_{d-1}$ denotes the surface area of $S^{d-1}$ and ${\omega }_{d-1}={\displaystyle \frac{2{\pi }^{\frac{d}{2}}}{\Gamma \left(\frac{d}{2}\right)}}$. Furthermore, by (1.2.8) in [63] we have where $C_n^{\eta }\left(t\right)$ is the n-th generalized Legendre polynomial the same as in (12). Combining (12) and (13) we have

We now make a combination of the zonal translation network with the kernel regularized regression.

To prove the main results, we need some lemmas.

Lemma 4.1.Let $(H,\parallel ·\parallel )$ be a Hilbert space and ξ be a random variable on $(Z,\rho )$ with values in H and ${\left\{z_i\right\}}_{i=1}^m$ be independent samples drawers of ρ.Assume ${\parallel \xi \parallel }_H\le \tilde{M}<+\infty$ almost surely. Then, for any $0<\delta <1$, with confidence $1-\delta$,it holds

Proof. Find it from [74].

Lemma 4.2.Let $(H,\parallel ·{\parallel }_H)$ be a Hilbert space over X with respect to kernel K. If $(E,\parallel ·{\parallel }_E)$ and $(F,\parallel ·{\parallel }_F)$ are closed subspaces of H such that $E\perp F$ and $E⨁F=H$, then $K=L+M$, where L and M are the reproducing kernels of E and F respectively. Moreover, for $h=e+f,e\in E,f\in F$, we have

Proof. See Corollary 1 in Chapter 31 of [65] or the Theorem in Section 6 in part I of [75].

Lemma 4.3. There hold the following equalities: and

Proof of (28). By the equality we have Since $g\left(x\right)={\langle g,K_x(\varphi ;·)\rangle }_{{\mathcal{H}}_{{\Omega }^{\left(m\right)}}}$ and the definition of $G\widehat{a}teaux$ derivative, we have by above equality that We then have (28). By the same way we can have (29).