Preprint
Article

Noncommutative Donoho-Elad-Gribonval-Nielsen-Fuchs Sparsity Theorem

Altmetrics

Downloads

142

Views

92

Comments

0

This version is not peer-reviewed

Submitted:

30 August 2024

Posted:

30 August 2024

You are already at the latest version

Alerts
Abstract
Breakthrough Sparsity Theorem, derived independently by Donoho and Elad \textit{[Proc. Natl. Acad. Sci. USA, 2003]}, Gribonval and Nielsen \textit{[IEEE Trans. Inform. Theory, 2003]} and Fuchs \textit{[IEEE Trans. Inform. Theory, 2004]} says that unique sparse solution to NP-Hard $\ell_0$-minimization problem can be obtained using unique solution of P-Type $\ell_1$-minimization problem. In this paper, we derive noncommutative version of their result using frames for Hilbert C*-modules.
Keywords: 
Subject: Computer Science and Mathematics  -   Analysis

1. Introduction

Let H be a finite dimensional Hilbert space over K ( C or R ). A finite collection { τ j } j = 1 n in H is said to be a frame (also known as dictionary) [1,2] for H if there are a , b > 0 such that
a h 2 j = 1 n | h , τ j | 2 b h 2 , h H .
A frame { τ j } j = 1 n for H is said to be normalized if τ j = 1 for all 1 j n . Note that any frame can be normalized by dividing each element by its norm. Given a frame { τ j } j = 1 n for H , we define the analysis operator
θ τ : H h θ τ h ( h , τ j ) j = 1 n K n .
Adjoint of the analysis operator is known as the synthesis operator whose expression is
θ τ * : K n ( a j ) j = 1 n θ τ * ( a j ) j = 1 n j = 1 n a j τ j H .
Given d K n , let d 0 be the number of nonzero entries in d. Following 0 -minimization problem appears in many of electronic devices.
Problem 1.
Let { τ j } j = 1 n be a normalized frame for H . Given h H , solve
minimize { d 0 : d K n } subject to θ τ * d = h .
Recall that c K n is said to be a unique solution to Problem 1 if it satisfies following two conditions.
(i)
θ τ * c = h .
(ii)
If d K n satisfies θ τ * d = h , then
d 0 > c 0 .
In 1995, Natarajan showed that Problem 1 is NP-Hard [3,4]. As the operator θ τ * is surjective, for a given h H , there is a d K n such that θ τ * d = h . Thus the central problem is to say when the solution to Problem 1 is unique. It is well-known that [5,6,7] following problem is the closest convex relaxation problem to Problem 1.
Problem 2.
Let { τ j } j = 1 n be a normalized frame for H . Given h H , solve
minimize { d 1 : d K n } subject to θ τ * d = h .
There are several linear programmings available to obtain solution of Problem 2 and it is a P-problem [8,9,10].
Most important result which shows that by solving Problem 2 we also get a solution to Problem 1 is obtained independently by Donoho and Elad [11], Gribonval and Nielsen [12] and Fuchs [13,14] is the following.
Theorem 3.
[11,12,13,14,15,16] (Donoho-Elad-Gribonval-Nielsen-Fuchs Sparsity Theorem) Let { τ j } j = 1 n be a normalized frame for H . If h H can be written as h = θ τ * c for some c K n satisfying
c 0 < 1 2 1 + 1 max 1 j , k n , j k | τ j , τ k | ,
then c is the unique solution to Problem 2 and Problem 1.
Our fundamental motivation comes from the following question: What is the noncommutative analogue of Theorem 3? This is then naturally connected with the notion of Hilbert C*-modules which are first introduced by Kaplansky [17] for modules over commutative C*-algebras and later developed for modules over arbitrary C*-algebras by Paschke [18] and Rieffel [19]. We end the introduction by recalling the definition of Hilbert C*-modules.
Definition 4.
[17,18,19] Let A be a unital C*-algebra. A left module E over A is said to be a (left) Hilbert C*-module if there exists a map · , · : E × E A such that the following hold.
(i)
x , x 0 , x E . If x E satisfies x , x = 0 , then x = 0 .
(ii)
x + y , z = x , z + y , z , x , y , z E .
(iii)
a x , y = a x , y , x , y E , a A .
(iv)
x , y = y , x * , x , y E .
(v)
E is complete w.r.t. the norm x x , x , x E .

2. Noncommutative Donoho-Elad-Gribonval-Nielsen-Fuchs Sparsity Theorem

Observe that the notion of frames is needed for Theorem 3. Thus we want noncommutative frames. These are introduced in 2002 by Frank and Larson in their seminal paper [20]. We begin by recalling the definition of noncommutative frames for Hilbert C*-modules. This notion is already well-developed in parallel with Hilbert space frame theory [21,22,23]. In the paper, we consider only finite rank modules.
Definition 5.
[20] Let E be a Hilbert C*-module over a unital C*-algebra A . A collection { τ j } j = 1 n in E is said to be a (modular)framefor E if there are real a , b > 0 such that
a x , x j = 1 n x , τ j τ j , x b x , x , x E .
A collection { τ j } j = 1 n in a Hilbert C*-module E over unital C*-algebra A with identity 1 is said to have unit inner product if
τ j , τ j = 1 , 1 j n .
Let A be a unital C*-algebra. For n N , let A n be the standard left Hilbert C*-module over A with inner product
( a j ) j = 1 n , ( b j ) j = 1 n j = 1 n a j b j * , ( a j ) j = 1 n , ( b j ) j = 1 n A n .
Hence norm on A n is
( a j ) j = 1 n 2 j = 1 n a j a j * 1 2 , ( a j ) j = 1 n A n .
We define
( a j ) j = 1 n 1 j = 1 n a j , ( a j ) j = 1 n A n .
A frame { τ j } j = 1 n for E gives the modular analysis morphism
θ τ : E x θ τ x ( x , τ j ) j = 1 n A n
and the modular synthesis morphism
θ τ * : A n ( a j ) j = 1 n θ τ * ( a j ) j = 1 n j = 1 n a j τ j E .
With these notions, we generalize Problems 1 and 2. In the entire paper, E denotes a finite rank Hilbert C*-module over a unital C*-algebra A .
Problem 6.
Let { τ j } j = 1 n be a unit inner product frame for E . Given x E , solve
minimize { d 0 : d A n } subject to θ τ * d = x .
Problem 7.
Let { τ j } j = 1 n be a unit inner product frame for E . Given x E , solve
minimize { d 1 : d A n } subject to θ τ * d = x .
A very powerful property used to show Theorem 3 is the notion of null space property (see [15,24]). We now define the same property for Hilbert C*-modules. We use following notations. Let { e j } j = 1 n be the canonical basis for A n . Given M { 1 , , n } and d = ( d j ) j = 1 n A n , define
d M j M d j e j .
Definition 8.
A unit inner product frame { τ j } j = 1 n for E is said to have the (modular)null space property(we write NSP) of order k { 1 , , n } if for every M { 1 , , n } with o ( M ) k , we have
d M 1 < 1 2 d 1 , d ker ( θ τ * ) , d 0 .
We first relate NSP with Problem 7.
Theorem 9.
Let { τ j } j = 1 n be a unit inner product frame for E and let 1 k n . The following are equivalent.
(i)
If x E can be written as x = θ τ * c for some c A n satisfying c 0 k , then c is the unique solution to Problem 7.
(ii)
{ τ j } j = 1 n satisfies the NSP of order k.
Proof. 
(i)
⇒ (ii) Let M { 1 , , n } with o ( M ) k and let d ker ( θ τ * ) , d 0 . Then we have
0 = θ τ * d = θ τ * ( d M + d M c ) = θ τ * ( d M ) + θ τ * ( d M c )
which gives
θ τ * ( d M ) = θ τ * ( d M c ) .
Define c d M A n and x θ τ * ( d M ) . Then we have c 0 o ( M ) k and
x = θ τ * c = θ τ * ( d M c ) .
By assumption (i), we then have
c 1 = d M 1 < d M c 1 = d M c 1 .
Rewriting previous inequality gives
d M 1 < d 1 d M 1 d M 1 < 1 2 d 1 .
Hence { τ j } j = 1 n satisfies the NSP of order k.
1.
⇒ (i) Let x E can be written as x = θ τ * c for some c A n satisfying c 0 k . Define M supp ( c ) . Then o ( M ) = c 0 k . By assumption (ii), we then have
d M 1 < 1 2 d 1 , d ker ( θ τ * ) , d 0 .
Let b A n be such that x = θ τ * b and b c . Define a b c A n . Then θ τ * a = θ τ * b θ τ * c = x x = 0 and hence a ker ( θ τ * ) , a 0 . Using Inequality (1), we get
a M 1 < 1 2 a 1 a M 1 < 1 2 ( a M 1 + a M c 1 ) a M 1 < a M c 1 .
Using Inequality (2) and the information that c is supported on M, we get
b 1 c 1 = b M 1 + b M c 1 c M 1 c M c 1 = b M 1 + b M c 1 c M 1 = b M 1 + ( b c ) M c 1 c M 1 = b M 1 + a M c 1 c M 1 > b M 1 + a M 1 c M 1 b M 1 + ( b c ) M 1 c M 1 b M 1 b M 1 + c M 1 c M 1 = 0 .
Hence c is the unique solution to Problem 7.
Using Theorem 9 we obtain modular version of Theorem 3.
Theorem 10.
Let { τ j } j = 1 n be a unit inner product frame for E . If x E can be written as x = θ τ * c for some c A n satisfying
c 0 < 1 2 1 + 1 max 1 j , k n , j k τ j , τ k ,
then c is the unique solution to Problem 7.
Proof. 
We show that { τ j } j = 1 n satisfies the NSP of order k c 0 . Then Theorem 9 says that c is the unique solution to Problem 7. Let x E can be written as x = θ τ * c for some c A n satisfying c 0 k . Let M { 1 , , n } with o ( M ) k and let d = ( d j ) j = 1 n ker ( θ τ * ) , d 0 . Then we have
θ τ θ τ * d = 0 .
For each fixed 1 k n , above equation gives
0 = θ τ θ τ * ( d j ) j = 1 n , e k = θ τ * ( d j ) j = 1 n , θ τ * e k = θ τ * ( d j ) j = 1 n , τ k = j = 1 n d j τ j , τ k = d k τ k , τ k + j = 1 , j k n d j τ j , τ k = d k + j = 1 , j k n d j τ j , τ k .
Therefore
d k = j = 1 , j k n d j τ j , τ k , 1 k n .
By taking norm,
d k = j = 1 , j k n d j τ j , τ k j = 1 , j k n d j τ j , τ k j = 1 , j k n d j τ j , τ k max 1 j , k n , j k τ j , τ k j = 1 , j k n d j = max 1 j , k n , j k τ j , τ k j = 1 n d j d k = max 1 j , k n , j k τ j , τ k d 1 d k , 1 k n .
By rewriting above inequality we get
1 + 1 max 1 j , k n , j k τ j , τ k d k d 1 , 1 k n .
Summing Inequality (4) over M leads to
1 + 1 max 1 j , k n , j k τ j , τ k d M 1 = 1 + 1 max 1 j , k n , j k τ j , τ k k M d k d 1 k M 1 = d 1 o ( M ) .
Finally using Inequality (3)
d M 1 1 + 1 max 1 j , k n , j k τ j , τ k 1 d 1 o ( M ) 1 + 1 max 1 j , k n , j k τ j , τ k 1 d 1 k = 1 + 1 max 1 j , k n , j k τ j , τ k 1 d 1 c 0 < 1 2 d 1 .
Hence { τ j } j = 1 n satisfies the NSP of order k. □
Theorem 11.
(Noncommutative Donoho-Elad-Gribonval-Nielsen-Fuchs Sparsity Theorem) Let { τ j } j = 1 n be a unit inner product frame for E . If x E can be written as x = θ τ * c for some c A n satisfying
c 0 < 1 2 1 + 1 max 1 j , k n , j k τ j , τ k ,
then c is the unique solution to Problem 6.
Proof. 
Theorem 10 says that c is the unique solution to Problem 7. Let d A n be such that x = θ τ * d . We claim that d 0 > c 0 . If this fails, we must have d 0 c 0 . We then have
d 0 < 1 2 1 + 1 max 1 j , k n , j k τ j , τ k .
Theorem 10 again says that d is also the unique solution to Problem 7. Therefore we must have c 1 < d 1 and c 1 > d 1 which is a contradiction. So claim holds and we have d 0 > c 0 . □

References

  1. Benedetto, J.J.; Fickus, M. Finite normalized tight frames. Adv. Comput. Math. 2003, 18, 357–385. [Google Scholar] [CrossRef]
  2. Han, D.; Kornelson, K.; Larson, D.; Weber, E. Frames for undergraduates; Vol. 40, Student Mathematical Library, American Mathematical Society, Providence, RI, 2007; pp. xiv+295. [CrossRef]
  3. Natarajan, B.K. Sparse approximate solutions to linear systems. SIAM J. Comput. 1995, 24, 227–234. [Google Scholar] [CrossRef]
  4. Foucart, S.; Rauhut, H. A mathematical introduction to compressive sensing; Applied and Numerical Harmonic Analysis, Birkhäuser/Springer, New York, 2013; pp. xviii+625. [CrossRef]
  5. Chen, S.S.; Donoho, D.L.; Saunders, M.A. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 1998, 20, 33–61. [Google Scholar] [CrossRef]
  6. Donoho, D.L.; Huo, X. Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 2001, 47, 2845–2862. [Google Scholar] [CrossRef]
  7. Bruckstein, A.M.; Donoho, D.L.; Elad, M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 2009, 51, 34–81. [Google Scholar] [CrossRef]
  8. Xue, G.; Ye, Y. An efficient algorithm for minimizing a sum of p-norms. SIAM J. Optim. 2000, 10, 551–579. [Google Scholar] [CrossRef]
  9. Terlaky, T. On lp programming. European J. Oper. Res. 1985, 22, 70–100. [Google Scholar] [CrossRef]
  10. Tillmann, A.M. Equivalence of Linear Programming and Basis Pursuit. Proc. Appl. Math. Mech. 2015, 15, 735–738. [Google Scholar] [CrossRef]
  11. Donoho, D.L.; Elad, M. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proc. Natl. Acad. Sci. USA 2003, 100, 2197–2202. [Google Scholar] [CrossRef] [PubMed]
  12. Gribonval, R.; Nielsen, M. Sparse representations in unions of bases. IEEE Trans. Inform. Theory 2003, 49, 3320–3325. [Google Scholar] [CrossRef]
  13. Fuchs, J.J. On sparse representations in arbitrary redundant bases. IEEE Trans. Inform. Theory 2004, 50, 1341–1344. [Google Scholar] [CrossRef]
  14. Fuchs, J.J. More on sparse representations in arbitrary bases. IFAC Proceedings Volumes 2003, 36, 1315–1320. [Google Scholar] [CrossRef]
  15. Kutyniok, G. Data separation by sparse representations. In Compressed sensing; Cambridge Univ. Press, Cambridge, 2012; pp. 485–514. [CrossRef]
  16. Elad, M. Sparse and redundant representations : From theory to applications in signal and image processing; Springer, New York, 2010; pp. xx+376. [CrossRef]
  17. Kaplansky, I. Modules over operator algebras. Amer. J. Math. 1953, 75, 839–858. [Google Scholar] [CrossRef]
  18. Paschke, W.L. Inner product modules over B*-algebras. Trans. Amer. Math. Soc. 1973, 182, 443–468. [Google Scholar] [CrossRef]
  19. Rieffel, M.A. Induced representations of C*-algebras. Advances in Math. 1974, 13, 176–257. [Google Scholar] [CrossRef]
  20. Frank, M.; Larson, D.R. Frames in Hilbert C*-modules and C*-algebras. J. Operator Theory 2002, 48, 273–314. [Google Scholar]
  21. Raeburn, I.; Thompson, S.J. Countably generated Hilbert modules, the Kasparov stabilisation theorem, and frames with Hilbert modules. Proc. Amer. Math. Soc. 2003, 131, 1557–1564. [Google Scholar] [CrossRef]
  22. Arambašić, L. On frames for countably generated Hilbert C*-modules. Proc. Amer. Math. Soc. 2007, 135, 469–478. [Google Scholar] [CrossRef]
  23. Han, D.; Jing, W.; Mohapatra, R.N. Perturbation of frames and Riesz bases in Hilbert C*-modules. Linear Algebra Appl. 2009, 431, 746–759. [Google Scholar] [CrossRef]
  24. Cohen, A.; Dahmen, W.; DeVore, R. Compressed sensing and best k-term approximation. J. Amer. Math. Soc. 2009, 22, 211–231. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated