On the low dimensional projections of high-dimensional point clouds
On the low dimensional projections of high-dimensional point clouds
Given a cloud of $n$ data points in $\mathbb{R}^d$, consider all projections onto $m$-dimensional subspacesÌýofÌý$\mathbb{R}^d$ and, for each such projection, the empirical distribution of theÌýprojected points.ÌýWhat does this collection of probability distributions look like when $n,d$Ìýgrow large?ÌýWe consider this question under the model in which the points are i.i.d.ÌýstandardÌýGaussian vectors, focusing on the asymptotic regime in which $n,d$ diverge,Ìýwith $n/d$Ìýconverging to a finite non-zero value, while $m$ is fixed. Denoting by $F_m$ theÌýset ofÌýprobability distributions in $\mathbb{R}^m$ that arise as low-dimensional projectionsÌýin this limit,ÌýI will present new inner and outer bounds on $F_m$.ÌýIn particular, these bounds determine the Wasserstein radius of $F_m$ up toÌýlogarithmic factors,Ìýand determine it exactly for $m=1$. I will also present bounds in terms ofÌýKullback-LeiblerÌýdivergence and Rényi information dimension.
The previous question has application to unsupervised learning methods,Ìýsuch as projection pursuit and independent component analysis. We introduceÌýaÌýversion of the same problem that is relevant for supervised learning. As anÌýapplication,Ìýwe establish an upper bound on the interpolation threshold of two-layersÌýneural networksÌýwith $m$ hidden neurons.
[Based on joint work with Kangjie Zhou]