Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction

Manifold Elastic Net is the work we have done one year ago and it is about to be published on Data Mining and Knowledge Discovery (Springer) recently. You can either find it on the website of DMKD:

http://www.springerlink.com/content/bk6301u1938104q6/

or the arXiv:

http://arxiv.org/abs/1007.3564

An earlier published work about Manifold Elastic Net on 2009 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2009) can be found on:

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5346879

The main contribution of this paper is to formulate the sparse dimension reduction problem into an penalized least square problem rather than a sparse PCA problem. For the former, there exists tremendous fast algorithms in compressed sensing and statistics. For the latter, the existing algorithms are not very satisfying in speed and the optimality of the solution.

Sparse dimension reduction can improve the dimension reduction results in several aspects, e.g., feature selection with clear interpretation, acceleration of subsequent machine learning tasks because of sparse representation. There is an interesting experiment in the paper to sequentially select the significant features on human face by using Manifold Elastic Net:

The solution paths of Manifold Elastic Net and feature selection on human face

The solution paths of Manifold Elastic Net and feature selection on human face

Another advantage of Manifold Elastic Net is that many existing manifold learning methods can be included into its framework. I also recommend Section 2.3 Classification error minimization of the paper, in which a novel indicator (label) matrix coding method is proposed. It can be used in other discriminative dimension reduction problems to obtain a least square regression formulation.

The framework of Manifold Elastic Net can also extended to sparse coding and manifold based compressed sensing. I believe there will be some future works about these important extensions.

Advertisements

About tianyizhou

Research Assistant at University Washington, Seattle, Working on Machine Learning & Statistics in MODE lab leaded by Prof. Carlos Guestrin, and MELODI lab leaded by Prof. Jeff Bilmes.
This entry was posted in Tianyi's work and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s