🍃Locally Linear Embedding (LLE)
LLE (Locally Linear Embedding) also projects the data into a lower dimensional space while preserving the local neighborhoods' information.
Similar to Isomap, it applies KNN to find the k nearest neighbors for every data point.
It then constructs a cost function to minimize the total absolute differences between each data point and its weighted neighbors. The sum of the neighbors' weights is set to 1 for each data point, and a weight matrix is created where each data point's weights are determined by minimizing the cost function.
When seeking a lower-dimensional representation, LLE tries to build a similar cost function where the data points are replaced by their lower-dimensional representations. The weights from step 2 are preserved, and LLE finds the lower-dimensional space that minimizes this new cost function.
To understand more details of LLE, check here.
The code of LLE is as simple as other dimensional reduction methods. One note for parameter method:
By default, the value is "standard", which is using the algorithm described above.
Value "modified", "hessian" and "lsta" are modified versions of the algorithm, aiming at improving the regularization of LLE. Check wiki to learn more about them. Most of the time, "modified" is a recommended choice.
But look at the data plot after LLE dimensional reduction, too simple to be true, right? 😅
As we can see, Isomap, MDS, t-SNE and LLE are all focusing on maintaining the local structure while doing the dimensional reduction, then what about the global structure? Is there any method takes care of both local and global data structure?
Last updated