t-SNE
Description
t-SNE (t-distributed Stochastic Neighbour Embedding) turns high-dimensional data into a two- or three-dimensional map that can be plotted and inspected. It is a non-linear dimensionality reduction method, so it can unfold curved structure that linear methods such as principal component analysis, which find only straight-line directions of greatest variation, would flatten. For each pair of points it works out how likely they are to be neighbours in the original space. It then moves the points around on the map until the same neighbour probabilities hold there as closely as possible. Because it concentrates on keeping near neighbours together, clusters of similar items usually appear as visible groups, whether the points are raw data or a model's internal representations such as embeddings.
Example Use Cases
Explainability
A genomics group maps the gene-expression profiles of several thousand tumour samples with t-SNE and checks whether samples of the same clinical subtype form groups before training a subtype classifier. Subtypes that overlap on the map tell the team where the classifier is likely to confuse them.
An image-classification team maps the final hidden-layer activations of a trained network with t-SNE, colouring points by predicted class. It checks whether cars, animals and furniture fall into separate regions and which classes the network places close together, before reviewing misclassifications.
A trust-and-safety team maps the sentence embeddings used by a content-moderation model, colouring each post by the moderators' label. It checks whether flagged posts form their own group or sit among harmless posts on the same topic, and so decides whether the embedding needs retraining.
Limitations
- Results depend on random initialisation and a non-convex optimisation, so separate runs can lay clusters out differently. Fix the random seed, or start from a principal component analysis, and compare several runs before drawing conclusions.
- Only local neighbourhoods are reliably preserved. Cluster sizes and the gaps between clusters carry no dependable meaning, so reading distance between groups as similarity will mislead a reviewer.
- Perplexity (roughly the number of neighbours each point considers) and the number of iterations change the picture a great deal. Low settings can make random noise look like clusters, so a single plot at a single setting is weak evidence.
- Standard t-SNE needs time and memory that grow with the square of the number of points, so datasets beyond roughly 10,000 points need an approximate variant or a subsample.
Resources
Research Papers
Visualizing Data using t-SNE
We present a new technique called "t-SNE" that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images ofobjects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large data sets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of data sets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the data sets.