There’s a core concept in AI called high-dimensional space. Here’s one way to wrap your head around this concept: think about people as being high-dimensional. For example, take famous entrepreneurs. You can think about when they were born, where they were born, what companies they founded, etc. Each of these attributes is like a dimension of that person. These dimensions become difficult to untangle when you think about different people because someone might be similar in some ways but very different in others – and this is the kind of thing you can use AI for.

With AI, the computer isn’t told the meaning of these dimensions. It just sees them as numbers, and it sees each set of numbers as a data point; by looking across all of these dimensions at once, it is able to place related points closer together in high-dimensional space.

To provide a more intuitive way to explore this process, we’ve built a simple web application for interactive visualization and analysis of image and text datasets. This visualization tool may be referred to as an Embedding Explorer.

Let’s consider the following concrete example: a data set in which words are treated as high-dimensional data points. The important thing to remember is that we haven’t told the computer the meaning of words. Instead, we’ve shown it millions of sentences as examples of how words get used. In the Embedding Explorer, each dot represents one word. Each word is a data point with high dimensions (depends on the configurations established when training the AI models).

Using a technique called t-SNE (short for T-distributed Stochastic Neighbour Embedding), the computer clusters words together that it considers related; clusters are formed based on the meaning, even though we’ve never taught it the meaning of words.

This approach works for more than just words. For example, you can also treat an image as a high-dimensional data point.

MNIST is a dataset in which lots of people hand-wrote digits between 0 and 9. People write in all kinds of different ways, so the question is, instead of us needing to manually code rules to anticipate all the ways that people could write, could a computer figure it out itself using AI?

Each image in the MNIST dataset is 784 pixels (28 by 28); the computer treats each pixel as a dimension. Again, using t-SNE, the AI clusters theses images in a high-dimensional space. We’ve colour-coded each cluster so that it’s easier for us to see what’s going on, and you can see groups of digits clustering together. It’s learned something about the meaning of these digits.

Our Embedding Explorer demo comes with five different datasets for you to play with, including two image-based datasets and two text-based datasets. For demonstration purposes, all data was pre-generated using a limited number of input parameters – a subset of 3000 samples – and thus displayed instantly.

If our demo gets you to come up with some good ideas for your business, don’t hesitate to contact us to discuss how our team of experienced professionals may help bring your idea to life.