Similarity search is the process of finding items in a dataset that are most similar to a given query item. It's widely used in:

• Recommendation systems

  (for example, suggesting similar movies)

• Image and video retrieval

• Natural language processing

  (for example, finding similar documents or sentences)

• Biometrics

  (for example, face recognition)

At the core of similarity search is a distance or similarity metric that quantifies how alike two data points are. The choice of metric depends on the nature of the data and the application

📏 Common Distance and Similarity Metrics

1. L2 Distance (Euclidean Distance)

The L2 distance between two vectors a and b is the square root of the sum of the squared differences between corresponding elements.

Here is mathematical overview:

Properties:

• The L2 distance measures the straight-line distance between two points in Euclidean space, following the principles of the Pythagorean theorem.

• It is sensitive to both the magnitude and direction of the vectors, meaning that changes in either can significantly affect the calculated distance.

• This distance metric is commonly used in applications involving spatial or geometric data, such as image analysis, computer vision, and geographic mapping.

Use Case:

• A common use case for L2 distance is determining the closest point to a given location in two-dimensional (2D) or three-dimensional (3D) space. This approach is frequently used in computer vision tasks, where spatial proximity between features or objects plays a critical role.

2. Dot Product (Inner Product) Similarity and Distance

Note: Dot product is a similarity metric: the greater the dot product, the more similar the vectors are. In order to convert dot product to a distance, one calculates the negative of the dot product: the greater the negative dot product, the greater the distance between two vectors.

Properties:

• The dot product of two vectors can be positive, negative, or zero, depending on the angle between them.

• Larger dot product values typically indicate a higher degree of similarity between the vectors, especially when they are pointing in similar directions.

• If a distance-like metric is needed, the negative of the dot product can be used. In this case, larger (more negative) values correspond to greater dissimilarity or distance.

• The dot product is sensitive to both the magnitude and direction of the vectors, meaning that changes in either will affect the result.

• It is frequently used in machine learning models, such as in neural networks for computing activations or in matrix factorization for recommendation systems.

Use Case: When the length of the vector (representing something such as relevance, confidence, or quantity) is meaningful. For instance, in recommendation systems, the direction of vectors typically indicates two products are about the same topic, but larger magnitudes might indicate that a product is more popular. In this case using dot product similarity makes sense, because one would want to recommend the products that are not only about the same topic, but also popular.

3. Cosine Similarity and Distance

Properties:

• Cosine similarity focuses on the orientation of vectors rather than their magnitude, measuring the cosine of the angle between them.

• It is particularly well-suited for high-dimensional, sparse data, such as text embeddings or term-frequency vectors in natural language processing.

• This metric is invariant to vector length, meaning that scaling a vector up or down does not affect the similarity score.

Use Case:

• A common use case for cosine similarity is measuring document similarity in natural language processing (NLP), where it helps identify texts with similar content regardless of their length.