Similarity
Artificial Intelligence
Similarity Search, Part 5: Locality Sensitive Hashing (LSH) Introduction Shingling MinHashing LSH Function Error rate Conclusion Resources
Explore how similarity information might be incorporated into hash functionS is an issue where given a question the goal is to search out probably the most similar documents to it amongst all of the...
Artificial Intelligence
Similarity Search, Part 5: Locality Sensitive Hashing (LSH)
Explore how similarity information may be incorporated into hash functionSimilarity search is an issue where given a question the goal is to search out probably the most similar documents to it amongst all of...
Artificial Intelligence
Cosine Similarity for 1 Trillion Pairs of Vectors Motivation ChunkDot Chunk size calculation Memory and speed Usage Conclusion
Introducing ChunkDotpip install -U chunkdotCalculate the 50 most similar and dissimilar items for 100K items.import numpy as npfrom chunkdot import cosine_similarity_top_kembeddings = np.random.randn(100000, 256)# using all you system's memorycosine_similarity_top_k(embeddings, top_k=50)# most dissimilar items using...
Recent posts
