HACKER Q&A
📣 jzombie

Semantic Vector Searching in WASM?


I'm working on a simple markdown-driven CMS, in Python, that outputs to static HTML files that can be served directly from GitHub (as an example).

I'm interested in implementing local-only search of this content, w/o using a backend, and preferably w/o cobbling together some NLP algorithms in JavaScript.

I've also been training ML models for other use cases and experimenting around w/ vector databases.

I started looking into creating embeddings using Python, from my markdown content, potentially using word2vec or finalfusion to help me get those embeddings into a Rust environment that I can compile w/ Rust into WASM, and use cosine similarity for my search.

Now, I neither have any Rust nor WASM experience, but that doesn't really deter me much. I'm just curious if the overhead would be significant, or if I should skip the semantic embeddings and do a different type of embedding such as TD-IDF.

  👤 cmcollier Accepted Answer ✓
As a general rule (for now), you'll get the best search result for your dev time, with straight ahead BM25 via a js lib.

In terms of overhead, with lower doc counts there's not much overhead with embeddings and knn/ann. Imagine 384 floats per doc or whatever embedding size. At scale it becomes more problematic and less comparable.

With all that said, messing around with vector ops and WASM sounds more fun :)

👤 TachyonicBytes
Would this[1] library help you? It's a Rust vector similarity search engine, written to be compiled to Wasm. I discovered it through articles like these[2].

    [1] https://github.com/tantaraio/voy
    [2] https://dawchihliou.github.io/articles/share-rust-types-with-typescript-for-webassembly-in-30-seconds
👤 dhruv_anand
Not sure if this helps, but you can create embeddings using transformers.js (feature extraction pipeline) in the browser.