A simple model to calculate the embeddings for the Segment Anything Model. The purpose is to calculate the embeddings on a GPU powered Replicate instance, and then calculate the segments on a CPU-only machine or even via WASM directly in the user’s browser.

PhotograftAI is an example demo where I use this architecture. The model generates a zip containing the embeddings you can store as a file. To calculate the mask you load the embeddings and add them to your input (e.g. the point, rect, or prompt that you wanna mask) and run the model. To calculate the masks directly in a browser you, can follow meta’s ONNX notebook