pablodawson / segment-anything-automatic

SegmentAnything Model (SAM) automatic mask generator

Cold

Public
4.1K runs
T4
GitHub
Paper
License

Iterate in the playground

Run with an API

Playground API Examples README Versions

Input

image

*file

Input image

resize_width

integer

The width to resize the image to before running inference.

Default: 1024

points_per_side

integer

The number of points to be sampled along one side of the image. The total number of points is points_per_side**2. If None, point_grids must provide explicit point sampling.

Default: 32

pred_iou_thresh

number

A filtering threshold in [0,1], using the model's predicted mask quality.

Default: 0.88

stability_score_thresh

number

A filtering threshold in [0,1], using the stability of the mask under changes to the cutoff used to binarize the model's mask predictions.

Default: 0.95

stability_score_offset

number

The amount to shift the cutoff when calculated the stability score.

Default: 1

box_nms_thresh

number

The box IoU cutoff used by non-maximal suppression to filter duplicate masks.

Default: 0.7

crop_n_layers

integer

If >0, mask prediction will be run again on crops of the image. Sets the number of layers to run, where each layer has 2**i_layer number of image crops

Default: 0

crop_nms_thresh

number

The box IoU cutoff used by non-maximal suppression to filter duplicate masks between different crops.

Default: 0.7

crop_overlap_ratio

number

Sets the degree to which crops overlap. In the first crop layer, crops will overlap by this fraction of the image length. Later layers with more crops scale down this overlap.

Default: 0.3413333333333333

crop_n_points_downscale_factor

integer

The number of points-per-side sampled in layer n is scaled down by crop_n_points_downscale_factor**n.

Default: 1

min_mask_region_area

integer

If >0, postprocessing will be applied to remove disconnected regions and holes in masks with area smaller than min_mask_region_area.

Default: 0

Run this model in Node.js with one line of code:

npx create-replicate --model=pablodawson/segment-anything-automatic

or set up a project from scratch

Install Replicate’s Node.js client library:

npm install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import and set up the client:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

Run pablodawson/segment-anything-automatic using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

const output = await replicate.run(
  "pablodawson/segment-anything-automatic:14fbb04535964b3d0c7fad03bb4ed272130f15b956cbedb7b2f20b5b8a2dbaa0",
  {
    input: {
      image: "https://replicate.delivery/pbxt/IbLtTz5PFfyk5W9GZCCKXyiyldxQyRGhmLlGo4zdCf2snIbW/chameleon.jpg",
      resize_width: 1080,
      crop_n_layers: 0,
      box_nms_thresh: 0.7,
      crop_nms_thresh: 0.7,
      points_per_side: 32,
      pred_iou_thresh: 0.88,
      crop_overlap_ratio: 0.3413333333333333,
      min_mask_region_area: 30,
      stability_score_offset: 1,
      stability_score_thresh: 0.95,
      crop_n_points_downscale_factor: 1
    }
  }
);

// To access the file URL:
console.log(output.url()); //=> "http://example.com"

// To write the file to disk:
fs.writeFile("my-image.png", output);

To learn more, take a look at the guide on getting started with Node.js.

Install Replicate’s Python client library:

pip install replicate

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Import the client:

import replicate

Run pablodawson/segment-anything-automatic using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

output = replicate.run(
    "pablodawson/segment-anything-automatic:14fbb04535964b3d0c7fad03bb4ed272130f15b956cbedb7b2f20b5b8a2dbaa0",
    input={
        "image": "https://replicate.delivery/pbxt/IbLtTz5PFfyk5W9GZCCKXyiyldxQyRGhmLlGo4zdCf2snIbW/chameleon.jpg",
        "resize_width": 1080,
        "crop_n_layers": 0,
        "box_nms_thresh": 0.7,
        "crop_nms_thresh": 0.7,
        "points_per_side": 32,
        "pred_iou_thresh": 0.88,
        "crop_overlap_ratio": 0.3413333333333333,
        "min_mask_region_area": 30,
        "stability_score_offset": 1,
        "stability_score_thresh": 0.95,
        "crop_n_points_downscale_factor": 1
    }
)
print(output)

To learn more, take a look at the guide on getting started with Python.

Set the REPLICATE_API_TOKEN environment variable:

export REPLICATE_API_TOKEN=<paste-your-token-here>

Find your API token in your account settings.

Run pablodawson/segment-anything-automatic using Replicate’s API. Check out the model's schema for an overview of inputs and outputs.

curl -s -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d $'{
    "version": "pablodawson/segment-anything-automatic:14fbb04535964b3d0c7fad03bb4ed272130f15b956cbedb7b2f20b5b8a2dbaa0",
    "input": {
      "image": "https://replicate.delivery/pbxt/IbLtTz5PFfyk5W9GZCCKXyiyldxQyRGhmLlGo4zdCf2snIbW/chameleon.jpg",
      "resize_width": 1080,
      "crop_n_layers": 0,
      "box_nms_thresh": 0.7,
      "crop_nms_thresh": 0.7,
      "points_per_side": 32,
      "pred_iou_thresh": 0.88,
      "crop_overlap_ratio": 0.3413333333333333,
      "min_mask_region_area": 30,
      "stability_score_offset": 1,
      "stability_score_thresh": 0.95,
      "crop_n_points_downscale_factor": 1
    }
  }' \
  https://api.replicate.com/v1/predictions

To learn more, take a look at Replicate’s HTTP API reference docs.

Output

Generated in

10.5 seconds

Tweak it Report

Examples

View more examples

Run time and cost

This model costs approximately $0.0035 to run on Replicate, or 285 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 16 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Segment Anything

Meta AI Research, FAIR

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alex Berg, Wan-Yen Lo, Piotr Dollar, Ross Girshick

[Paper] [Project] [Demo] [Dataset]

The Segment Anything Model (SAM) produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image. It has been trained on a dataset of 11 million images and 1.1 billion masks, and has strong zero-shot performance on a variety of segmentation tasks.

Note: I’m not the author of this model. Please refer to their official page for questions. Here I use the included “SamAutomaticMaskGenerator” to auto-generate masks.