Vision Segmentation (SAM3 Image/Video)

Image segmentation with POST /v1/vision-segment/predictions and video segmentation with /v1/vision-segment/video, powered by SAM3.

Inference Space provides vision segmentation powered by SAM3 (Segment Anything Model 3): perform instance segmentation on images or videos with text, point, or box prompts, and receive masks, overlays, bounding boxes, and confidence scores.

Image segmentation: POST https://ai.inf.space/v1/vision-segment/predictions, application/json.
Video segmentation: POST https://ai.inf.space/v1/vision-segment/video, application/json.
Default model: facebook/sam3; default provider: wujie (self-hosted).

The private deployment API is identical; only replace the Base domain → Enterprise private deployment.

Authentication and Base

Host: Vision segmentation is a master/control-plane capability at https://ai.inf.space; send curl and SDK requests to the documented /v1/vision-segment/* paths.
Authentication header: Authorization: Bearer $TOS_API_KEY. Keys start with gk_.
scope: ai:vision-segment is required. Select it when creating the key; the ai:* wildcard also works.

See Authentication.

Request structure

Both endpoints share the following outer structure:

{
  "provider": "wujie",
  "model": "facebook/sam3",
  "input": { /* see below */ }
}

provider: default wujie (self-hosted SAM3).
model: default facebook/sam3.
input: Segmentation parameters; fields differ between images and videos.

input.image / input.video accept either an https://... URL or a data:image/*;base64,... / data:video/*;base64,... base64 data URI.

Image segmentation

Send JSON to /v1/vision-segment/predictions. input fields:

Field	Type	Required	Default	Description
`image`	string	Yes	—	HTTPS URL or `data:image/*` base64 URI
`prompt`	string	No	`person`	Text prompt describing the target to segment
`threshold`	number	No	`0.5`	Confidence threshold, 0–1
`point_prompts`	array	No	—	Point prompts; each item is `{ x, y, positive }`
`box_prompts`	array	No	—	Box prompts; each item is `{ x_min, y_min, x_max, y_max }`
`mask_color`	string	No	`green`	Mask color: `green` / `red` / `blue` / `yellow` / `cyan` / `magenta`
`mask_opacity`	number	No	`0.5`	Mask opacity, 0–1
`save_overlay`	boolean	No	`false`	Whether to output an overlay image
`mask_only`	boolean	No	`false`	Output only the mask
`return_zip`	boolean	No	`true`	Whether to package multi-file results as a zip
`include_scores`	boolean	No	`false`	Whether to return `scores`
`include_boxes`	boolean	No	`false`	Whether to return `boxes`

  curl "https://ai.inf.space/v1/vision-segment/predictions" \
  -H "Authorization: Bearer $TOS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "wujie",
    "model": "facebook/sam3",
    "input": {
      "image": "https://example.com/street.jpg",
      "prompt": "person",
      "threshold": 0.5,
      "mask_color": "green",
      "include_scores": true,
      "include_boxes": true
    }
  }'

import os
import requests

resp = requests.post(
    "https://ai.inf.space/v1/vision-segment/predictions",
    headers={"Authorization": f"Bearer {os.environ['TOS_API_KEY']}"},
    json={
        "provider": "wujie",
        "model": "facebook/sam3",
        "input": {
            "image": "https://example.com/street.jpg",
            "prompt": "person",
            "threshold": 0.5,
            "mask_color": "green",
            "include_scores": True,
            "include_boxes": True,
        },
    },
    timeout=300,
)
resp.raise_for_status()
result = resp.json()
print(result["status"], result.get("output"))

const resp = await fetch("https://ai.inf.space/v1/vision-segment/predictions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.TOS_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    provider: "wujie",
    model: "facebook/sam3",
    input: {
      image: "https://example.com/street.jpg",
      prompt: "person",
      threshold: 0.5,
      mask_color: "green",
      include_scores: true,
      include_boxes: true,
    },
  }),
});
const result = await resp.json();
console.log(result.status, result.output);

Video segmentation

Send JSON to /v1/vision-segment/video. input fields (prompt is required for the video endpoint):

Field	Type	Required	Default	Description
`video`	string	Yes	—	HTTPS URL or `data:video/*` base64 URI
`prompt`	string	Yes	—	Text prompt describing the target to segment and track
`output_format`	string	No	`mp4`	Output format: `mp4` / `frames` / `masks` / `coco_rle`
`threshold`	number	No	`0.5`	Confidence threshold, 0–1
`frame_stride`	integer	No	`1`	Frame sampling stride
`start_time`	number	No	`0`	Start time in seconds
`end_time`	number	No	—	End time in seconds
`max_frames`	integer	No	—	Maximum number of frames to process
`prompt_propagation`	boolean	No	`true`	Whether to propagate the prompt across frames
`re_detect_every`	integer	No	`30`	Number of frames between detections
`mask_color`	string	No	`green`	Mask color; same options as the image endpoint
`mask_opacity`	number	No	`0.5`	Mask opacity, 0–1
`fps`	number	No	—	Output frame rate

curl "https://ai.inf.space/v1/vision-segment/video" \
  -H "Authorization: Bearer $TOS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "wujie",
    "model": "facebook/sam3",
    "input": {
      "video": "https://example.com/clip.mp4",
      "prompt": "the running dog",
      "output_format": "mp4",
      "frame_stride": 1,
      "max_frames": 300
    }
  }'

Response

Both endpoints return the same structure. status is the task state, and output contains the result file:

{
  "id": "vs_8f3c...",
  "status": "succeeded",
  "input": { "...": "echoed request input" },
  "output": "data:image/png;base64,iVBORw0KGgo...",
  "logs": "...",
  "error": null,
  "metrics": { "...": "timing and other metrics" },
  "scores": [0.97],
  "boxes": [[10, 20, 200, 400]]
}

status: starting / processing / succeeded / canceled / failed.
output: Result file, either a data: base64 URI or a same-origin download URL materialized by the gateway at /api/vision-segment/outputs/<filename>.
error: Error details when the task fails, or null on success.
scores / boxes: Returned by the image endpoint when include_scores / include_boxes is true.

When output is a same-origin download URL, download the file through /api/vision-segment/outputs/.... It is also protected by the ai:vision-segment scope, so use the same gk_ key in the Authorization: Bearer header.

Video segmentation is compute-intensive. Runtime increases significantly with video duration, frame_stride, and max_frames. Set the client timeout to at least 300 seconds, and determine success from the response status rather than the HTTP status code alone.