Visual grounding — locate objects in images or video

post/ground

Open-vocabulary visual grounding over images or video clips. Finds objects matching question in the provided image(s) and returns bounding boxes.

Images (images) — finds regions matching question and returns {label, box: [x1, y1, x2, y2]} normalized to [0, 1000].
Video (video) — scans a clip for objects (kind: "object", returning start, end, peak_t, and peak-frame boxes) or events (kind: "event", returning onset_t timestamps).

Body ParametersExpand Collapse

allow_abstain: optional boolean

Allow status: "abstained" (value null) when no candidate clears the confidence bar.

audio: optional array of object { b64, mime, url }

Audio clips to decide over: each {b64, mime} or {url}. Only on deployments where /capabilities reports audio available.

b64: optional string

Base64-encoded audio bytes.

mime: optional string

MIME type for b64 content.

url: optional string

Public HTTP(S) audio URL.

boxes_on_hits: optional boolean

Video only: also return regions on each hit's peak frame.

coarse_step: optional number

Video only: seconds between coarse presence probes.

exclusiveMinimum0

documents: optional array of object { b64, filename, mime, url }

b64: optional string

Base64-encoded file bytes.

filename: optional string

Helps format detection (e.g., memo.txt, contract.pdf).

mime: optional string

MIME type of the document.

url: optional string

Public HTTP(S) URL to fetch the document from.

domain: optional string

fine_resolution: optional number

Video only: seconds; precision of the onset search.

exclusiveMinimum0

images: optional array of object { b64, mime, url }

The image(s) to search.

b64: optional string

Base64-encoded image bytes (raw base64, not a data: URL). Pair with mime.

mime: optional string

MIME type for b64 content. Must be an image/* type.

url: optional string

Public HTTP(S) image URL. The server fetches it within a shared 60-second budget.

kind: optional "object" or "event"

Video only: "object" or "event".

Accepts one of the following:

"object"

"event"

max_probes: optional number

Video only: maximum frames to examine.

minimum1

min_region_patches: optional number

Deep mode only: ignore regions smaller than this many patches (>= 1).

minimum1

mode: optional "fast" or "deep"

Images: "fast" returns the single best match, "deep" every match.

Accepts one of the following:

"fast"

"deep"

question: optional string

What to find (e.g., "the invoice total", "the crack"). Empty detects regions broadly.

state: optional string

video: optional object { b64, max_edge, mime, url }

A video clip instead of images: the response is a video scan (VideoGroundResult).

b64: optional string

Base64-encoded video bytes.

max_edge: optional number

Downscale frame long edge to this size before processing.

minimum64

maximum1792

mime: optional string

url: optional string

Public HTTP(S) video URL.

ReturnsExpand Collapse

GroundDecision = object { billable_tokens, count, found, 6 more }

billable_tokens: number

Billing token count = max(1000, input_tokens).

count: number

Number of matched bounding box regions.

found: boolean

True when at least one region matched (count > 0).

grounded: boolean

Always true for grounding responses.

image_tokens: number

Image-specific subset of input_tokens.

input_tokens: number

Total tokens processed (text + images + documents + audio).

type: "ground"

Accepts one of the following:

"ground"

value: array of object { box, label, score }

Detected regions. Empty array when nothing found.

box: array of number

[x1, y1, x2, y2] coordinates normalized to [0, 1000].

label: string

score: optional number

Detection confidence.

read_as: optional map[unknown]

Present when /decide auto-framed the question as grounding.

VideoGroundDecision = object { duration_s, hits, kind, 6 more }

duration_s: number

Total clip duration in seconds.

hits: array of unknown

For kind: "object" — each hit has start, end, peak_t, confidence, and optional boxes (array of GroundRegion). For kind: "event" — each hit has onset_t and confidence.

kind: "object" or "event"

Accepts one of the following:

"object"

"event"

type: "video_ground"

Accepts one of the following:

"video_ground"

billable_tokens: optional number

image_tokens: optional number

input_tokens: optional number

scanned: optional number

Number of frames probed.

truncated: optional boolean

Whether scanning was truncated by max_probes.

Visual grounding — locate objects in images or video

curl https://api.articul8.ai/bivia/ground \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $A8_API_KEY" \
    -d '{}'

{
  "type": "ground",
  "value": [
    {
      "label": "the red square",
      "box": [
        197,
        200,
        812,
        807
      ]
    }
  ],
  "count": 1,
  "found": true,
  "grounded": true,
  "input_tokens": 286,
  "image_tokens": 256,
  "billable_tokens": 1000
}

Returns Examples

{
  "type": "ground",
  "value": [
    {
      "label": "the red square",
      "box": [
        197,
        200,
        812,
        807
      ]
    }
  ],
  "count": 1,
  "found": true,
  "grounded": true,
  "input_tokens": 286,
  "image_tokens": 256,
  "billable_tokens": 1000
}