Trex - Generic Inference

The TRex Generic Inference algorithm enables user prompting on multiple images and get the boxes, scores on one target image.

This algorithm hypothesis that there is only one category per batch image, and it does not support batch inference.

Usage Pattern

First of all, make sure you have installed this SDK by pip:

pip install dds-cloudapi-sdk

Then trigger the algorithm by TRexGenericInfer class:

from dds_cloudapi_sdk import Config
from dds_cloudapi_sdk import Client
from dds_cloudapi_sdk import BatchRectPrompt
from dds_cloudapi_sdk import TRexGenericInfer

# Step 1: initialize the config
token = "Your API token here"
config = Config(token)

# Step 2: initialize the client
client = Client(config)

# Step 3: run the task by TRexGenericInfer class

image_url = "https://dev.deepdataspace.com/static/04_a.ae28c1d6.jpg"
# if you are processing local image file, upload them to DDS server to get the image url
# image_url = client.upload_file("/path/to/your/infer/image.png")

# the generic inference supports prompts from multiple images,
# but the prompts must be the same type, e.g. all point prompts or all rect prompts
batch_prompts = [
    BatchRectPrompt(
        image=image_url,
        rects=[[475.18413597733706, 550.1983002832861, 548.1019830028329, 599.915014164306]]
    )
]

task = TRexGenericInfer(
    image_url=image_url,
    batch_prompts=batch_prompts
)

client.run_task(task)
for obj in task.result.objects:
    print(obj.score)
    print(obj.bbox)
    break

API Reference

class TRexGenericInfer(image_url, batch_prompts)

Trigger the Trex Generic Inference algorithm.

This task can process prompts from multiple images, and each image can have several prompts. However, each task is limited to one type of prompt, either point or rect.

Parameters:

• image_url (str) – the image to be inferred on.

• batch_prompts (List[BatchRectPrompt] | List[BatchPointPrompt]) – list of BatchRectPrompt objects or BatchPointPrompt.

property result: TaskResult

Get the formatted TaskResult object.

class BatchPointPrompt(*, points, image=None, category_id=None)

A batch of point prompts.

Parameters:

• points (List[List[float]]) – a list of point locations in [[x1, y1], [x2, y2]]

• image (str) – the image url the point prompts are acting on, if not provided, the infer image url in context will be used

• category_id (int) – the category id of the points

points: List[List[float]]

a list of point locations in [[x1, y1], [x2, y2]]

image: str

the image url the point prompts are acting on

category_id: int

the category id of the points, only required for TRexInteractiveInfer task.

class BatchRectPrompt(*, rects, image=None, category_id=None)

A batch of rectangle prompts.

Parameters:

• rects (List[List[float]]) – a list of rect locations in [[[upper_left_x, upper_left_y, lower_right_x, lower_right_y], …]

• image (str) – the image url the rectangle prompts are acting on

• category_id (int) – the category id of the rects

rects: List[List[float]]

a list of rect locations in [[[upper_left_x, upper_left_y, lower_right_x, lower_right_y], …]

image: str

the image url the rectangle prompts are acting on, if not provided, the infer image url in context will be used

category_id: int

the category id of the rects, only required for TRexInteractiveInfer task.

class TaskResult(*, objects)

The task result of TRexGenericInfer task.

Parameters:

objects (List[TRexObject]) – a list of detected objects of TRexObject

objects: List[TRexObject]

a list of detected objects of TRexObject

class TRexObject(*, score, bbox)

The object detected by TRexGenericInfer task.

Parameters:

• score (float) – the prediction score

• bbox (List[float]) – the bounding box, [upper_left_x, upper_left_y, lower_right_x, lower_right_y]

score: float

the prediction score

bbox: List[float]

the bounding box, [upper_left_x, upper_left_y, lower_right_x, lower_right_y]