# frames

In the VisionAI format, a frame serves as a container for dynamic, time-based information, encompassing both object and context data. They are used to represent data that is dynamic and time-based in nature, such as information about the location, velocity, and orientation of objects within a sequence.

Each frame is represented by a series of 12 digital numbers that are arranged in order to indicate the temporal relationship.

## Schema

{% @mermaid/diagram content="erDiagram
visionai ||--|| frames : contains
frames ||--|{ FRAME\_ID : contains
FRAME\_ID ||--|| frame\_properties : contains
FRAME\_ID ||--|| objects : contains
FRAME\_ID ||--|| contexts : contains
frame\_properties ||--|| streams : contains
streams ||--|{ STREAM\_NAME : contains
STREAM\_NAME ||--|| stream\_properties : contains
stream\_properties ||--|| sync : contains
visionai {
}
frames {
}
FRAME\_ID {
contexts object
objects object
frame\_properties object
}
frame\_properties {
timestamp time
streams object
}
streams{
STREAM\_NAME object
}
STREAM\_NAME {
uri string
stream\_properties object
}
stream\_properties{
sync object
}
sync{
timestamp time
}" %}

### Example

```json
"frames": {
    "000000000000": {
        "frame_properties": {
            "timestamp": "2020-04-11T12:00:01.000+0100",
            "streams": {
    	        "camera1": {
                    "uri": "https://image1.path.url.png",
                    "stream_properties": {
          	            "sync": {"timestamp": "2020-04-11T12:00:07.010+0100"}
                    }
                },
                "lidar1": {
                    "uri": "https://lidar0.path.url.pcd",
                    "stream_properties": {
                        "sync": {"timestamp": "2020-04-11T12:00:10.087+0100"}
                    }
                }
            }
        },
        "contexts": {
            "3456a995-a39c-41f2-a9ab-a4c9be693456": { ... },
            "6789a995-a39c-41f2-a9ab-a4c9be697892": { ... }
        },
        "objects":{
            "2eeda995-a39c-41f2-a9ab-a4c9be695beb": {...},
            "cf66fbb2-30b7-43b9-96db-85f70b3619f9": {...}
        }
    },
    "000000000001": {...}
}
```

### frames {}

<table><thead><tr><th width="195">name</th><th width="363">description</th><th width="108">type</th><th>required</th></tr></thead><tbody><tr><td>${FRAME_ID}</td><td>The id of frame in this annotation.<br>It is the frame order containing 12 digital numbers.<br>(ex. 000000000000, 000000000001, 000000000002)</td><td>object</td><td>true</td></tr><tr><td>frame_properties</td><td>Show the information of this frame.</td><td>object</td><td>true</td></tr><tr><td>contexts</td><td>Shows all contexts’ dynamic information in this frame.<br>Refer to frames-contexts.</td><td>object</td><td>false</td></tr><tr><td>objects</td><td>Shows all objects’ dynamic information in this frame.<br>Refer to <a href="https://linkervision.getoutline.com/doc/objects-zeQrea9iMt">frames-objects</a><a href="frames/objects">.</a></td><td>object</td><td>false</td></tr></tbody></table>

### frame\_properties {}

It describes this frame’s information including the reference time, and corresponding streams.

<table><thead><tr><th width="144">name</th><th width="420">description</th><th>type</th><th>required</th></tr></thead><tbody><tr><td>timestamp</td><td>A relative or absolute time reference that specifies the time instant this frame corresponds to. Representation of date and time format: "yyyy-MM-ddTHH:mm:ss.FFFZ"<br>（ex. 2021-09-03T11:23:56.000+0100）</td><td>time</td><td>false</td></tr><tr><td>streams</td><td>The “streams" information refers to the streams(sensors) that were used to capture data in this particular frame. This information helps to identify the sources of data that are included in that frame.</td><td>object</td><td>true</td></tr></tbody></table>

### streams {}

It describes this frame’s stream information including the corresponding stream name and related data path.

<table><thead><tr><th width="204">name</th><th width="354">description</th><th width="117">type</th><th>required</th></tr></thead><tbody><tr><td>${STREAM_NAME}</td><td>The name of the stream under this frame.<br>All the streams are matched and described under the visionai - streams.</td><td>object</td><td>true</td></tr><tr><td>uri</td><td>The url or file path for this stream data.</td><td>object</td><td>true</td></tr><tr><td>stream_properties</td><td>Additional properties of this stream. ex. sync infomation</td><td>object</td><td>false</td></tr></tbody></table>

### stream\_properties {}

It contains the additional properties of this stream including the absolute time of the sensor in this frame.

<table><thead><tr><th width="142">name</th><th width="399">description</th><th>type</th><th>required</th></tr></thead><tbody><tr><td>sync</td><td>Define the frame number of the stream that corresponds to this frame, along with timestamping information.</td><td>object</td><td>false</td></tr><tr><td>timestamp</td><td>An absolute time reference that specifies the time instant from the sensor of this frame. Representation of date and time format: "yyyy-MM-ddTHH:mm:ss.FFFZ"<br>（ex. 2021-09-03T11:23:56.000+0100）</td><td>time</td><td>false</td></tr></tbody></table>

***

## Use Case

More infomation on&#x20;

{% content-ref url="../use-case" %}
[use-case](https://linkervision.gitbook.io/dataverse/visionai-format/use-case)
{% endcontent-ref %}

### bbox

To describe a bbox dataset with one camera sensor:

{% content-ref url="../use-case/bbox" %}
[bbox](https://linkervision.gitbook.io/dataverse/visionai-format/use-case/bbox)
{% endcontent-ref %}

### bbox + cuboid (3D)

To describe a dataset with one camera sensor (bbox annotation) and one lidar sensor (cuboid annotation) in the coordinate system of iso8855-1:

{% content-ref url="../use-case/bbox-+-cuboid-3d" %}
[bbox-+-cuboid-3d](https://linkervision.gitbook.io/dataverse/visionai-format/use-case/bbox-+-cuboid-3d)
{% endcontent-ref %}

### semantic segmentation

To describe a semantic segmentation dataset with one camera sensor:

{% content-ref url="../use-case/semantic-segmetation" %}
[semantic-segmetation](https://linkervision.gitbook.io/dataverse/visionai-format/use-case/semantic-segmetation)
{% endcontent-ref %}

### tagging

To describe a dataset with taggings:

{% content-ref url="../use-case/tagging" %}
[tagging](https://linkervision.gitbook.io/dataverse/visionai-format/use-case/tagging)
{% endcontent-ref %}