Exporting Data to DataVerse

VisionAI Observ can export data to DataVerse. Here, you can easily filter and curate image samples to continuously improve and evolve your AI models.

Quick Start

DataVerse is a powerful tool for data filtering and sampling, enabling AI models to learn continuously. Click on the video below for a quick overview on how to export data to DataVerse:

View Your Exported Data

Under the AI Detection tab, select Dataset to view the status of the data exported to DataVerse.

Once in the Dataset tab, clicking View allows you to review the data in Dataverse. Note that this requires DataVerse access.

• Dataset / Description: Name and description of the dataset.

• Model: The AI model related to the exported data.

• Status: The current status.

• View: When the data export is successful, you can click View in Dataverse to check the data condition, deciding how to proceed with labeling and AI model training.

When exporting large batches of data, processing and data transfer time is required.

Filtering and Data Export to DataVerse

Click Export to DataVerse in the top right corner to import event images and pre-collected images into Dataverse. You will first need to log into your DataVerse account.

How to visit DataVerse? Start using it here 👉 https://linkervision.gitbook.io/dataverse/

You can choose to import data into an existing DataVerse project or create a new one. Upon exporting, make sure to input the following fields:

• DataVerse Project: Select from the existing projects already created in DataVerse.

• Dataset Name: The name of the dataset to be exported.

• Description: A description of the dataset.

• Tasks: Filter the tasks to be exported. Select from several cameras that require training.

• Event Type & Status: Filter the events and their status for the data to be exported. You can select information such as false alerts to export.

• Time Range: Filter the time for the data to be exported. Select specific times for particular situations.

• Generate IQA: Whether to generate Image Quality information in DataVerse. Refer to details + DataVerse Generate IQA.

Click Create to establish the dataset and export data to DataVerse.

Tips

We strongly recommend formulating a data filtering strategy before exporting to Dataverse because excessive similar data consume labeling resources and offer limited aid in AI model training.

When needing to retrain AI models, we can first understand the current detection status of the AI model, label events (correct or false alerts), and filter data for particularly inaccurate scenarios or cameras before importing into DataVerse. This prevents massive exports and facing an overwhelming amount of data in DataVerse, making it difficult to find suitable training material.

DataVerse Visualization: Viewing the Exported Dataset

DataVerse is designed with a data-centric concept, making it easy to find and filter samples, select appropriate data for labeling and AI training, and then return the well-trained AI model to Observ for continuous evolution.

In DataVerse's Data Visualization, you can see the exported data and also:

• Class: Filter related categories through Class. You can specifically select objects that are harder to detect.

• IQA: Through image analysis, filter out images that are too dark, overexposed, or blurry, or use this method to find day or night images. Refer to details + DataVerse Generate IQA

• Annotation: Filter by Annotation to compare the results produced by different AI Models.

• Tag: Narrow down the scope with information carried by Tags. After exporting data from Observ, the attributes of the original tasks are still included in Tags for easy querying. This includes the scene, task, camera, event, event status, image type (event or collected images), and more.

• Finally, you can use DataVerse's Data Slice sampling mechanism to select appropriate data for labeling and training. Refer to details + DataVerse Data Sampling

Practical Consideration

In practical use of Observ, you'll find that the images from fixed cameras are very similar.

Through the Image Collection settings, you can set an AI Confidence Score range or extend the interval for capturing images to select more unique photos + Collecting AI Data.

Or, make a good data filtering decision before exporting to DataVerse, pre-filtering once. Additionally, you can collect special photos through some custom events.

Once you have imported data into DataVerse, you can also take IQA filtering, select data by Class, or pick data by Tag, and use the sampling mechanism (e.g., average sampling by camera) to create your own dataset for model retraining.