Add new inference node
Overview
The Observ platform is composed of two subsystems:
Web platform
A cluster of inference servers.
This document describes the steps of adding an inference server node. It is assumed that the the Web platform is ready.
Deploy an inference node
To ease the process of deployment, Linker Vision prepares a Docker image that runs deployment scripts automatically. The following content of this document describes the process of setting up environment and run the deployment Docker image.
Step 0. Prerequisites
Minimum requirement of hardware
CPU: 48 core
Memory: 96 G
GPU: NVIDIA L4
Disk: 1 TB
Minimum requirement of software
OS: Ubuntu 20.04 (22.04 is recommended.)
NVIDIA driver (version >= 530)
NVIDIA Container Toolkit
Docker
Step 1. Generate Configuration File
Go to the System Management panel and click the “+ Computer Target” button on the upper-right corner.
In the “Compute Target Setup” form, fill in the Cluster Name and Organization fields.
WebRTC and TURN Server Configuration
If you don’t need Live View on the inference node, just leave the fields blank, except for the WebRTC Domain:
WebRTC Domain: This field cannot be blank. Fill in placeholder text, e.g. node.observ.linkervision.ai
TURN Server URL, Username, Password : leave it blank
If you need Live View:
If you want to enable Live View on the inference node, you need to open the firewall port (443, 80) of the node. In addition, an HTTPS domain as well as its SSL certificate is required. Then, fill in the related fields.
WebRTC Domain: the HTTPS domain for WebRTC
TURN Server URL: IP address of the TURN server along with the required port.
TURN Username: username for the TURN server
TURN Password: password for the TURN server
When finished, Click the “Download Configuration” button. A JSON file named config.json will be downloaded. Note that it may take several seconds to generate the JSON file. Please wait and do not click the “Download Configuration” button.
Step 2. Pull the Docker image of installer
Login to docker registry by running the “docker login” command as shown below.
The values of username and password required to login the docker registry can be found in the config.json file generated in the previous step.(The keys are registry_username and registry_password, respectively.)
After you login to docker registry successfully, you can pull the latest version of the Docker image of installer.
Step 3. Run the installer
Run the installer using the following docker command.
config.json is the JSON file generated in previous step. host_ip is the ip address of the inference node. host_username and ssh_password are the login account name and password of the target inference node, respectively.
If the installing machine (the machine where the installer is running) and the target machine (where the agent is deployed) are the same machine, you will need to run the above “docker run” command twice.
Check the installation result
The status of the deployment process will be shown on the screen. If the deployment is successfully completed, you will see failed=0 and unreachable=0 on the screen. (As shown bellow)
Please note that it will take about 20 minutes for all the functions to be initialized before you can see the cluster on the System Management panel.
Remove the node
To remove the node from the cluster, run the following command.
Troubleshooting
“UNAHUTORIZED” error when trying to login docker registry
The “UNAHUTORIZED” error shown when running docker login command is due to the wrong password.
One reason of wrong password is because the Download Configuration button in the “Generate Configuration File” step is clicked more than one time. Each time the Download Configuration button is clicked, a new configuration file along with a new password will be generated. Only the latest version of configuration file is valid. In such a case, use the password stored in the latest configuration file.
“permission denied” error when running docker pull command
If you encounter “permission denied” error when running docker pull command, like the one shown below, refer to https://docs.docker.com/engine/install/linux-postinstall/ for steps to allow running Docker commands without using sudo.
Last updated