Common Items

Analyzer and Type

Select the analyzer to use in AI Analyzer. Analyzers can be added, edited, and deleted directly in the selection popup, and also managed in Settings > AI Analyzer. The analysis type is determined by the analyzer's AI Model, so the action dialog shows only the commands and fields of the model that the selected analyzer uses.

The Type field decides what this action does to the analyzer.

Command

The Command differs per analysis type and decides what is computed and which variables receive it. Choosing a command shows only the settings and outputs that the command needs. The commands of each type are listed in the tables of the sections below.

Common Settings

These fields have the same meaning across several types. Where a range or default differs per type, it is noted in that type's section.

Output Variables and Value Conventions

A variable bound to an output field is updated continuously while the analysis runs. Unbound outputs are not written, so bind variables only to the fields you need. The variable type of each field is fixed.

• Digital: true/false fields such as Detected, Matched, and Is Match.
• Number: counts, coordinates, width and height, confidence, probability, angle, and distance. Coordinates and sizes are in pixels of the camera frame, and a center point is the center of the detection box. Confidence and probability are 0 to 100 (OCR is the exception, see its section).
• Text: class names, recognized names, data, and JSON output.
• Color: the average color of Color Tracking.

The value written when nothing is detected is defined per type: text becomes none or an empty string, and numbers become -1 or 0, as listed in each section's output table. The Output (JSON Text) field writes the whole detection result as a JSON string to a text variable. Types that return several results use an array, which is empty ([]) when nothing is detected. The key names and samples are given under JSON Output in each section; parse the string in block coding to pick out values.

Object Detection

Detects 80 kinds of common objects such as people and vehicles in real time from the camera video. The classes are the 80 of the COCO dataset, and several can be selected together.

Commands

When a Detection Zone is set, every command considers only the objects inside it.

Settings

Output Variables

JSON Output

An array with, per object, class_name, confidence (0 to 100), the two box corners in bbox, the center, width, and height.

[
  {
    "class_name": "person",
    "confidence": 85,
    "bbox": { "x1": 100, "y1": 50, "x2": 300, "y2": 400 },
    "center": { "x": 200, "y": 225 },
    "width": 200,
    "height": 350
  },
  {
    "class_name": "car",
    "confidence": 72,
    "bbox": { "x1": 400, "y1": 200, "x2": 550, "y2": 300 },
    "center": { "x": 475, "y": 250 },
    "width": 150,
    "height": 100
  }
]

Face Detection

Detects human faces in real time from the camera video and tracks their positions. Includes emotion recognition and age and gender estimation.

Commands

When a Detection Zone is set, only faces inside it are considered.

Settings

Output Variables

JSON Output

Each face carries score (confidence, 0 to 100), bbox, center, width, and height. The emotion keys (emotion, emotion_id, emotion_confidence) are present only while Recognize Emotion runs, and the age and gender keys (age, gender, gender_confidence) only while Estimate Age/Gender runs. Note that the confidence key is score, unlike confidence in Object Detection.

[
  {
    "score": 95,
    "bbox": { "x1": 120, "y1": 80, "x2": 250, "y2": 300 },
    "center": { "x": 185, "y": 190 },
    "width": 130,
    "height": 220,
    "emotion": "happy",
    "emotion_id": 4,
    "emotion_confidence": 87,
    "age": 28,
    "gender": "male",
    "gender_confidence": 92
  }
]

Face Recognition

An extension of Face Detection that compares detected faces against enrolled faces to identify who they are. Enrolling and deleting faces is also done through this action's commands.

Commands

Settings

The face store is chosen by the analyzer setting DB Name (default default); analyzers with the same name share data. Anti-Spoofing, which blocks faces shown on photos or screens, is also an analyzer setting (Part 6, AI Analyzer).

Output Variables

This type has no JSON output.

Pose Estimation

Estimates a person's pose in real time from the camera video and provides the coordinates of 17 key points (joints). Distances between key points and joint angles can be computed directly.

Commands

Settings

Output Variables

JSON Output

An array with, per key point, id, name, x, y, and confidence (0 to 100). The sample shows only a few entries.

[
  { "id": 0, "name": "nose", "x": 320, "y": 180, "confidence": 95 },
  { "id": 5, "name": "left_shoulder", "x": 280, "y": 260, "confidence": 91 },
  { "id": 6, "name": "right_shoulder", "x": 360, "y": 258, "confidence": 93 },
  { "id": 15, "name": "left_ankle", "x": 270, "y": 470, "confidence": 78 }
]

Hand Tracking

Tracks hands in real time from the camera video and provides the coordinates of 21 key points (finger joints). Includes left/right handedness, finger extension state, and gesture recognition.

Commands

Settings

Output Variables

JSON Output

An array with, per key point, id, name, x, y, and z. The sample shows only a few entries.

[
  { "id": 0, "name": "wrist", "x": 320, "y": 350, "z": 0 },
  { "id": 4, "name": "thumb_tip", "x": 280, "y": 310, "z": -15 },
  { "id": 8, "name": "index_tip", "x": 310, "y": 260, "z": -25 },
  { "id": 12, "name": "middle_tip", "x": 330, "y": 255, "z": -22 }
]

Color Tracking

Finds regions of a specified color range in real time from the camera video and provides their position, size, area, and average color.

Commands

When a Detection Zone is set, only color regions inside it are considered.

Settings

Output Variables

JSON Output

An array with, per region, the center x and y, width, height, area, and the average color as a hex string.

[
  { "x": 320, "y": 240, "width": 50, "height": 60, "area": 3000, "color": "#FF5733" },
  { "x": 500, "y": 300, "width": 40, "height": 45, "area": 1800, "color": "#33FF57" }
]

QR / Barcode

Detects and decodes QR codes and several one-dimensional barcode formats in real time.

Commands

Settings

Output Variables

JSON Output

An array with, per code, data, type, the center x and y, width, and height.

[
  { "data": "https://example.com", "type": "QR-Code", "x": 320, "y": 240, "width": 100, "height": 100 },
  { "data": "1234567890123", "type": "EAN-13", "x": 500, "y": 300, "width": 80, "height": 40 }
]

OCR (Text Recognition)

Detects and reads text in real time from the camera video. The recognition language is chosen in the analyzer setting OCR Language: English (default), Korean, Japanese, Chinese (Simplified), German, French, Spanish, Portuguese, Hindi, Arabic, or Russian. One analyzer recognizes one language.

• Temporal stabilization suppresses results that flicker from frame to frame.
• Detected text is sorted into reading order (top to bottom, left to right).

Commands

Settings

Output Variables

JSON Output

An array with, per item, text, confidence (a fraction 0 to 1), the center x and y, width, and height.

[
  { "text": "Hello", "confidence": 0.95, "x": 100, "y": 50, "width": 80, "height": 25 },
  { "text": "World", "confidence": 0.88, "x": 190, "y": 50, "width": 70, "height": 25 }
]

Line Tracking

For line-following robots. Detects a line on the floor and provides its offset from the frame center, its tilt angle, and its vector, which are used to correct steering.

Commands

Settings

Output Variables

When no line is detected, Detected becomes false and the remaining numeric values are written as 0.

JSON Output

A single object, not an array. offset_x and line_width are fractions as described in the table.

{
  "detected": true,
  "offset_x": 0.15,
  "angle": 12.5,
  "line_width": 0.06,
  "vector_x0": 100,
  "vector_y0": 200,
  "vector_x1": 540,
  "vector_y1": 210
}

Line Crossing Counter

Automatically counts people entering and exiting across a virtual line drawn on the camera preview. Used for entrance footfall and store visitor counting.

Commands

Settings

Output Variables

JSON Output

tracks holds, per tracked person, the track ID, class, center x and y, size w and h, crossing direction (in or out), and whether it has crossed.

{
  "in_count": 5,
  "out_count": 3,
  "current_count": 2,
  "tracks": [
    { "id": 1, "class": "person", "x": 320, "y": 240, "w": 80, "h": 200, "direction": "in", "crossed": true }
  ]
}

Fire Detection

Detects flames in real time from the camera video. A digital variable that turns true when the fire probability exceeds the threshold drives automatic responses such as alarms, notifications, and equipment shutdown.

Commands

Settings

Output Variables

This type has no JSON output. The analyzer's Mode field runs the same model whichever value is chosen.

Action Recognition

Recognizes what a person is doing from 400 action categories (walking, running, sitting, clapping, cooking, and more). Used for fall detection, exercise form checks, and behavior analysis.

Commands

Settings

Output Variables

JSON Output

An array with, per result, label, confidence (0 to 100), and the model class number index, ordered from the highest probability.

[
  { "label": "walking", "confidence": 85, "index": 0 },
  { "label": "running", "confidence": 10, "index": 1 }
]

For the full list of the 400 actions see the official label list of the Kinetics-400 dataset (github.com/cvdfoundation/kinetics-dataset).

Hand Motion Recognition

Recognizes 25 hand motions performed in front of the camera (swipes, thumb up, stop sign, and more). Used for touchless control and game input. Unlike the gesture recognition of Hand Tracking, which classifies a hand shape in a single frame, hand motion recognition classifies movement over time.

Commands

Settings

Output Variables

JSON Output

An array with, per result, label, confidence (0 to 100), and the model class number index, ordered from the highest probability.

[
  { "label": "Swiping Left", "confidence": 85, "index": 0 },
  { "label": "Thumb Up", "confidence": 10, "index": 1 }
]

License Plate Recognition

Detects vehicle license plates in real time from the camera video and reads their text. The plate model is chosen in the analyzer setting Region: Latin (EU/US/SouthAmerica) (default) or Korea.

• Results from several frames are corrected by majority voting, which automatically fixes single-frame misreads.
• A minimum confidence filter removes noisy results.

Commands

Settings

Output Variables

JSON Output

An array with, per plate, text, region, confidence (text recognition confidence, 0 to 100), the plate detection stage score det_score, the center x and y, width, and height.

[
  { "text": "ABC1234", "region": "Argentina", "confidence": 92, "det_score": 0.97, "x": 320, "y": 240, "width": 120, "height": 40 }
]

Teachable Machine Classification

Classifies the camera video with an image classification model trained in Google Teachable Machine. The model is uploaded in the analyzer settings. Model Path (.tflite) accepts only 224×224 RGB models in TensorFlow Lite format, exported as the Floating Point or Quantized type. Label Path is the labels.txt exported with the model, with one class name per line in order. For the procedure to create a model see Appendix B, External Services.

Commands

Settings

Output Variables

JSON Output

An array with, per class, label and confidence (0 to 100), ordered from the highest probability.

[
  { "label": "apple", "confidence": 92 },
  { "label": "banana", "confidence": 6 },
  { "label": "orange", "confidence": 2 }
]

Edge Impulse Classification

Classifies images or sound with a classification model trained in Edge Impulse. Video and audio analyzers use the same commands and fields; the model is uploaded as the .eim file exported from Edge Impulse Studio in the analyzer setting Model Path (.eim). For the procedure to create a model see Appendix B, External Services.

Commands

Settings

Output Variables

JSON Output

An array with, per class, label and confidence (0 to 100), ordered from the highest probability.

[
  { "label": "normal", "confidence": 89 },
  { "label": "defective", "confidence": 11 }
]

Edge Impulse Object Detection

Detects objects in the camera video with an object detection model trained in Edge Impulse. The model is uploaded in the analyzer setting Model Path (.eim).

Commands

When a Detection Zone is set, only objects inside it are considered.

Settings

Output Variables

JSON Output

A flat array with, per object, label, confidence (0 to 100), the box's top-left x and y, width, and height. Unlike the built-in Object Detection there are no nested bbox and center objects, and x and y are not the center.

[
  { "label": "person", "confidence": 87, "x": 100, "y": 50, "width": 200, "height": 350 },
  { "label": "car", "confidence": 75, "x": 400, "y": 200, "width": 150, "height": 100 }
]

Edge Impulse Visual Anomaly

Judges normal versus abnormal states with a visual anomaly detection model trained in Edge Impulse. The frame is divided into a grid, each cell receives an anomaly score, and the mean and maximum over the grid are provided. The model is uploaded in the analyzer setting Model Path (.eim).

Commands

Settings

Output Variables

JSON Output

A single object, not an array. Each cell in grid has its position x and y, size width and height, and anomaly score value (0 to 100).

{
  "mean": 45,
  "max": 78,
  "grid": [
    { "x": 0, "y": 0, "width": 32, "height": 32, "value": 12 },
    { "x": 32, "y": 0, "width": 32, "height": 32, "value": 67 },
    { "x": 64, "y": 0, "width": 32, "height": 32, "value": 78 }
  ]
}

Sound Classification

A type of the audio analyzer. Classifies the sound from the microphone input (or the speaker output) into one of the 521 AudioSet categories learned by Google's YAMNet model (speech, music, dog barking, alarms, glass breaking, screaming, car horns, and more). Used for abnormal sound detection and environmental sound analysis. The input is chosen in the analyzer setting Audio Source: Microphone (Input) or Speaker (Output).

Commands

Settings

Output Variables

JSON Output

An array with, per result, label and confidence (0 to 100), ordered from the highest probability. Unlike Action Recognition and Hand Motion Recognition there is no index key.

[
  { "label": "Speech", "confidence": 85 },
  { "label": "Music", "confidence": 10 }
]

For the full list of the 521 categories see YAMNet's official class map (yamnet_class_map.csv).

Edge Impulse Classification (Audio)

Choosing Edge Impulse Classification as the audio analyzer's AI Model recognizes and classifies sound with an audio classification model trained in Edge Impulse. The commands, settings, output variables, and JSON format are the same as Edge Impulse Classification.

• The model is uploaded in the analyzer setting Model Path (.eim), and the input is chosen in Audio Source as the microphone or the speaker.
• Results are written only after the Start Analysis command is run following Add Analysis.