How real time works in QTM

Real time processing is always active when QTM is in preview mode. The processing steps are selected in the Real time actions list on the Processing page in the Project options dialog.

It is also possible to stream data from a file. Streaming from a file is started by clicking Play with Real-Time output in the Play menu (keyboard shortcut: Ctrl + Shift + Space). This can be useful for testing or developing real-time client applications.

The port settings and the client control settings can be accessed and modified on the Real-Time Output page in the Project Options dialog. For detailed information about the protocol and the commands, refer to the QTM RT protocol documentation.

Streaming data

The data that is processed in real time can be viewed in QTM and streamed via TCP/IP through the supported protocols (RT protocol and OSC). The TCP/IP server is always active and waits for a connection from a client program. Almost all data that is acquired and processed in QTM can be streamed, including:

• 2D data (unlinearized, linearized)

• 3D data (Identified, Unidentified, Residuals)

• 6DOF data (Position, Rotation matrix, Euler angles, Residuals)

• Skeleton data (T-pose, Segment positions and orientations in global or local coordinates)

• Analog data (Analog boards and EMG)

  Data is buffered in the analog equipment, which means that there will not always be new analog data for every marker frame.

• Force data (Force, Moment and COP)

  Data is buffered in the analog equipment, which means that there will not always be new force data for every marker frame.

• Eye tracker data (gaze vector, pupil size, etc.)

• Parameters (e.g. Label name, Statistics for the RT performance, Force plate settings)

• Events

• Images (from cameras in Marker intensity and Video mode; only supported in preview mode, not when streaming data from file)

Controlling QTM

QTM can also be controlled via the RT protocol, for example for:

• Starting/stopping of a calibration or a capture

• Changing camera mode

• Modifying settings under Project Options

Real-time frequency

The real-time marker frequency is set on the Camera system page in the Project options dialog. You can use the Reduced real time frequency option if a high capture rate is required but you still want real time data during the capture. It is recommended that the reduced frequency is an integer divisor of the capture rate.

These are the different steps in the real-time process:

1. The cameras capture data at a fixed frequency.

2. QTM processes the data in real-time as fast as possible. The processing steps are performed in parallel so that for example the 3D tracking can run faster than AIM or the Skeleton solver.

3. Data is sent on the TCP/IP output and the frequency displayed in the status bar is equal to the last processing step for 3D data. If QTM takes too long time processing a frame, the next frame captured by the cameras will not be processed for that processing step. This is shown in the main status bar as a decreased RT frequency.

Check that the processing can be done fast enough in QTM by looking at the status bar, the RT frequency should be the same as the camera frequency. There will also be a message in the status bar if any RT frames are dropped or if there is a mismatch between the camera frames. For more information see chapter Main status bar.

Optimization of real-time performance

Use the following guidelines to get as fast real-time as possible:

• Activate only the necessary processing steps on the Processing page.

• Set the GUI update to 15 Hz on the GUI page, or shut it off completely to get the maximum performance.

• Check that the RT frequency is stable during the RT measurement. Lower the rate if it changes to much from the specified capture rate.

Real-time streaming of data from external input devices

All external equipment samples, such as analog, EMG, force and eye-tracker samples, are sent with each camera frame, e.g. if the analog capture rate is three times the camera capture rate there will be in average three analog samples sent with each frame. However, because of the buffering in the external equipment the number of samples sent with each marker frame can differ, but the total number of samples will be correct.

The analog and EMG is started in synchronization with the markers. The analog boards are always started in synchronization with the sync out signal. The EMG on the other hand needs to be started with the trigger button to be synchronized, see respective EMG system in chapter Wireless EMG systems.

Real time latency

Real time latency is the time it takes from the marker exposure to the time when the TCP/IP signal can be received on another computer. The latency can be divided into the parts in the image below. The size of the different parts shows approximately how long time the different steps will take.

This delay will depend on the following factors:

• Number of cameras
  Increasing the number of cameras can increase the delay. Both because of the extra data and extra complexity in tracking

• Computer
  The computer performance will influence the latency and because QTM runs on Windows the latency may also differ depending on which other programs that are running.

Because the latency is system and computer dependent it is not possible to give a latency for any system. For Oqus cameras the system latency can be monitored in QTM. Enable the display of latency with the Show Real-Time Latency option on the GUI page in the Project options dialog. When enabled the latency will continuously be updated in the status bar. The latency is calculated by comparing the time stamp at exposure with the time stamp of a UDP packet sent to the cameras by QTM at the end of the calculation pipeline. With this method the latency will match the latency of a third-party software receiving the data stream over UDP/IP from QTM’s real-time server. Contact Qualisys AB for if you want more help on how to estimate the latency.