PointTracker

High-precision infrared point tracking in OpenTrack

PointTracker is OpenTrack’s implementation of 3-point infrared LED tracking, offering the highest precision and lowest latency of all camera-based trackers. It’s similar to commercial solutions like TrackIR but completely free and open source.

How It Works

PointTracker detects 3 bright infrared (IR) LED points arranged in a specific geometry (cap, clip, or custom configuration). Using computer vision and pose estimation algorithms, it calculates your head’s position and orientation with sub-millimeter accuracy.

PointTracker requires a camera with the IR filter removed or a PS3 Eye camera, plus 3 IR LEDs arranged in a known pattern.

Requirements

Hardware

• Camera: PS3 Eye camera (recommended) or any webcam with IR filter removed
• IR LEDs: 3x infrared LEDs (850nm or 940nm wavelength)
• Power source: Battery pack or USB power for LEDs (typically 3-5V)
• LED mounting: Cap, clip, or headset to hold LEDs in fixed geometry

Software

• OpenCV (included with OpenTrack)
• Camera drivers (PS3 Eye requires CL-Eye driver on Windows)

Setup Instructions

1. Build or Buy IR LED Setup

   You need 3 IR LEDs mounted in one of these configurations:

   • Clip model: LEDs mounted on a hat clip
     • Point positions: (0, 40, -30), (0, -70, -80), (0, 0, 0)
   • Cap model: LEDs mounted on a cap
     • Point positions: (40, -60, -100), (-40, -60, -100), (0, 0, 0)
   • Custom: Define your own LED positions

   The points should form a triangle that’s never parallel to the camera. Keep the model asymmetric for better pose resolution.

2. Prepare Camera

   For best results with standard webcams:

   1. Remove the IR filter from your webcam (requires disassembly)
   2. Alternatively, use a PS3 Eye camera (no modification needed)
   3. Mount camera on monitor, facing your head position
   4. Ensure camera has clear view of LED points in all head positions

   Removing the IR filter will affect normal color vision. Use a dedicated camera for tracking.

3. Configure Camera Settings

   In OpenTrack tracker settings:

   Device: Select your camera
   Resolution: 640x480 (higher = more accurate)
   FPS: 30-60 (higher = smoother)
   FOV: 56 degrees (adjust based on your camera)
   
4. Adjust Threshold Settings

   The threshold determines which bright areas are detected as points:

   1. Enable Automatic Threshold initially
   2. View the video preview to see detected points (shown in blue)
   3. If auto-threshold doesn’t work well, disable it and adjust manually
   4. Set threshold so only the 3 LED points are detected

   Point Size Limits:

   • Min diameter: 2.5 pixels (filter out small reflections)
   • Max diameter: 50 pixels (avoid false detections)
5. Configure Point Model

   Select your model type and enter dimensions:

   For Cap model:

   Active Model: Cap
   Cap Width (X): 40mm per side
   Cap Height (Y): 60mm back
   Cap Depth (Z): 100mm back
   

   For Clip model:

   Active Model: Clip
   Enter clip dimensions in millimeters
   

   For Custom model:

   Active Model: Custom
   M01 (X, Y, Z): Position of first point
   M02 (X, Y, Z): Position of second point
   M00 is at origin (0, 0, 0)
   
6. Calibrate Head Center

   Calibrate the offset from the LED model to your head’s center of rotation:

   1. Click Calibrate button in tracker settings
   2. Rotate your head (yaw, pitch, roll) without moving your shoulders
   3. Keep moving for 10-15 seconds
   4. The model position offset updates in real-time
   5. Click Calibrate again to finish

   Good calibration is crucial for accurate translation tracking. Don’t move your shoulders during calibration - only rotate your head.

Configuration Options

Camera Settings

Option	Default	Description
Camera Name	-	Select your camera device
Resolution (X)	640	Width in pixels (higher = more accurate)
Resolution (Y)	480	Height in pixels
FPS	30	Target framerate (30-120)
FOV	56°	Camera field of view
Use MJPEG	false	Enable MJPEG compression (may increase FPS)

Point Detection

Option	Default	Description
Auto Threshold	true	Automatically adjust detection threshold
Threshold	128	Manual threshold value (0-255)
Min Point Size	2.5	Minimum blob diameter in pixels
Max Point Size	50	Maximum blob diameter in pixels
Blob Color	BT709	Color extraction method

Available Blob Color Modes

• BT709: Standard color-to-grayscale conversion
• Hardware: Use camera’s native grayscale mode
• Red/Green/Blue Only: Extract single color channel
• Red/Green/Blue/Cyan/Yellow/Magenta Chromakey: Advanced color filtering

Pose Estimation

Option	Default	Description
Dynamic Pose	false	Use predictive pose resolution for extreme angles
Init Phase Timeout	250ms	Time before auto-reset during initialization

Dynamic Pose Resolution uses velocity prediction to handle extreme poses and ambiguous point correspondence. Enable this if you need to track large head rotations, but it may occasionally require a reset if it gets stuck in wrong pose.

Point Filtering

Option	Default	Description
Enable Point Filter	false	Enable exponential smoothing on detected points
Filter Coefficient	1.0	Filter strength (0-4)
Filter Limit	0.1	Maximum filter correction per frame
Filter Deadzone	0	Dead zone for small movements

Troubleshooting

Points not detected

• Check if camera can see IR light (view through phone camera)
• Increase LED brightness or use more LEDs in parallel
• Lower the threshold value
• Ensure IR filter is removed from camera
• Check for ambient IR interference (sunlight, fluorescent lights)

False point detections

• Increase minimum point size
• Reduce maximum point size
• Increase threshold value
• Block reflective surfaces in camera view
• Use narrower FOV or zoom in camera

Jittery tracking

• Enable point filter with low coefficient (0.5-1.0)
• Increase camera resolution
• Ensure LEDs are securely mounted
• Check for camera focus issues
• Use Accela filter in OpenTrack filter settings

Wrong pose/inverted tracking

• Verify model dimensions are correct
• Check point correspondence (model matches physical arrangement)
• Try manual reset if using dynamic pose resolution
• Recalibrate head center offset
• Ensure points form asymmetric triangle

Low framerate

• Reduce camera resolution
• Enable MJPEG compression
• Close unnecessary applications
• Disable video preview when not needed
• Use USB 3.0 port for camera

Performance Tips

Higher FPS

• Use PS3 Eye camera (up to 120 FPS at 320x240)
• Lower resolution increases FPS
• Enable MJPEG compression
• Disable video preview

Better Accuracy

• Use higher resolution (640x480 or 1280x720)
• Ensure good LED brightness
• Mount camera close to eye level
• Use larger LED spacing (bigger model)

Reduce Noise

• Enable point filter
• Use Accela filter in OpenTrack
• Increase min point size
• Use good quality LEDs with stable power

Better Range

• Enable dynamic pose resolution
• Use wider FOV camera
• Increase max point size
• Use brighter LEDs

See Also

• Easy Tracker - Simplified alternative to PointTracker
• Hardware Guide - Building your IR LED setup
• Wiimote Tracker - Alternative IR tracking with Wiimote