INTERACTIVE RETAIL · COMPUTER VISION · TECHNICAL PROTOTYPE

Tracking the right user for a touch-free interactive kiosk.

We delivered the computer-vision layer for a gesture-controlled marketing kiosk, using stereo depth to isolate the interaction zone and pose estimation to produce stable 2.5D body tracking.

Visitor interacting with a gesture-controlled kiosk while pose tracking marks the active user in the interaction zone — INTERACTIVE KIOSK · Computer-vision prototype

Client: Confidential German company
Sector: Interactive advertising
Engagement: Computer-vision prototype
Delivered: 2020

INPUT

Stereo

Synchronized cameras converted to depth

OUTPUT

2.5D skeleton

Image-space keypoints with usable depth

DELIVERY

Accepted milestones

Background removal, pose and user tracking

Accepted outcome

All three core milestones were accepted: depth-based background removal, 2D pose estimation and active-user tracking.

The pipeline produced structured, frame-indexed 2.5D tracking data for downstream gesture recognition.
Docker and NVIDIA runtime configuration made the prototype reproducible on compatible kiosk and development machines.

THE PROBLEM AND HYPOTHESIS

Detecting a person was easy. Keeping the correct person in control was the product problem.

An advertising kiosk can stand in a busy retail or exhibition environment. Several people may be visible at once, and the closest detection in the image is not always the person occupying the intended physical interaction zone.

The client already had stereo-camera hardware and planned downstream gesture recognition and augmented rendering. The missing layer was a stable representation of the active user: body keypoints connected to depth and retained across frames.

PERCEPTION PIPELINE

Depth constrained the scene before pose and user-state logic.

One annotated pipeline connects the stereo hardware input to the 2.5D output accepted by the downstream interaction system.

01 · Test stage
Synchronised stereo frames
02 · Test stage
Rectification
03 · Test stage
Disparity and depth
04 · Test stage
Range isolation
05 · Test stage
Pose estimation
06 · Test stage
Active-user state
07 · Test stage
2.5D output

ENGINEERING DECISIONS

Using depth to simplify a crowded physical scene.

Use depth to simplify the scene before pose inference

Background removal reduces the number of irrelevant people presented to the pose model and makes the active-user rule relate to physical distance rather than apparent image size.

Computer vision

Deliver 2.5D evidence instead of overbuilding full 3D pose

The downstream experience needed stable spatial gesture data, not a research-grade reconstruction of the entire body. Combining 2D keypoints with stereo depth reduced technical risk and fitted the hardware already chosen.

Scope judgement

Acceptance was deliberately bounded

The engagement established the three agreed perception milestones against the supplied hardware and acceptance videos. It did not establish a general-purpose people-tracking platform for arbitrary cameras or environments.

RESPONSIBILITY