Anonymous project page

TSD: A Kinematics-Based Trajectory Saliency Detector for Data Efficient Robot Imitation Learning

Anonymous Authors

Detecting precise and agile phases from demonstration kinematics for informative dataset compression and targeted data collection.

Overview video

TSD finds precise interactions and agile redirections, then uses these phases to retain or collect informative demonstration data.

Paper

Abstract

Robot imitation learning relies on expert demonstrations, yet collecting complete trajectories is labor-intensive and training on every transition is computationally costly. Most pipelines allocate both budgets at the trajectory level, overlooking the distinct spatial and dynamic characteristics of task phases. We present the Trajectory Saliency Detector (TSD), a training-free method that identifies precise and agile phases directly from demonstration kinematics. TSD aligns demonstrations, detects precise operations from cross-demonstration pose dispersion, and locates agile maneuvers using centripetal acceleration. The detected phases support dataset compression by combining complete base demonstrations with salient source segments, and dataset expansion by collecting local demonstrations at identified phases. Across five simulated tasks with two policy architectures, TSD uses 25.4--32.0% fewer transitions and reduces policy-training time by 13.4--26.4% relative to the corresponding complete datasets. Its average success remains within 1.5 percentage points of full-data performance or exceeds it, while outperforming all comparable-budget baselines. Across three real-world tasks, TSD-guided collection outperforms matched-budget complete-trajectory collection, approaches full-data performance, and reduces collection time by 15.2%. These results show that phase-level trajectory saliency enables more efficient allocation of demonstration and training resources.

Framework

One detector, two data-efficient usages

TSD identifies salient phases directly from aligned trajectory kinematics and applies the same detections to dataset compression and expansion.

TSD framework showing precise and agile segment detection, dataset compression, and dataset expansion
Precise segments are identified from cross-demonstration pose dispersion. Agile segments are identified from centripetal acceleration with geometric verification.

Method

Detecting salient trajectory phases

Precise

Cross-demonstration pose convergence

After pose-aware temporal alignment, TSD measures local end-effector pose dispersion across demonstrations. Concentrated, low-speed regions provide evidence of spatially constrained operations such as grasping and placement.

Agile

Dynamic response with geometric verification

Centripetal acceleration captures rapid changes in motion direction. RDP points and trajectory tortuosity provide geometric verification that suppresses derivative-induced spurious peaks.

Precise and agile segments detected in the Book Fetching task
Qualitative localization on Book Fetching. Precise intervals are shown in red and agile intervals in green. TSD identifies grasping and placement as precise phases and obstacle avoidance as an agile phase.

Detection results across tasks

Select a task to inspect the corresponding precise and agile detections. For bimanual tasks, the arm selector switches between the two detection outputs.

TSD detection result for Can
Can

Evaluation

Experimental setup

Five robomimic simulation tasks evaluate dataset compression with Diffusion Policy (DP) and ACT. Three real-world tasks evaluate TSD-guided dataset expansion with DP.

Five simulation tasks and three real-world manipulation tasks
Simulation success is evaluated over 450 randomized rollouts per condition; real-world success is evaluated over 30 trials per condition. Every policy is trained for 1,200 epochs on its corresponding dataset.

Dataset compression

Simulation results

TSD reduces the number of retained transitions while preserving the performance of the corresponding complete datasets. Budget-controlled comparisons test whether the retained phases are more informative than randomly or retrospectively selected data.

Simulation success rates (%) for DP and ACT. Results use 450 randomized rollouts per condition. Ret. is the mean percentage of Full-100 transitions retained across the five tasks. TSD rows are highlighted, and the best budget-matched average is bold.
Pool Dataset Ret. ↓ DP ACT
Lift Can Square Tool Trans. Avg. ↑ Lift Can Square Tool Trans. Avg. ↑
20 Base 20.1 96.6 53.3 11.1 5.1 24.6 38.1 96.4 41.8 11.8 10.4 62.2 44.5
50 Random-Traj 38.0 98.8 72.8 66.2 35.7 49.5 64.6 98.7 64.9 32.7 25.1 66.9 57.6
Random-Seg 37.2 96.2 66.2 24.4 27.6 59.1 54.7 99.8 49.8 26.7 14.0 46.9 47.4
IWR 39.6 98.7 79.1 64.2 18.7 60.2 64.2 99.6 68.4 37.6 15.3 66.4 57.5
DemInf 37.7 99.1 81.6 64.7 20.4 56.4 64.4 99.6 66.9 40.7 13.6 79.3 60.0
TSD 37.2 99.3 82.0 68.8 35.7 69.5 71.1 100.0 73.8 41.1 33.3 76.7 65.0
Full-50 49.9 98.6 83.1 68.6 36.4 73.3 72.0 99.8 72.0 38.7 28.7 70.4 61.9
100 Random-Traj 68.5 98.6 91.7 73.3 46.6 88.8 79.8 99.6 80.2 53.1 35.3 81.3 69.9
Random-Seg 68.7 98.2 86.2 38.2 33.3 73.1 65.8 100.0 67.8 45.8 20.7 51.3 57.1
IWR 69.1 99.6 90.4 73.6 40.0 88.7 78.4 100.0 86.0 52.4 27.1 72.0 67.5
DemInf 68.8 99.8 90.2 73.3 49.6 92.0 81.0 99.8 83.1 56.1 30.9 78.9 69.8
TSD 68.0 99.7 92.0 76.8 56.8 95.3 84.1 100.0 90.7 61.3 46.9 81.8 76.1
Full-100 100.0 99.3 93.7 77.5 58.6 89.3 83.7 100.0 90.9 66.7 48.2 82.4 77.6

Policy rollouts

Compare policies trained on TSD segments and the corresponding complete dataset.

Policy
Pool
TSD-50
Full-50

Targeted data collection

Real-world results

TSD identifies where additional local demonstrations should be collected. The resulting datasets are compared with transition-matched complete-trajectory collection and the corresponding complete-data references.

Real-world dataset expansion with DP. Success rates use 30 trials per condition; size is the percentage of task-specific Full-100 transitions.
Task Metric Base 50% pool 100% pool
Rand.-Traj TSD Full Rand.-Traj TSD Full
Tray Setting Suc. (%) ↑ 0.0 46.7 60.0 53.3 76.7 86.6 83.3
Size (%) ↓ 21.0 45.3 45.0 50.6 84.9 84.9 100.0
Water Stowing Suc. (%) ↑ 40.0 53.3 56.7 63.3 70.0 70.0 76.6
Size (%) ↓ 20.5 40.4 40.8 50.3 75.6 75.8 100.0
Book Fetching Suc. (%) ↑ 46.7 53.3 60.0 66.7 76.7 93.3 93.3
Size (%) ↓ 19.7 42.5 42.3 49.7 80.1 80.3 100.0
Additional real-world data collection time. Times include phase resetting and teleoperation and exclude the shared base set.
Task Complete
(min)		 TSD
(min)		 Save
(%)
Tray Setting 52.5 46.6 11.2
Water Stowing 43.5 35.0 19.5
Book Fetching 54.0 46.0 14.8
Mean 50.0 42.5 15.2

Real-world policy rollouts

Compare DP policies trained with TSD-guided local demonstrations and complete demonstrations.

Pool
TSD-50
Full-50

Detector analysis

Detector analysis

Component ablation

The complete detector combines precise interactions with agile redirections. Single-branch datasets and transition-matched complete-trajectory controls clarify the contribution of each branch.

Component ablation. P and A denote precise and agile segments; “matched” denotes complete-trajectory controls with similar transition counts.
Book Fetching Tool Hang
Dataset Transitions Succ. (%) Transitions Succ. (%)
TSD (P+A) 14921 93.3 64237 56.8
Precise only 12938 63.3 62283 52.9
Full-Traj (P-matched) 13015 70.0 60930 46.2
Agile only 7220 40.0 22179 16.0
Full-Traj (A-matched) 7110 50.0 22870 30.0
Failure without agile segments
Without agile segments: hesitation or collision during obstacle avoidance.
Failure without precise segments
Without precise segments: inaccurate grasping.

Ablation rollouts

Book Fetching — TSD (P+A)

Detection convergence

Mean changes in both detector branches decrease as additional demonstrations are used.

agility convergence results under randomization
precision convergence results under randomization

Robustness and sensitivity

We examine the pose metric, reference trajectory, and shared detector parameters while keeping the remaining settings fixed.

Pose-metric robustness averaged over simulation tasks. ΔRet. is the change in retained-data ratio in percentage points.
Metric IoU F1 ΔRet.
7D Euclidean 1.000 1.000 0.000
SE(3), η=0.5 0.925 0.961 1.458
SE(3), η=1 0.949 0.981 0.806
SE(3), η=2 0.994 0.997 −0.048
Quat. chordal 0.949 0.981 0.806
Reference sensitivity over 10 alternatives per task. A hit denotes normalized Chamfer distance below 5%.

Task Hit rate Norm. Chamfer
Lift 0.9780±0.0207 0.0091±0.0031
Can 0.9730±0.0131 0.0102±0.0033
Square 0.9340±0.0255 0.0162±0.0038
Tool Hang 0.9082±0.0363 0.0150±0.0059
Transport 0.9101±0.0475 0.0170±0.0036
Parameter sensitivity measured by mean frame IoU over simulation tasks.
Setting Prec. Agile Union
β=0.05 1.000 0.993 1.000
β=0.20 0.841 0.972 0.843
Lmax=4 0.934 — 0.941
Lmax=16 0.930 — 0.935
(ρD,ρW)=(.025,.010) 0.998 0.973 0.997
(ρD,ρW)=(.100,.040) 0.982 0.715 0.975

Computational efficiency

Lower downstream training cost

TSD introduces a small one-time processing cost while reducing the substantially larger cost of downstream policy training.

Downstream policy training. Save is the mean ± standard deviation of task-wise training-time reduction; Perf. is the TSD-to-Full success-rate ratio.
Setting Pool Full (h) TSD (h) Save (%) Perf. (%)
Sim.–DP 50 9.43 7.77 13.4±4.5 98.7
Sim.–DP 100 17.99 13.46 19.9±5.4 100.5
Sim.–ACT 50 13.03 10.83 19.7±8.8 105.0
Sim.–ACT 100 26.54 20.29 26.4±11.2 98.1
Real–DP 50 15.86 13.45 14.5±2.2 96.4
Real–DP 100 28.15 23.67 16.5±3.6 98.7
Dataset curation time.
Method Computation Mean time Range
TSD Trajectory processing 42.51±49.76 s 2.02–106.80 s
DemInf Two VAE stages 1.54 h 1.46–1.74 h
IWR State–action VAE 2.56 h 2.55–2.59 h