Directed Temporal Representations
for Offline Visual Control

Chenyang Yuan¹ Haoyu Wang² Zhuo Sun^3,† Xiaoyuan Cheng^4,†

¹University of Sheffield²Shanghai Jiao Tong University

³Shanghai University of Finance and Economics⁴University College London

^† Corresponding authors.

Paper Code Training evolution

Abstract

World models provide predictive visual features. Directed Temporal Representations for Control (DTRC) organizes these features around the cost of reaching a goal. From offline trajectories, DTRC learns a temporal distance in environment steps, allowing forward and reverse costs to differ. The temporal critic guides action learning through recorded behavior and model-predicted improvements. We study the resulting geometry and control performance across ten visual tasks.

Learning a geometry of goal reaching

Offline images provide temporal supervision for a directed representation. The temporal critic evaluates progress and guides policy training. — Temporal supervision organizes frozen visual features around goal-reaching cost and guides action learning.

From predictive features to temporal structure

PushT: frozen visual features, control representations, symmetric coordinates, and directed potentials. — The same observations in four learned spaces, projected independently with PCA. Color: normalized episode time.

PushT: temporal calibration, symmetric and directed costs, and one-step progress on held-out state–goal pairs. — Gray curves show the symmetric component of DTRC; colored curves show its directed temporal distance.

Representation evolution

The same observations throughout training, in a fixed PCA projection.

PushT: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

TwoRoom: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Reacher: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Cube: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

PointMaze: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Cube-Double: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Scene: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

AntMaze: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

HumanoidMaze: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Puzzle-3x3: EMA control representation across 29 saved training checkpoints, in fixed PCA coordinates.

Episode time01

Temporal geometry that informs behavior

Goal-conditioned control across ten visual tasks.

Success-rate comparison of DTRC, GC-IDM, GCBC, GC-IQL, HILP, and LeWM on four visual tasks. — Success rate: mean ± standard deviation over five evaluation seeds.

DTRC improves over LeWM by 14.7, 24.0, and 30.0 percentage points on TwoRoom, Reacher, and Cube.

Success rate (%) · mean ± standard deviation over five evaluation seeds
Method	PointMaze	Cube-Double	Scene	AntMaze-M	HumanoidMaze-M	Puzzle-3x3

DTRC ranks among the top two methods on five of six tasks. Bold: best mean; underline: second-best mean.

Success rate (%) · mean ± standard deviation over five evaluation seeds
Configuration	PushT	TwoRoom	Reacher	Cube

Temporal representation learning and advantage weighting improve the same policy by 5.3–32.0 percentage points, without model-assisted targets.

From a visual goal to coordinated action

Selected successful rollouts in the environment (left) and control representation (right).

PushT: synchronized environment and latent trajectory. — PushT

Reacher: synchronized environment and latent trajectory. — Reacher

TwoRoom: synchronized environment and latent trajectory. — TwoRoom

Cube: synchronized environment and latent trajectory. — Cube

PointMaze: synchronized environment and latent trajectory. — PointMaze

Puzzle-3x3: synchronized environment and latent trajectory. — Puzzle-3x3

Gray: reference states. Color: executed trajectory. Star: goal.