Video Subtitle Erasure

ReST

Residual Spatio-Temporal Diffusion for One-Step Video Subtitle Erasure

ByteDance

Try ReST Code Video Gallery BibTeX

One model evaluation Source-preserving restoration mask-free mode + guided mode

Qualitative ReST results on multilingual subtitles, transition captions, watermarks, and titles. — ReST removes diverse overlaid text while retaining scene structure, motion, and fine appearance.

34.01 PSNR / dB +5.95 dB vs. strongest baseline

0.031 Seconds / frame One-step endpoint prediction

32.1 Frames / second About 4x faster than fastest baseline

73.2% Lower VFID Relative to DiffuEraser

Abstract

Restore the evidence. Preserve the video.

Video subtitle erasure requires removing overlaid text while faithfully preserving spatio-temporal content. Existing inpainting methods treat the affected area as a hole, discarding useful evidence that survives beneath translucent glyphs, shadows, and antialiased boundaries.

ReST instead anchors editing at the subtitled source latent and predicts the complete source-to-clean residual in one model evaluation. Masked Temporal Evidence Aggregation retrieves uncontaminated observations from nearby frames, while Global Appearance Conditioning maintains clip-level illumination and color. Region-aware latent and RGB supervision recovers fine text boundaries, and conservative composition limits synthesis to subtitle-affected regions when localization is available.

Method

A short path from source to clean

Most pixels are already correct. ReST predicts only the localized correction instead of regenerating the full video from Gaussian noise.

Overview of the ReST restoration framework with GAC, META, and erasure latent prediction. — Source and noised latents are aligned with global appearance conditioning and masked temporal aggregation, then decoded from the predicted erasure latent into the subtitle-free video.

Source-Anchored Residual

Uses the structured source latent as the starting point and predicts the complete source-to-clean endpoint correction.

Masked Temporal Evidence

MTEA aggregates only visible, uncontaminated observations in a normalized five-frame window.

Global Appearance

GAC broadcasts a pooled clip descriptor to keep restored content consistent in color and illumination.

Regional Supervision

Weighted latent learning and localized RGB patch decoding recover thin strokes, outlines, and shadows.

Long-Video Inference

Shot-aware chunks, overlap blending, and optional conservative paste-back prevent drift over long sequences.

Subtitled latent z^s

Predicted residual f_θ(z^s)

Clean latent ẑ⁰

Results

See ReST in motion

Watch the narrated overview, then explore individual restoration cases in the interactive gallery below.

1:35 6 benchmark + 5 real cases Narrated

Interactive Gallery

Sixteen cases. One live preview.

Hover to preview comparison clips ranging from 6 to 15 seconds. Select a case to inspect it in the focus player, or leave auto-cycle on to let the gallery move through the collection.

16 local video cases

Live preview 01 / 16

Real-world case

Landscape Animation

Dense dialogue and stylized overlays.

Preview clips are compressed from the local comparison videos for fast browsing. The full narrated demo remains available above.

Full-frame and subtitle-region comparison of ReST against CLEAR, DiffuEraser, MiniMax-Remover, and ProPainter. — Each cell shows the full frame and an aligned subtitle-region crop. ReST removes text while preserving local structure and texture.

Benchmark

Fidelity without the iterative cost

Paired Chinese-English subtitle benchmark, 200 videos. Best values are highlighted.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	VFID ↓	Flow Mean ↓	s/frame ↓
CLEAR	21.02	0.77	0.20	3.12	0.22	0.85
ProPainter	28.06	0.87	0.15	1.89	0.09	0.12
MiniMax-Remover	27.27	0.85	0.16	1.69	0.15	0.35
DiffuEraser	28.00	0.89	0.13	1.52	0.10	0.44
ReST Ours	34.01	0.95	0.05	0.41	0.07	0.03

26.03 dB Subtitle-region PSNR

+5.09 dB over ProPainter inside the edited region.

9.15 Subtitle-region MAE

45.2% lower masked error than ProPainter.

2.5x Faster training

N video tokens and 6 days, versus 2N tokens and 15 days.

Deployment

One model, two inference modes

No localization required

mask-free mode

The model jointly identifies and removes subtitles from the complete source observation. Long-video chunks are joined using overlap blending.

Strict background identity

guided mode

A supplied localization prior activates MTEA and conservative feathered paste-back, copying source pixels outside subtitle neighborhoods.

Citation

Reference ReST

Cite the project page when referencing ReST results, videos, or implementation details.

@misc{rest_project,
  title     = {ReST: Residual Spatio-Temporal Diffusion for
               One-Step Video Subtitle Erasure},
  organization = {ByteDance},
  howpublished = {\url{https://yahooo-m.github.io/ReST/}},
  year      = {2026}
}