Learning from distributed touch

Electronic skins have sparse, irregular sensor layouts. Tactile-JEPA learns directly from their tactile time series, using the sensor connectivity graph to choose which sensing elements to mask. By predicting their embeddings from the visible context, it learns representations that capture both local contact details and the global state of the sensing surface.

The pretrained encoder transfers to force estimation, in-hand and full-body pose estimation, object and action classification, and visuo-tactile policy learning. The paper reports 6.3% lower force error and 20.8% lower in-hand orientation error relative to the strongest reported baseline for each task. See the project results for the full comparisons.

Get the models

Each repository contains one pretrained encoder and its matching downstream checkpoint(s), with original PyTorch checkpoints, safetensors weights, saved configurations, metrics and checksums.

Socks Action and Socks Pose use separate pretraining sources. The three Xela heads share a single pretrained encoder. The model cards document the selected checkpoints and their evaluation protocols; individual checkpoint scores differ from the multi-seed averages in the paper.

How it works

1. Tokenize touch. Embed a short signal history for each sensing element.
2. Mask with topology. Combine connected local patches and globally distributed targets over the sensor graph.
3. Predict representations. Train a context encoder and predictor to match embeddings from an exponential-moving-average target encoder.
4. Reuse the encoder. Freeze the pretrained target encoder and train a task-specific head. The sensor graph is used for pretraining masks; downstream encoding does not require it as an input.

Explore the sensor layouts and masking strategies in the interactive visualization.

Download a pretrained encoder

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

weights_path = hf_hub_download(
    repo_id="Tactile-JEPA/tactile-jepa-xela",
    filename="pretrain/encoder.safetensors",
)
encoder_state_dict = load_file(weights_path)

Use the matching architecture and preprocessing from the code repository. These are project-native PyTorch models. Each model card provides the file layout and loading details; the code README covers installation, data preparation and training.

Datasets

Our experiments use existing public datasets. Please refer to their original releases for data access and licensing:

• Sparsh-skin — magnetic tactile skin on an Allegro hand.
• Tactile Socks / SensTextile — pressure-sensing textiles for activity and pose estimation.
• DECO-50 — bimanual manipulation; our policy experiments use the Task 4 Assembly subset.

Cite Tactile-JEPA

@article{kovtun2026tactilejepa,
  title   = {Tactile-JEPA: Topology-Aware Self-Supervised Representation
             Learning for Distributed Tactile Sensors},
  author  = {Kovtun Elizaveta and Konovalov Matvey and
             Sakhovskiy Andrey and Budennyy Semen},
  journal = {arXiv preprint arXiv:2609.24385},
  year    = {2026},
  url     = {https://arxiv.org/abs/2609.24385}
}

Released checkpoints are licensed under CC BY-NC 4.0; see each model repository for its license. Our code builds on Sparsh / Sparsh-X / Sparsh-Skin. We thank the authors of the original methods and datasets for making their work available.