TacStack — Cross-sensor tactile semantics for robotics

Principles that hold the stack together

Four commitments, applied to every adapter, model and integration decision.

Raw-first

Keep source payloads and metadata alongside derived representations, so processing remains traceable.

Semantics-first

One Observation and Event contract spans sensors and modalities — images, taxel matrices, force/torque.

Edge-first

Core depends on neither ROS2, cloud services nor training frameworks. Runs where the robot runs.

Integration-first

Record with MCAP, inspect with Rerun and exchange datasets with LeRobot. ROS2 support is on the roadmap.

From a raw signal to a useful event.

An offline workflow for developing and evaluating tactile applications, with the original data kept alongside derived outputs.

Dataset

Open-X-Tactile tar / zarr episodes; MCAP replay for recorded runs.

Observation

TactileObservation — shared metadata, sensor-specific arrays and raw payloads.

Runtime

Contact / slip baselines via builtin scorers or ONNX artifacts.

Events

TactileEvents marked on a synchronized Rerun timeline, annotatable.

Benchmark

Reproducible per-episode reports with quality and calibration tracking.

Five sensors. One tactile interface.

Five sensor protocol layers — serial matrix, six-axis F/T, joint torque and vision-based tactile — fully tested offline with unit tests pinned to vendor documentation.

Sensor	Output	Link	Status
M0404Sresistive matrix kit	4×4 matrix · 16 taxels	UART 115200, active push	● ready
PaXini PX-6AX GEN3tactile skin	per-point 3-axis force + resultant	UART 921600, req–resp	● ready
PaXini PX6Dsix-axis F/T	Fx, Fy, Fz, Mx, My, Mz · float32	USB / RS485 · 1 kHz	● ready
PaXini PX3Qjoint torque	Mx, My, Mz · N·m · 30/50/100 FS	USB / RS485 · 921600	● ready
Meta DIGITvision-based tactile	320×240 RGB gel image	USB UVC · QVGA 60fps	● ready

Milestones

From core contracts to cross-sensor semantics — delivered milestone by milestone.

PHASE 0–2

Contracts, data path, replay

Package and core contracts; Open-X-Tactile adapter for image / matrix / F/T streams; MCAP export and Rerun replay with C/S/U annotation.

PHASE 3

Runtime & benchmark

Builtin and ONNX scoring backends, temporal contact / slip baselines and repeatable reports. Model-quality evaluation requires labeled data.

PHASE 4

Quality & calibration

Two-level capability validation, dataset quality health reports, calibration provenance, per-stream benchmark grouping.

PHASE 5

First live sensor

Live serial adapter, live record / replay / inference, device reconnect handling and a 30-minute stability run.

PHASE 6

v0.1 release

External-user Quickstart, English documentation and the v0.1 release.

Start with a working offline demo.

Python 3.12+. Install the alpha from PyPI, or clone from source for the full offline tour. No sensor or dataset download needed.

The bundled fixture and the event below are synthetic. Use your own recordings to evaluate model behavior. PyPI hosts the alpha (0.1.0a1); the guided demo needs a repository checkout.

terminal

# install the alpha from PyPI (Python 3.12+)
pip install --pre tacstack

# verify offline — no data download, no hardware
tacstack version
tacstack contract-demo

# from source: guided demo + bundled fixture + Rerun replay
git clone https://github.com/mailes/TacStack.git && cd TacStack
uv sync --extra rerun
uv run tacstack demo

contract_example.py

from tacstack import TactileEvent

sample = TactileEvent(
    timestamp_ns=0,
    sensor_id="synthetic",
    kind="contact_begin",
    probability=1.0,
    model_id="contract-example",
    latency_ms=0.0,
)