Self-Supervised WiFi Sensing for 3D Human Pose

Dec 2025 – present  ·  Project page ↗

I study whether WiFi channel measurements (CSI) can recover 3D human pose without cameras. Robust WiFi pose estimation faces three bottlenecks: (i) cross-domain generalization is fragile — accuracy drops in a new room or with moved transceivers; (ii) label scalability is limited — pose labels need camera-based annotation in a few fixed-hardware rooms; (iii) CSI is noisy and hardware-dependent — flattening it into image-like 2D grids mixes its subcarrier, time, and link axes. My answer is WiFi-JEPA: self-supervised learning redesigned around CSI’s physical structure — far fewer labels, robust transfer across rooms and hardware. Accepted at ECCV 2026 [1].

Encoding WiFi CSI: axis-preserving tokenization & link masking

Axis-preserving CSI tokenization vs. flattened spectrogram
Axis-preserving tokenization keeps the (C, T, L) tensor — versus a flattened spectrogram that mixes the time and link axes.
Masking strategies on the time-link token grid
Link masking (a) removes whole antenna links, versus time, multi-block, and random masking.
  • Axis-preserving tokenization: keep the CSI tensor (C, T, L) = (60, 20, 9) — subcarriers × time × antenna links — instead of flattening it into a 2D image; each of the 180 tokens is one antenna link at one time step, not an arbitrary patch.
  • Link masking: hide 5 of 9 antenna links entirely and predict them from the rest — the encoder must learn the cross-link spatial correlation that encodes 3D structure.
  • Result: tokenization alone cuts 14.77 mm MPJPE (111.85 → 97.08) vs. a flattened-spectrogram baseline; link masking beats time, block, and random masking — 97.08 mm / 92.56% PCK@50, 8.06 mm ahead of random [1].
Key takeaway — Reading CSI on its true physical axes, and masking whole antenna links, teaches the encoder the cross-link structure that flattening destroys.

Ray-traced synthetic CSI (Sionna RT)

Ray-tracing CSI simulation pipeline
Randomized scenes of geometric primitives (Stage 1) rendered with Sionna RT at 20 passes per frame (Stage 2).
WiFi-JEPA overview: simulated/real CSI to 3D human pose
Simulated and real CSI generation → self-supervised model → 3D human-pose prediction (ground truth vs. prediction).
  • Hypothesis: for pretraining, dynamics diversity matters more than anatomical fidelity.
  • Why simulate: real CSI needs dedicated rooms, synchronized receivers, and motion capture for labels — datasets stay small and hardware-specific.
  • Ray-tracing pipeline (NVIDIA Sionna RT): benchmark-compatible CSI from randomly moving geometric primitives — spheres, cubes, cylinders — with no human meshes and no motion capture; ~90K frames in ~10 GPU-hours on one RTX 4090.
  • Result: real + simulated pretraining cuts multi-person MPJPE from 97.1 to 93.5 mm; primitives alone reach 100.1 mm, nearly matching real data [1].
Key takeaway — Randomly moving primitives — no human bodies at all — pretrain CSI nearly as well as real data: motion diversity beats anatomical realism.

Self-supervised learning: a CSI-native JEPA

WiFi-JEPA architecture (Phase 1 pre-training, Phase 2 fine-tuning) Phase 1 — self-supervised pre-training: the context encoder predicts the link-masked tokens’ latent embeddings, supervised by an EMA target encoder (Smooth L1). Phase 2 — fine-tuning the pretrained encoder with a PETR pose head.

  • CSI-native JEPA: predict the masked links’ latent embeddings instead of reconstructing raw CSI — the encoder learns pose-relevant structure, not hardware artifacts.
  • Comparison with SSL objectives: under the same backbone, SimMIM, MAE, BYOL, and MoCo v3 all fall below training from scratch — WiFi-JEPA is the only objective that improves it.
  • End-to-end: state-of-the-art 76.8 mm single-person / 93.5 mm multi-person MPJPE (−14.7% / −12.8% vs. prior WiFi methods), with cross-environment error halved (324.2 vs. 626.4 mm) in leave-one-room-out tests [1].
Key takeaway — Predicting latent link embeddings rather than raw CSI is what makes self-supervision transfer on WiFi — SOTA accuracy that holds in unseen rooms.
  1. Doeon Kim, Jungyoon Lee, Seongheum Kim, Seongsin Kim. “WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation.” ECCV 2026.