Self-Supervised WiFi Sensing for 3D Human Pose

Full-time Researcher · VIP Lab, Soongsil University (Prof. Seongheum Kim) · Dec 2025 – present

I study whether WiFi channel measurements (CSI) can recover 3D human pose without cameras. Robust WiFi pose estimation faces three bottlenecks: (i) cross-domain generalization is fragile — accuracy drops under domain shift, e.g. a new room or moved transceivers; (ii) label scalability is limited — CSI pose datasets need camera-based annotation in a few fixed-hardware rooms; and (iii) CSI is noisy and hardware-dependent — flattening it into 2D image grids mixes its subcarrier, time, and link axes. I redesigned self-supervised learning around CSI’s physical structure, needing far fewer labels and generalizing across rooms and hardware. This work was accepted at ECCV 2026 [1].

Encoding WiFi CSI: axis-preserving tokenization & link masking

• Axis-preserving tokenization: kept CSI’s physical tensor (C, T, L) = (60, 20, 9) — subcarriers × time × antenna links — rather than flattening it into a 2D image, giving 180 tokens, one per (time, link); each token then holds the representation of a specific antenna link at a specific time, not an arbitrary image patch.
• Link masking: masked entire antenna-link columns (5 of 9 links) and predicted them from the rest, forcing the encoder to learn the cross-link spatial correlation that encodes 3D structure.
• Result: axis-preserving tokenization cut error by 14.77 mm MPJPE (111.85 → 97.08) over a flattened-spectrogram baseline; link masking was best of four masking strategies — 97.08 mm / 92.56% PCK@50, beating random masking by 8.06 mm [1].

Ray-traced synthetic CSI (Sionna RT)

• Hypothesis: for self-supervised pretraining, dynamics diversity matters more than anatomical fidelity.
• Why simulate: real CSI needs dedicated rooms, synchronized receivers, and co-located motion capture for labels, so datasets stay small and tied to one hardware setup.
• Ray-tracing pipeline (NVIDIA Sionna RT): generates benchmark-compatible CSI from simple geometric primitives — spheres, cubes, cylinders with randomized trajectories — with no human meshes and no motion capture; ~90K frames in ~10 GPU-hours on one RTX 4090.
• Result: real + simulated pretraining beat real-only, cutting multi-person MPJPE from 97.1 to 93.5 mm (−3.6 mm); even geometric primitives alone reached 100.1 mm, approaching real data [1].

Self-supervised learning: a CSI-native JEPA

Phase 1 — self-supervised pre-training: the context encoder predicts the link-masked tokens’ latent embeddings, supervised by an EMA target encoder (Smooth L1). Phase 2 — fine-tuning the pretrained encoder with a PETR pose head.

• CSI-native JEPA: instead of reconstructing raw CSI (which would force the encoder to memorize hardware artifacts), the model predicts the masked links’ latent embeddings — a ViT context encoder (12 layers, 512-d) with an EMA target encoder (Smooth L1 loss), fine-tuned with a PETR-style decoder for multi-person 3D keypoints.
• Result: under an identical backbone/decoder, four vision-native SSL objectives (SimMIM, MAE, BYOL, MoCo v3) all degraded results below training from scratch (102.4 mm), while WiFi-JEPA was the only one to improve it (97.1 mm, −33.2 mm vs. the best baseline) [1].
• End-to-end: state-of-the-art 76.8 mm single-person / 93.5 mm multi-person MPJPE (−14.7% / −12.8% over prior WiFi methods), with cross-environment error halved (324.2 vs. 626.4 mm) in leave-one-room-out tests [1].