Physics-Aware Multimodal Fusion for Dense EMF Exposure Map Prediction

Graduate Researcher · SmartAILab, Soongsil University (Prof. Seongsin Kim) · Jan 2025 – present

Ray-tracing simulators can accurately estimate how 5G electromagnetic fields (EMF) spread across complex environments, but they are often slow and expensive. Motivated by this, I study how to predict dense EMF exposure maps directly with Vision Transformers — by feeding them the physics of propagation rather than treating the task as black-box image prediction.

Physics-aware input encoding

Example encoded input channels Example encoded channels — building layout & beam, plus transmitter-relative distance, proximity, and bearing maps [3].

Input modalities
Input modalities and the per-pixel EMF prediction target [5].
2D spatial mapping
2D spatial mapping → an 8×500×500 ViT input (early fusion) [2].
  • Turned antenna radiation patterns, transmitter coordinates, numeric base-station specs, and GIS / building layouts into physically-meaningful 2-D maps stacked as ViT input channels [1, 2].
  • Designed transmitter-relative spatial channels (distance, proximity, bearing from the antenna) so the model knows where the source is [3].
  • Built a physically-informed multi-channel input so the network sees the same variables a deterministic propagation model would [1].

Fusion and conditioning of physical modalities

Token fusion architecture
Token fusion (mid fusion): physical modalities are tokenized and fused with image tokens inside the Transformer [2].
Feature-level conditioning architecture (HRFormer with FiLM and cross-attention)
Feature-level conditioning architecture — HRFormer backbone with per-stage FiLM from numeric antenna parameters and cross-attention over radiation-pattern tokens [3].
  • Token fusion: converted radiation patterns and numeric antenna parameters into tokens and fused them with image tokens inside the Transformer [2].
  • Spatial early fusion: projected physical variables into 2D maps and stacked them with image channels to preserve spatial alignment [2].
  • Feature-level conditioning: used FiLM to inject scalar antenna parameters into backbone features and cross-attention to fuse radiation-pattern tokens with spatial features [3].
  • Compared early fusion, token-level fusion, and feature-level conditioning to analyze how different physical modalities should interact with visual representations [2, 3].

Benchmarking & comparative analysis

Predicted EMF maps across CNN / ViT models vs. ground truth EMF predictions: ground truth, baseline, token fusion, and 2D spatial mapping
Predicted EMF distributions across CNN / ViT models vs. ground truth [1]. Ground truth vs. baseline and multimodal fusion — token fusion and 2D spatial mapping [2].
Model MAE (%) ↓ PSNR (dB) ↑ SSIM ↑
IEEE Access [1] — standard CNN / ViT architectures
U-Net 7.92 21.41 0.90
ViT 6.00 20.74 0.89
SegFormer 5.86 21.26 0.89
Swin Transformer 5.69 22.15 0.90
HRFormer 5.45 22.39 0.91
CEIC [2] — own protocol · multimodal fusion · ImageNet-pretrained (*)
ViT* (token fusion) 4.00 25.46 0.93
ViT* (2D mapping) 2.00 28.33 0.94

[1] reports masked MAE, [2] reports MAE (both shown as %); * = ImageNet-pretrained. Absolute PSNR / SSIM follow each paper's own evaluation protocol, so [1] and [2] are not a controlled side-by-side benchmark.

selected qualitative and quantitative results [1, 2]

  • Benchmarked CNN and Vision Transformer architectures for dense EMF prediction, identifying HRFormer as a strong high-resolution backbone [1].
  • Conducted controlled ablations to quantify the contribution of spatial channels, FiLM conditioning, radiation-pattern cross-attention, and loss design [3].
  • Multi-metric quantitative analysis (MAE, RMSE, PSNR, SSIM, Hotspot IoU) and qualitative EMF-map inspection [1, 3].
  1. Doeon Kim, Dongryul Park, et al. “Estimation of Electromagnetic Field Strength: Experiments Using Vision Transformers.” IEEE Access, 2025.
  2. Doeon Kim, Seongsin Kim. “Multimodal Physical Data Fusion using ViT for 5G Electromagnetic Field Estimation.” CEIC, 2025.
  3. Doeon Kim, Dongryul Park, et al. “Multi-Modal Conditioned High-Resolution Transformer for Urban Electromagnetic Field Map Prediction.” arXiv, 2026.
  4. Hyunwoo Choi, Doeon Kim, Seongsin Kim. “Multimodal pretraining for radio-map estimation.” Under review, 2026.
  5. D. Park, S. Ryu, et al. “5G Base Station Electromagnetic Field Strength Estimation Method in Complex Hotspot Area using Deep Learning.” IEEE EMC+SIPI, 2024.