Presentation
Light-Bound Transformers: Hardware-Anchored Robustness for Silicon-Photonic Computer Vision Systems
DescriptionDeploying Vision Transformers (ViTs) on near-sensor analog accelerators demands training pipelines that are explicitly aligned with device-level noise and energy constraints. We introduce a compact framework for silicon-photonic execution of ViTs that integrates \textbf{measured hardware noise}, \textbf{robust attention training}, and an \textbf{energy-aware processing flow}. We first characterize bank-level noise in microring-resonator (MR) arrays—including fabrication variation, thermal drift, and amplitude noise—and convert these measurements into closed-form, activation-dependent variance proxies for attention logits and feed-forward activations. Using these proxies, we develop \emph{Chance-Constrained Training} (CCT), which enforces variance-normalized logit margins to bound attention rank flips, and a \emph{noise-aware LayerNorm} that stabilizes feature statistics without changing the optical schedule. These components yield a practical ``measure $\rightarrow$ model $\rightarrow$ train $\rightarrow$ run'' pipeline that optimizes accuracy under noise while respecting system energy limits. Hardware-in-the-loop experiments with MR photonic banks show that our approach restores near-clean accuracy under realistic noise budgets, with no in-situ learning or additional optical MACs.
Event Type
Research Manuscript
TimeTuesday, July 2812:16pm - 12:30pm PDT
LocationMtg Room 101B
Similar Presentations