Presentation
FBS: Accelerating CNN Inference over RNS-CKKS with Fewer Bootstrapping Sparsity
DescriptionRNS-CKKS is a fully homomorphic encryption scheme supporting fixed-point arithmetic, widely used in privacy-preserving convolutional neural network (CNN) inference.
However, its significant computational overhead, especially from bootstrapping—the most costly operation—raises deployment costs for CNN inference over RNS-CKKS.
While sparsity has proven effective in reducing computational overhead for unencrypted CNN inference, its application to large datasets (e.g., ImageNet) with RNS-CKKS-based CNN inference remains under-explored, particularly in optimizing bootstrapping operations that dominate computation time.
In this work, we observe that sparsity in CNN can be exploited to reduce the bootstrapping overhead in RNS-CKKS-based CNN inference.
Based on this observation, we propose FBS, a framework that accelerates CNN inference over RNS-CKKS by leveraging Fewer Bootstrapping Sparsity to reduce bootstrapping costs.
We propose two sparsity patterns: eliminate missing input sparsity pattern and channel sparsity pattern, to reduce the number of bootstrapping calls during CNN inference.
An iterative latency optimization framework is then presented to identify the key layers for pruning and determine the sparsity patterns to achieve effective performance.
Results show that FBS can accelerate CNN inference over RNS-CKKS by up to 1.91 times with negligible accuracy loss.
FBS will be open-sourced.
However, its significant computational overhead, especially from bootstrapping—the most costly operation—raises deployment costs for CNN inference over RNS-CKKS.
While sparsity has proven effective in reducing computational overhead for unencrypted CNN inference, its application to large datasets (e.g., ImageNet) with RNS-CKKS-based CNN inference remains under-explored, particularly in optimizing bootstrapping operations that dominate computation time.
In this work, we observe that sparsity in CNN can be exploited to reduce the bootstrapping overhead in RNS-CKKS-based CNN inference.
Based on this observation, we propose FBS, a framework that accelerates CNN inference over RNS-CKKS by leveraging Fewer Bootstrapping Sparsity to reduce bootstrapping costs.
We propose two sparsity patterns: eliminate missing input sparsity pattern and channel sparsity pattern, to reduce the number of bootstrapping calls during CNN inference.
An iterative latency optimization framework is then presented to identify the key layers for pruning and determine the sparsity patterns to achieve effective performance.
Results show that FBS can accelerate CNN inference over RNS-CKKS by up to 1.91 times with negligible accuracy loss.
FBS will be open-sourced.
Event Type
Research Manuscript
TimeMonday, July 2710:56am - 11:10am PDT
LocationMtg Room 203C