Federated learning boosts large model customization in web3
DICE: Decentralized Incentive-driven Federated Learning for Efficient & Scalable Customization of Pretrained Models in Web3
Cryptography and SecurityDistributed, Parallel, and Cluster ComputingMachine Learning
Summary
Customizing big AI models for specific tasks usually needs lots of data, which many Web3 apps have. The authors present a new method called DICE that helps many users customize these large models safely and efficiently without needing powerful computers. It lets users work on parts of a model rather than the whole, includes checks to keep the model accurate, and uses a special voting system to add security when many people join. Their tests show this method works well for both vision and language AI models.
What this means in practice
- •For blockchain platform developers: Implement scalable and secure customization of large pretrained AI models with diverse Web3 users who have varying hardware capabilities.
- •For ai service providers: Build AI customization services that leverage federated learning on blockchain, allowing many participants to collaboratively improve large models without full data sharing.$Commercial implications: Enables offering secure, privacy-preserving AI customization services to decentralized user bases, differentiating product features.
Authors
Hong Lin, Zhongle Xie, Lidan Shou, Ke Chen, Gang Chen
Abstract
Customizing Pretrained Models (PMs) for downstream tasks demands task-specific data, which burgeoning Web3 applications provide abundantly. Blockchain-powered federated learning (BCFL) offers a secure and privacy-preserving paradigm to customize PMs among self-interested and untrusted Web3 participants. Yet, existing BCFL algorithms face severe efficiency and scalability challenges when applied to large PMs, due to their substantial computational and memory requirements and the hardware constraints of many participants. This paper introduces DICE, a novel BCFL algorithm that addresses the efficiency and scalability challenges of customizing large PMs in Web3 via two sets of synergistic techniques: (i) a dynamic progressive learning scheme and a model slice verification scheme to enable a wider range of honest participants, including those unable to deploy the full model, to engage in customizing PMs, while preserving model utility; (ii) a tree-structured committee consensus protocol and an efficiency-driven incentive scheme to address the dominant bottleneck that hinders existing BCFL algorithms from scaling to a larger number of data holders for customizing data-hungry large PMs, while maintaining security. Theoretical analysis and extensive experimental results on customizing vision and language PMs demonstrate DICE's efficiency, effectiveness, and security.