Abstract
Large scale artificial intelligence (AI) models possess excellent capabilities in semantic representation and understanding, making them particularly well-suited for semantic encoding and decoding. However, the substantial scale of these AI models imposes unacceptable computational resources and communication delays. To address this issue, we propose a semantic communication scheme based on robust knowledge distillation (RKD-SC) for large scale model enabled semantic communications. In the considered system, a transmitter extracts the features of the source image for robust transmission and accurate image classification at the receiver. To effectively utilize the superior capability of large scale model while make the cost affordable, we first transfer knowledge from a large scale model to a smaller scale model to serve as the semantic encoder. Then, to enhance the robustness of the system against channel noise, we propose a channel-aware autoencoder (CAA) based on the Transformer architecture. Experimental results show that the encoder of proposed RKD-SC system can achieve over 93.3% of the performance of a large scale model while compressing 96.67% number of parameters. Code: https://github.com/echojayne/RKD-SC.