In order to solve the problem that the audio signal processing in the voice communication system has a large amount of data, a lot of stray signals, and the received audio signals of the frequency modulation receiver are large and small, a lightweight audio signal processing algorithm is proposed, and based on this algorithm, the audio signal receiving and automatic gain control are realized on the FPGA platform. The algorithm combines digital down-conversion technology, multistage extraction filtering technology and digital AGC technology, and is applied to the audio signal processing system. The RF analog signal received from the upper antenna is converted into baseband audio signal through analog-to-digital conversion and digital down-conversion, and the stray signal in the baseband signal is filtered through four-stage extraction filtering, reducing the complexity and power consumption of the system. At the same time, the digital AGC controls and adjusts the baseband audio signal to output a more stable audio signal. The experimental results show that the algorithm can effectively reduce the information rate from 102.4MHz to 32KHz, reduce the computation burden, improve the signal quality, and reduce the resource utilization of FPGA. And the automatic gain control adjustment of audio signal is realized, and the adjustment time is only 12.8μs, which meets the power stability time of the receiver.