In order to fully explore the potential connection between the rolling bearing fault types and the vibration signal to improve the diagnosis accuracy, a fault diagnosis method based on the hybrid model of scale adaptive convolutional neural network (SACNN) and modified gated recurrent unit (MGRU) is proposed. To begin with, a scale adaptive factor is proposed to obtain appropriate CNN window size for extracting local fault information from the raw signal more effectively, and scaled exponential liner unit (SELU) is introduced into CNN to improve the robustness of its training process. Subsequently, SELU is embedded into GRU to further enhance the network stability and the network structure of GRU is ameliorated to enhance the temporal feature extraction ability, thereby extracting temporal feature from the local fault information more fully. Finally, the softmax function is applied for recognizing fault types. The experimental comparison and analysis reveal that the proposed method achieves better convergence and stability, can effectively mine the fault information contained in the vibration signal for accurately recognizing the rolling bearing fault types at different speeds with the recognition accuracies higher than 99.5%, which has certain application value.