Device to device (D2D) communication can be based on cellular facilities to improve resource utilization, user throughput and save battery energy. In D2D network, mode selection and resource allocation are the key issues. In order to improve the sum rate and spectrum efficiency of D2D communication, a scheme of joint mode selection, power and resource block allocation is proposed. Firstly, the mode selection criteria are selected according to the user′s geographical location to help the user select the corresponding communication mode; Then, for the multiplexing communication mode, the asynchronous dominant action evaluation (A3C) algorithm based on deep reinforcement learning is used to allocate resource blocks and power to different D2D users. The simulation results show that the joint optimization scheme based on A3C algorithm proposed in this paper has fast convergence speed and better performance than other algorithms.