Vehicle to Everything (V2X) communication, which can effectively improve traffic safety and mobility, is one of the key technologies in vehicle deployment scenarios. V2X communication links need to meet different Quality of Service (QoS) requirements for different applications, such as the latency and reliability requirements of Vehicle to Vehicle (V2V) links. Considering rapid changes in wireless channels due to vehicles high mobility, while ensuring QoS constraints for different vehicle links and improving the robustness of dynamic networks, a joint optimization framework based on Federal Deep Reinforcement Learning (FDRL) for spectrum allocation and power control is proposed. The framework first proposes the corresponding optimization problems according to different vehicle link requirements, and defines the state space, action space and reward function for reinforcement learning. Then the joint deep learning reinforcement learning training framework is given. Finally, the optimal spectrum allocation and power control strategies are found by distributed vehicle-side reinforcement learning and base station aggregation averaging training. Simulation results show that the proposed framework can improve the total transmission rates of all the vehicle-to-infrastructure (V2I) users and guarantee the robustness of the network when new vehicles are connected to the network compared with other comparative algorithms.