In this paper, we investigate the effect of modulation rate on the energy efficiency (EE) of a twohop communication system with the assistance of a halfduplex decodeandforward (DF) relay, considering the given bit error rate (BER) and using MQAM modulation. First, we formulate the model of optimum transmission power, subject to the given BER. Then the minimized transmission power under the constraint of the given BER is obtained. For EE problem, the circuit power consumption of devices can not be ignored, so with the circuit power consumption considered, we derive and analyze the expression of EE based on the modulation rate. According to convex optimization theory, the fact that EE is a quasiconcave function of modulation rate has been proven. We find that when the modulation rate increases, the EE will first increases and then decreases. Finally, the theoretical analysis is corroborated by simulation results. We also show that the location of relay node has effect on EE.