Ensuring fairness is a crucial aspect of Federated Learning (FL), which enables the model to perform consistently across all clients. However, designing an FL algorithm that simultaneously improves global model performance and promotes fairness remains a formidable challenge, as achieving the latter often necessitates a trade-off with the former.To address this challenge, we propose a new FL algorithm, FedEBA+, which enhances fairness while simultaneously improving global model performance. Our approach incorporates a fair aggregation scheme that assigns higher weights to underperforming clients and a novel model update method for FL. Besides, we show the theoretical convergence analysis and demonstrate the fairness of our algorithm. Experimental results reveal that FedEBA+ outperforms other SOTA fairness FL methods in terms of both fairness and global model's performance.