On unbiased performance evaluation for protein inference

This letter is a response to the comments of Serang (2012) on Huang and He (2012) in Bioinformatics. Serang (2012) claimed that the parameters for the Fido algorithm should be specified using the grid search method in Serang et al. (2010) so as to generate a deserved accuracy in performance comparison. It seems that it is an argument on parameter tuning. However, it is indeed the issue of how to conduct an unbiased performance evaluation for comparing different protein inference algorithms. In this letter, we would explain why we don't use the grid search for parameter selection in Huang and He (2012) and show that this procedure may result in an over-estimated performance that is unfair to competing algorithms. In fact, this issue has also been pointed out by Li and Radivojac (2012).