Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Exact Exponent in Optimal Rates for Crowdsourcing

May 26, 2016
Chao Gao, Yu Lu, Dengyong Zhou


Share this with someone who'll enjoy it:


In many machine learning applications, crowdsourcing has become the primary means for label collection. In this paper, we study the optimal error rate for aggregating labels provided by a set of non-expert workers. Under the classic Dawid-Skene model, we establish matching upper and lower bounds with an exact exponent $mI(\pi)$ in which $m$ is the number of workers and $I(\pi)$ the average Chernoff information that characterizes the workers' collective ability. Such an exact characterization of the error exponent allows us to state a precise sample size requirement $m>\frac{1}{I(\pi)}\log\frac{1}{\epsilon}$ in order to achieve an $\epsilon$ misclassification error. In addition, our results imply the optimality of various EM algorithms for crowdsourcing initialized by consistent estimators.

* To appear in the Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 2016 


   Access Paper Source



Share this with someone who'll enjoy it: