In large-scale classification problems, the data set may be faced with frequent updates, e.g., a small ratio of data is added to or removed from the original data set. In this case, incremental learning, which updates an existing classifier by explicitly modeling the data modification, is more efficient than retraining a new classifier from scratch. Conventional incremental learning algorithms try to solve the problem exactly. However, for some tasks, we are only interested in the lower and upper bound for some values relevant to the coefficient vector of the updated classifier without really solving it, e.g., determining whether we should update the classifier or performing some sensitivity analysis tasks. To deal with these such tasks, we propose an algorithm to make rational inferences about the updated classifier with low computational complexity. Specifically, we present a method to calculate tighter bounds of a general linear score for the updated classifier such that it's more accurate to estimate the range of interest than existing papers. The proposed method can be applied to any linear classifiers with differentiable convex L2 regularization loss function. Both theoretical analysis and experiment results show that the proposed approach is superior to existing methods.