The Kalman filter provides an optimal estimation for a linear system with Gaussian noise. However when the noises are non-Gaussian in nature, its performance deteriorates rapidly. For non-Gaussian noises, maximum correntropy Kalman filter (MCKF) is developed which provides an improved result. But when the system model differs from nominal consideration, the performance of the MCKF degrades. For such cases, we have proposed a new robust filtering technique which maximize a cost function defined by exponential of weighted past and present errors along with the Gaussian kernel function. By solving this cost criteria we have developed prior and posterior mean and covariance matrix propagation equations. By maximizing the correntropy function of error matrix, we have selected the kernel bandwidth value at each time step. Further the conditions for convergence of the proposed algorithm is also derived. Two numerical examples are presented to show the usefulness of the new filtering technique.