We study the machine learning techniques applied to the lattice gauge theory's critical behavior, particularly to the confinement/deconfinement phase transition in the SU(2) and SU(3) gauge theories. We find that the neural network of the machine-learning algorithm, trained on 'bare' lattice configurations at an unphysical value of the lattice parameters as an input, builds up a gauge-invariant function, and finds correlations with the target observable that is valid in the physical region of the parameter space. In particular, if the algorithm aimed to predict the Polyakov loop as the deconfining order parameter, it builds a trace of the gauge group matrices along a closed loop in the time direction. As a result, the neural network, trained at one unphysical value of the lattice coupling $\beta$ predicts the order parameter in the whole region of the $\beta$ values with good precision. We thus demonstrate that the machine learning techniques may be used as a numerical analog of the analytical continuation from easily accessible but physically uninteresting regions of the coupling space to the interesting but potentially not accessible regions.