Cross-dataset emotion recognition as an extremely challenging task in the field of EEG-based affective computing is influenced by many factors, which make the universal models yield unsatisfactory results. Facing the situation that lack of EEG information decoding researches, we first analyzed the impact of different EEG information(individual, session, emotion, trial) to emotion recognition by sample space visualization, sample aggregation phenomenon quantification, and energy pattern analysis on five public datasets. And based on these phenomena and patterns, we provided the processing methods and interpretable work of various EEG differences. Through the analysis of emotional feature distribution patterns, Individual Emotional Feature Distribution Difference(IEFDD) was found. After analyzing the limitations of traditional modeling approach suffering from IEFDD, we proposed the Weight-based Channel-model Matrix Framework(WCMF). In order to characterize emotional feature distribution patterns reasonably, four weight extraction methods were designed, and the optimal of them is Correction T-test(CT) weight extraction method. Finally, the performance of WCMF was validated on cross-dataset tasks in two kinds of experiments that simulated different practical scenarios, the results showed WCMF had more stable and better emotion recognition ability.