Previous hypergraph expansions are solely carried out on either vertex level or hyperedge level, thereby missing the symmetric nature of data co-occurrence, and resulting in information loss. To address the problem, this paper treats vertices and hyperedges equally and proposes a new hypergraph formulation named the \emph{line expansion (LE)} for hypergraphs learning. The new expansion bijectively induces a homogeneous structure from the hypergraph by treating vertex-hyperedge pairs as "line nodes". By reducing the hypergraph to a simple graph, the proposed \emph{line expansion} makes existing graph learning algorithms compatible with the higher-order structure and has been proven as a unifying framework for various hypergraph expansions. For simple graphs, we demonstrate that learning algorithms defined on \emph{LE}s tie with their performance on the original graphs, implying that no loss of information occurs in the expansion. For hypergraphs, we show that learning over the new representation leads to algorithms that beat all prior state-of-the-art hypergraph learning baselines.