The irregularity and disorder of point clouds bring many challenges to point cloud analysis. PointMLP suggests that geometric information is not the only critical point in point cloud analysis. It achieves promising result based on a simple multi-layer perception (MLP) structure with geometric affine module. However, these MLP-like structures aggregate features only with fixed weights, while differences in the semantic information of different point features are ignored. So we propose a novel Point-Vector Representation of the point feature to improve feature aggregation by using inductive bias. The direction of the introduced vector representation can dynamically modulate the aggregation of two point features according to the semantic relationship. Based on it, we design a novel Point2Vector MLP architecture. Experiments show that it achieves state-of-the-art performance on the classification task of ScanObjectNN dataset, with 1% increase, compared with the previous best method. We hope our method can help people better understand the role of semantic information in point cloud analysis and lead to explore more and better feature representations or other ways.