In this work, we investigate a novel simultaneous transmission and reflection reconfigurable intelligent surface (RIS)-assisted multiple-input multiple-output downlink system, where three practical transmission protocols, namely, energy splitting (ES), mode selection (MS), and time splitting (TS), are studied. For the system under consideration, we maximize the weighted sum rate with multiple coupled variables. To solve this optimization problem, a block coordinate descent algorithm is proposed to reformulate this problem and design the precoding matrices and the transmitting and reflecting coefficients (TARCs) in an alternate manner. Specifically, for the ES scheme, the precoding matrices are solved using the Lagrange dual method, while the TARCs are obtained using the penalty concave-convex method. Additionally, the proposed method is extended to the MS scheme by solving a mixed-integer problem. Moreover, we solve the formulated problem for the TS scheme using a one-dimensional search and the Majorization-Minimization technique. Our simulation results reveal that: 1) Simultaneous transmission and reflection RIS (STAR-RIS) can achieve better performance than reflecting-only RIS; 2) In unicast communication, TS scheme outperforms the ES and MS schemes, while in broadcast communication, ES scheme outperforms the TS and MS schemes.