Yi: Open Foundation Models by 01.AI

We introduce the Yi model family, a series of language and multimodal models that demonstrate strong multi-dimensional capabilities. The Yi model family is based on 6B and 34B pretrained language models, then we extend them to chat models, 200K long context models, depth-upscaled models, and vision-language models. Our base models achieve strong performance on a wide range of benchmarks like MMLU, and our finetuned chat models deliver strong human preference rate on major evaluation platforms like AlpacaEval and Chatbot Arena. Building upon our scalable super-computing infrastructure and the classical transformer architecture, we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts. For pretraining, we construct 3.1 trillion tokens of English and Chinese corpora using a cascaded data deduplication and quality filtering pipeline. For finetuning, we polish a small scale (less than 10K) instruction dataset over multiple iterations such that every single instance has been verified directly by our machine learning engineers. For vision-language, we combine the chat language model with a vision transformer encoder and train the model to align visual representations to the semantic space of the language model. We further extend the context length to 200K through lightweight continual pretraining and demonstrate strong needle-in-a-haystack retrieval performance. We show that extending the depth of the pretrained checkpoint through continual pretraining further improves performance. We believe that given our current results, continuing to scale up model parameters using thoroughly optimized data will lead to even stronger frontier models.

Evaluating the Effect of Crutch-using on Trunk Muscle Loads

As a traditional tool of external assistance, crutches play an important role in society. They have a wide range of applications to help either the elderly and disabled to walk or to treat certain illnesses or for post-operative rehabilitation. But there are many different types of crutches, including shoulder crutches and elbow crutches. How to choose has become an issue that deserves to be debated. Because while crutches help people walk, they also have an impact on the body. Inappropriate choice of crutches or long-term misuse can lead to problems such as scoliosis. Previous studies were mainly experimental measurements or the construction of dynamic models to calculate the load on joints with crutches. These studies focus only on the level of the joints, ignoring the role that muscles play in this process. Although some also take into account the degree of muscle activation, there is still a lack of quantitative analysis. The traditional dynamic model can be used to calculate the load on each joint. However, due to the activation of the muscle, this situation only causes part of the load transmitted to the joint, and the work of the chair will compensate the other part of the load. Analysis at the muscle level allows a better understanding of the impact of crutches on the body. By comparing the levels of activation of the trunk muscles, it was found that the use of crutches for walking, especially a single crutch, can cause a large difference in the activation of the back muscles on the left and right sides, and this difference will cause muscle degeneration for a long time, leading to scoliosis. In this article taking scoliosis as an example, by analyzing the muscles around the spine, we can better understand the pathology and can better prevent diseases. The objective of this article is to analyze normal walking compared to walking with one or two crutches using OpenSim software to obtain the degree of activation of different muscles in order to analyze the impact of crutches on the body.