Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs

Using FPGAs to accelerate ConvNets has attracted significant attention in recent years. However, FPGA accelerator design has not leveraged the latest progress of ConvNets. As a result, the key application characteristics such as frames-per-second (FPS) are ignored in favor of simply counting GOPs, and results on accuracy, which is critical to application success, are often not even reported. In this work, we adopt an algorithm-hardware co-design approach to develop a ConvNet accelerator called Synetgy and a novel ConvNet model called DiracNet. Both the accelerator and ConvNet are tailored to FPGA requirements. DiractNet, as the name suggests, is a ConvNet with only 1x1 convolutions while spatial convolutions are replaced by more efficient shift operations. DiracNet achieves competitive accuracy on ImageNet (89.0% top-5), but with 48x fewer parameters and 65x fewer OPs than VGG16. We further quantize DiracNet's weights to 1-bit and activations to 4-bits, with less than 1% accuracy loss. These quantizations exploit well the nature of FPGA hardware. In short, DiracNet's small model size, low computational OP count, ultra-low precision and simplified operators allow us to co-design a highly customized computing unit for an FPGA. We implement the computing units for DiracNet on an Ultra96 SoC system through high-level synthesis. The implementation only took 2 people 1 month to complete. Our accelerator's final top-5 accuracy of 88.3% on ImageNet, is higher than all the previously reported embedded FPGA accelerators. In addition, the accelerator reaches an inference speed of 72.8 FPS on the ImageNet classification task, surpassing prior works with similar accuracy by at least 12.8x.

Determination of subject-specific muscle fatigue rates under static fatiguing operations

Cumulative local muscle fatigue may lead to potential musculoskeletal disorder (MSD) risks {\color{red}, and subject-specific muscle fatigability needs to be considered to reduce potential MSD risks.} This study was conducted to determine local muscle fatigue rate at shoulder joint level based on an exponential function derived from a muscle fatigue model. Forty male subjects participated in a fatiguing operation under a static posture with a range of relative force levels (14% - 33%). Remaining maximum muscle strengths were measured after different fatiguing sessions. The time course of strength decline was fitted to the exponential function. Subject-specific fatigue rates of shoulder joint moment strength were determined. Good correspondence ($R^2>0.8$) was found in the regression of the majority (35 out of 40 subjects). Substantial inter-individual variability in fatigue rate was found and discussed.

Human Arm simulation for interactive constrained environment design

During the conceptual and prototype design stage of an industrial product, it is crucial to take assembly/disassembly and maintenance operations in advance. A well-designed system should enable relatively easy access of operating manipulators in the constrained environment and reduce musculoskeletal disorder risks for those manual handling operations. Trajectory planning comes up as an important issue for those assembly and maintenance operations under a constrained environment, since it determines the accessibility and the other ergonomics issues, such as muscle effort and its related fatigue. In this paper, a customer-oriented interactive approach is proposed to partially solve ergonomic related issues encountered during the design stage under a constrained system for the operator's convenience. Based on a single objective optimization method, trajectory planning for different operators could be generated automatically. Meanwhile, a motion capture based method assists the operator to guide the trajectory planning interactively when either a local minimum is encountered within the single objective optimization or the operator prefers guiding the virtual human manually. Besides that, a physical engine is integrated into this approach to provide physically realistic simulation in real time manner, so that collision free path and related dynamic information could be computed to determine further muscle fatigue and accessibility of a product design

Human Muscle Fatigue Model in Dynamic Motions

Human muscle fatigue is considered to be one of the main reasons for Musculoskeletal Disorder (MSD). Recent models have been introduced to define muscle fatigue for static postures. However, the main drawbacks of these models are that the dynamic effect of the human and the external load are not taken into account. In this paper, each human joint is assumed to be controlled by two muscle groups to generate motions such as push/pull. The joint torques are computed using Lagrange's formulation to evaluate the dynamic factors of the muscle fatigue model. An experiment is defined to validate this assumption and the result for one person confirms its feasibility. The evaluation of this model can predict the fatigue and MSD risk in industry production quickly.

Can virtual reality predict body part discomfort and performance of people in realistic world for assembling tasks?

This paper presents our work on relationship of evaluation results between virtual environment (VE) and realistic environment (RE) for assembling tasks. Evaluation results consist of subjective results (BPD and RPE) and objective results (posture and physical performance). Same tasks were performed with same experimental configurations and evaluation results were measured in RE and VE respectively. Then these evaluation results were compared. Slight difference of posture between VE and RE was found but not great difference of effect on people according to conventional ergonomics posture assessment method. Correlation of BPD and performance results between VE and RE are found by linear regression method. Moreover, results of BPD, physical performance, and RPE in VE are higher than that in RE with significant difference. Furthermore, these results indicates that subjects feel more discomfort and fatigue in VE than RE because of additional effort required in VE.

A novel approach for determining fatigue resistances of different muscle groups in static cases

In ergonomics and biomechanics, muscle fatigue models based on maximum endurance time (MET) models are often used to integrate fatigue effect into ergonomic and biomechanical application. However, due to the empirical principle of those MET models, the disadvantages of this method are: 1) the MET models cannot reveal the muscle physiology background very well; 2) there is no general formation for those MET models to predict MET. In this paper, a theoretical MET model is extended from a simple muscle fatigue model with consideration of the external load and maximum voluntary contraction in passive static exertion cases. The universal availability of the extended MET model is analyzed in comparison to 24 existing empirical MET models. Using mathematical regression method, 21 of the 24 MET models have intraclass correlations over 0.9, which means the extended MET model could replace the existing MET models in a general and computationally efficient way. In addition, an important parameter, fatigability (or fatigue resistance) of different muscle groups, could be calculated via the mathematical regression approach. Its mean value and its standard deviation are useful for predicting MET values of a given population during static operations. The possible reasons influencing the fatigue resistance were classified and discussed, and it is still a very challenging work to find out the quantitative relationship between the fatigue resistance and the influencing factors.

A framework of motion capture system based human behaviours simulation for ergonomic analysis

With the increasing of computer capabilities, Computer aided ergonomics (CAE) offers new possibilities to integrate conventional ergonomic knowledge and to develop new methods into the work design process. As mentioned in [1], different approaches have been developed to enhance the efficiency of the ergonomic evaluation. Ergonomic expert systems, ergonomic oriented information systems, numerical models of human, etc. have been implemented in numerical ergonomic software. Until now, there are ergonomic software tools available, such as Jack, Ergoman, Delmia Human, 3DSSPP, and Santos, etc. [2-4]. The main functions of these tools are posture analysis and posture prediction. In the visualization part, Jack and 3DSSPP produce results to visualize virtual human tasks in 3-dimensional, but without realistic physical properties. Nowadays, with the development of computer technology, the simulation of physical world is paid more attention. Physical engines [5] are used more and more in computer game (CG) field. The advantage of physical engine is the nature physical world environment simulation. The purpose of our research is to use the CG technology to create a virtual environment with physical properties for ergonomic analysis of virtual human.

Using virtual human for an interactive customer-oriented constrained environment design

For industrial product design, it is very important to take into account assembly/disassembly and maintenance operations during the conceptual and prototype design stage. For these operations or other similar operations in a constrained environment, trajectory planning is always a critical and difficult issue for evaluating the design or for the users' convenience. In this paper, a customer-oriented approach is proposed to partially solve ergonomic issues encountered during the design stage of a constrained environment. A single objective optimization based method is taken from the literature to generate the trajectory in a constrained environment automatically. A motion capture based method assists to guide the trajectory planning interactively if a local minimum is encountered within the single objective optimization. At last, a multi-objective evaluation method is proposed to evaluate the operations generated by the algorithm

Integrating digital human modeling into virtual environment for ergonomic oriented design

Virtual human simulation integrated into virtual reality applications is mainly used for virtual representation of the user in virtual environment or for interactions between the user and the virtual avatar for cognitive tasks. In this paper, in order to prevent musculoskeletal disorders, the integration of virtual human simulation and VR application is presented to facilitate physical ergonomic evaluation, especially for physical fatigue evaluation of a given population. Immersive working environments are created to avoid expensive physical mock-up in conventional evaluation methods. Peripheral motion capture systems are used to capture natural movements and then to simulate the physical operations in virtual human simulation. Physical aspects of human's movement are then analyzed to determine the effort level of each key joint using inverse kinematics. The physical fatigue level of each joint is further analyzed by integrating a fatigue and recovery model on the basis of physical task parameters. All the process has been realized based on VRHIT platform and a case study is presented to demonstrate the function of the physical fatigue for a given population and its usefulness for worker selection.

A new muscle fatigue and recovery model and its ergonomics application in human simulation

Although automatic techniques have been employed in manufacturing industries to increase productivity and efficiency, there are still lots of manual handling jobs, especially for assembly and maintenance jobs. In these jobs, musculoskeletal disorders (MSDs) are one of the major health problems due to overload and cumulative physical fatigue. With combination of conventional posture analysis techniques, digital human modelling and simulation (DHM) techniques have been developed and commercialized to evaluate the potential physical exposures. However, those ergonomics analysis tools are mainly based on posture analysis techniques, and until now there is still no fatigue index available in the commercial software to evaluate the physical fatigue easily and quickly. In this paper, a new muscle fatigue and recovery model is proposed and extended to evaluate joint fatigue level in manual handling jobs. A special application case is described and analyzed by digital human simulation technique.