On Applying Machine Learning/Object Detection Models for Analysing Digitally Captured Physical Prototypes from Engineering Design Projects

While computer vision has received increasing attention in computer science over the last decade, there are few efforts in applying this to leverage engineering design research. Existing datasets and technologies allow researchers to capture and access more observations and video files, hence analysis is becoming a limiting factor. Therefore, this paper is investigating the application of machine learning, namely object detection methods to aid in the analysis of physical porotypes. With access to a large dataset of digitally captured physical prototypes from early-stage development projects (5950 images from 850 prototypes), the authors investigate applications that can be used for analysing this dataset. The authors retrained two pre-trained object detection models from two known frameworks, the TensorFlow Object Detection API and Darknet, using custom image sets of images of physical prototypes. As a result, a proof-of-concept of four trained models are presented; two models for detecting samples of wood-based sheet materials and two models for detecting samples containing microcontrollers. All models are evaluated using standard metrics for object detection model performance and the applicability of using object detection models in engineering design research is discussed. Results indicate that the models can successfully classify the type of material and type of pre-made component, respectively. However, more work is needed to fully integrate object detection models in the engineering design analysis workflow. The authors also extrapolate that the use of object detection for analysing images of physical prototypes will substantially reduce the effort required for analysing large datasets in engineering design research.

Digitally Capturing Physical Prototypes During Early-Stage Product Development Projects for Analysis

Aiming to help researchers capture early-stage Product Development (PD) activity, this article presents a new method for digitally capturing prototypes. The motivation for this work is to understand prototyping in the early stages of PD projects, and this article investigates if and how digital capture of physical prototypes can be used for this purpose. In PD case studies, such early-stage prototypes are usually rough and of low-fidelity and are thus often discarded or substantially modified through the projects. Hence, retrospective access to prototypes is a challenge when trying to gather accurate empirical data. To capture the prototypes developed through the early stages of a project, a new method has been developed for digitally capturing physical prototypes through multi-view images, along with metadata describing by who, when and where the prototypes were captured. In this article, one project is shown in detail to demonstrate how this capturing system can gather empirical data for enriching PD case studies on early-stage projects that focus on prototyping for concept generation. The first approach is to use the multi-view images for a qualitative assessment of the projects, which can provide new insights and understanding on various aspects like design decisions, trade-offs and specifications. The second approach is to analyse the metadata provided by the system to give understanding into prototyping patterns in the projects. The analysis of metadata provides insight into prototyping progression, including the frequency of prototyping, which days the project participants are most active, and how the prototyping changes over time.