Alert button
Picture for Timothy Bretl

Timothy Bretl

Alert button

The Impact of Time Step Frequency on the Realism of Robotic Manipulation Simulation for Objects of Different Scales

Oct 12, 2023
Minh Q. Ta, Holly Dinkel, Hameed Abdul-Rashid, Yangfei Dai, Jessica Myers, Tan Chen, Junyi Geng, Timothy Bretl

This work evaluates the impact of time step frequency and component scale on robotic manipulation simulation accuracy. Increasing the time step frequency for small-scale objects is shown to improve simulation accuracy. This simulation, demonstrating pre-assembly part picking for two object geometries, serves as a starting point for discussing how to improve Sim2Real transfer in robotic assembly processes.

* 3 pages, 3 figures, Best Poster Finalist at the 2023 Robotics and AI in Future Factory Workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Video presentation [https://www.youtube.com/watch?v=JOXrBpMmI0A]. Robotics and AI in Future Factory workshop [https://sites.google.com/view/robot-ai-future-factory/] 
Viaarxiv icon

Dynamic Manipulation of a Deformable Linear Object: Simulation and Learning

Oct 02, 2023
Qi Jing Chen, Timothy Bretl, Nghia Vuong, Quang-Cuong Pham

Figure 1 for Dynamic Manipulation of a Deformable Linear Object: Simulation and Learning
Figure 2 for Dynamic Manipulation of a Deformable Linear Object: Simulation and Learning
Figure 3 for Dynamic Manipulation of a Deformable Linear Object: Simulation and Learning
Figure 4 for Dynamic Manipulation of a Deformable Linear Object: Simulation and Learning

We show that it is possible to learn an open-loop policy in simulation for the dynamic manipulation of a deformable linear object (DLO) -- e.g., a rope, wire, or cable -- that can be executed by a real robot without additional training. Our method is enabled by integrating an existing state-of-the-art DLO model (Discrete Elastic Rods) with MuJoCo, a robot simulator. We describe how this integration was done, check that validation results produced in simulation match what we expect from analysis of the physics, and apply policy optimization to train an open-loop policy from data collected only in simulation that uses a robot arm to fling a wire precisely between two obstacles. This policy achieves a success rate of 76.7% when executed by a real robot in hardware experiments without additional training on the real task.

* 7 pages, 8 figures 
Viaarxiv icon

The Use of Multi-Scale Fiducial Markers To Aid Takeoff and Landing Navigation by Rotorcraft

Sep 15, 2023
Jongwon Lee, Su Yeon Choi, Timothy Bretl

Figure 1 for The Use of Multi-Scale Fiducial Markers To Aid Takeoff and Landing Navigation by Rotorcraft
Figure 2 for The Use of Multi-Scale Fiducial Markers To Aid Takeoff and Landing Navigation by Rotorcraft
Figure 3 for The Use of Multi-Scale Fiducial Markers To Aid Takeoff and Landing Navigation by Rotorcraft

This paper quantifies the impact of adverse environmental conditions on the detection of fiducial markers (i.e., artificial landmarks) by color cameras mounted on rotorcraft. We restrict our attention to square markers with a black-and-white pattern of grid cells that can be nested to allow detection at multiple scales. These markers have the potential to enhance the reliability of precision takeoff and landing at vertiports by flying vehicles in urban settings. Prior work has shown, in particular, that these markers can be detected with high precision (i.e., few false positives) and high recall (i.e., few false negatives). However, most of this prior work has been based on image sequences collected indoors with hand-held cameras. Our work is based on image sequences collected outdoors with cameras mounted on a quadrotor during semi-autonomous takeoff and landing operations under adverse environmental conditions that include variations in temperature, illumination, wind speed, humidity, visibility, and precipitation. In addition to precision and recall, performance measures include continuity, availability, robustness, resiliency, and coverage volume. We release both our dataset and the code we used for analysis to the public as open source.

* Extended abstract accepted at the 2024 AIAA SciTech 
Viaarxiv icon

Comparative Study of Visual SLAM-Based Mobile Robot Localization Using Fiducial Markers

Sep 08, 2023
Jongwon Lee, Su Yeon Choi, David Hanley, Timothy Bretl

Figure 1 for Comparative Study of Visual SLAM-Based Mobile Robot Localization Using Fiducial Markers
Figure 2 for Comparative Study of Visual SLAM-Based Mobile Robot Localization Using Fiducial Markers
Figure 3 for Comparative Study of Visual SLAM-Based Mobile Robot Localization Using Fiducial Markers

This paper presents a comparative study of three modes for mobile robot localization based on visual SLAM using fiducial markers (i.e., square-shaped artificial landmarks with a black-and-white grid pattern): SLAM, SLAM with a prior map, and localization with a prior map. The reason for comparing the SLAM-based approaches leveraging fiducial markers is because previous work has shown their superior performance over feature-only methods, with less computational burden compared to methods that use both feature and marker detection without compromising the localization performance. The evaluation is conducted using indoor image sequences captured with a hand-held camera containing multiple fiducial markers in the environment. The performance metrics include absolute trajectory error and runtime for the optimization process per frame. In particular, for the last two modes (SLAM and localization with a prior map), we evaluate their performances by perturbing the quality of prior map to study the extent to which each mode is tolerant to such perturbations. Hardware experiments show consistent trajectory error levels across the three modes, with the localization mode exhibiting the shortest runtime among them. Yet, with map perturbations, SLAM with a prior map maintains performance, while localization mode degrades in both aspects.

* IEEE 2023 IROS Workshop "Closing the Loop on Localization". For more information, see https://oravus.github.io/vpr-workshop/index 
Viaarxiv icon

Learning from Integral Losses in Physics Informed Neural Networks

May 27, 2023
Ehsan Saleh, Saba Ghaffari, Timothy Bretl, Luke Olson, Matthew West

Figure 1 for Learning from Integral Losses in Physics Informed Neural Networks
Figure 2 for Learning from Integral Losses in Physics Informed Neural Networks
Figure 3 for Learning from Integral Losses in Physics Informed Neural Networks
Figure 4 for Learning from Integral Losses in Physics Informed Neural Networks

This work proposes a solution for the problem of training physics informed networks under partial integro-differential equations. These equations require infinite or a large number of neural evaluations to construct a single residual for training. As a result, accurate evaluation may be impractical, and we show that naive approximations at replacing these integrals with unbiased estimates lead to biased loss functions and solutions. To overcome this bias, we investigate three types of solutions: the deterministic sampling approach, the double-sampling trick, and the delayed target method. We consider three classes of PDEs for benchmarking; one defining a Poisson problem with singular charges and weak solutions, another involving weak solutions on electro-magnetic fields and a Maxwell equation, and a third one defining a Smoluchowski coagulation problem. Our numerical results confirm the existence of the aforementioned bias in practice, and also show that our proposed delayed target approach can lead to accurate solutions with comparable quality to ones estimated with a large number of samples. Our implementation is open-source and available at https://github.com/ehsansaleh/btspinn.

Viaarxiv icon

Hypergraph-based Multi-Robot Task and Motion Planning

Oct 09, 2022
James Motes, Tan Chen, Timothy Bretl, Marco Morales, Nancy M. Amato

Figure 1 for Hypergraph-based Multi-Robot Task and Motion Planning
Figure 2 for Hypergraph-based Multi-Robot Task and Motion Planning
Figure 3 for Hypergraph-based Multi-Robot Task and Motion Planning
Figure 4 for Hypergraph-based Multi-Robot Task and Motion Planning

We present a multi-robot task and motion planning method that, when applied to the rearrangement of objects by manipulators, produces solution times up to three orders of magnitude faster than existing methods. We achieve this improvement by decomposing the planning space into subspaces for independent manipulators, objects, and manipulators holding objects. We represent this decomposition with a hypergraph where vertices are substates and hyperarcs are transitions between substates. Existing methods use graph-based representations where vertices are full states and edges are transitions between states. Using the hypergraph reduces the size of the planning space-for multi-manipulator object rearrangement, the number of hypergraph vertices scales linearly with the number of either robots or objects, while the number of hyperarcs scales quadratically with the number of robots and linearly with the number of objects. In contrast, the number of vertices and edges in graph-based representations scale exponentially in the number of robots and objects. Additionally, the hypergraph provides a structure to reason over varying levels of (de)coupled spaces and transitions between them enabling a hybrid search of the planning space. We show that similar gains can be achieved for other multi-robot task and motion planning problems.

* This work has been submitted for review 
Viaarxiv icon

Truly Deterministic Policy Optimization

May 30, 2022
Ehsan Saleh, Saba Ghaffari, Timothy Bretl, Matthew West

Figure 1 for Truly Deterministic Policy Optimization
Figure 2 for Truly Deterministic Policy Optimization
Figure 3 for Truly Deterministic Policy Optimization
Figure 4 for Truly Deterministic Policy Optimization

In this paper, we present a policy gradient method that avoids exploratory noise injection and performs policy search over the deterministic landscape. By avoiding noise injection all sources of estimation variance can be eliminated in systems with deterministic dynamics (up to the initial state distribution). Since deterministic policy regularization is impossible using traditional non-metric measures such as the KL divergence, we derive a Wasserstein-based quadratic model for our purposes. We state conditions on the system model under which it is possible to establish a monotonic policy improvement guarantee, propose a surrogate function for policy gradient estimation, and show that it is possible to compute exact advantage estimates if both the state transition model and the policy are deterministic. Finally, we describe two novel robotic control environments -- one with non-local rewards in the frequency domain and the other with a long horizon (8000 time-steps) -- for which our policy gradient method (TDPO) significantly outperforms existing methods (PPO, TRPO, DDPG, and TD3). Our implementation with all the experimental settings is available at https://github.com/ehsansaleh/code_tdpo

Viaarxiv icon

Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories

May 29, 2022
Jongwon Lee, David Hanley, Timothy Bretl

Figure 1 for Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories
Figure 2 for Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories
Figure 3 for Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories
Figure 4 for Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories

We present a method of extrinsic calibration for a system of multiple inertial measurement units (IMUs) that estimates the relative pose of each IMU on a rigid body using only measurements from the IMUs themselves, without the need to prescribe the trajectory. Our method is based on solving a nonlinear least-squares problem that penalizes inconsistency between measurements from pairs of IMUs. We validate our method with experiments both in simulation and in hardware. In particular, we show that it meets or exceeds the performance -- in terms of error, success rate, and computation time -- of an existing, state-of-the-art method that does not rely only on IMU measurements and instead requires the use of a camera and a fiducial marker. We also show that the performance of our method is largely insensitive to the choice of trajectory along which IMU measurements are collected.

* in IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2055-2062, April 2022  
* RA-L with ICRA 2022 
Viaarxiv icon

Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration

May 28, 2022
Tan Chen, Zhe Huang, James Motes, Junyi Geng, Quang Minh Ta, Holly Dinkel, Hameed Abdul-Rashid, Jessica Myers, Ye-Ji Mun, Wei-che Lin, Yuan-yung Huang, Sizhe Liu, Marco Morales, Nancy M. Amato, Katherine Driggs-Campbell, Timothy Bretl

Figure 1 for Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration
Figure 2 for Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration
Figure 3 for Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration
Figure 4 for Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration

Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the project is to develop a flexible, safe, and productive manufacturing cell for sub-centimeter precision assembly. Solving this problem in a high-mix, low-volume production line motivates multiple research thrusts in robotics. This work identifies new directions in collaborative robotics for industrial applications and offers insight toward strengthening collaborations between institutions in academia and industry on the development of new technologies.

* Spotlight presentation at ICRA 2022 Workshop on Collaborative Robots and the Work of the Future (ICRA 2022 CoR-WotF); see the spotlight presentation at https://sites.google.com/view/icra22ws-cor-wotf/accepted-papers?authuser=0 
Viaarxiv icon

iCaps: Iterative Category-level Object Pose and Shape Estimation

Dec 31, 2021
Xinke Deng, Junyi Geng, Timothy Bretl, Yu Xiang, Dieter Fox

Figure 1 for iCaps: Iterative Category-level Object Pose and Shape Estimation
Figure 2 for iCaps: Iterative Category-level Object Pose and Shape Estimation
Figure 3 for iCaps: Iterative Category-level Object Pose and Shape Estimation
Figure 4 for iCaps: Iterative Category-level Object Pose and Shape Estimation

This paper proposes a category-level 6D object pose and shape estimation approach iCaps, which allows tracking 6D poses of unseen objects in a category and estimating their 3D shapes. We develop a category-level auto-encoder network using depth images as input, where feature embeddings from the auto-encoder encode poses of objects in a category. The auto-encoder can be used in a particle filter framework to estimate and track 6D poses of objects in a category. By exploiting an implicit shape representation based on signed distance functions, we build a LatentNet to estimate a latent representation of the 3D shape given the estimated pose of an object. Then the estimated pose and shape can be used to update each other in an iterative way. Our category-level 6D object pose and shape estimation pipeline only requires 2D detection and segmentation for initialization. We evaluate our approach on a publicly available dataset and demonstrate its effectiveness. In particular, our method achieves comparably high accuracy on shape estimation.

Viaarxiv icon