This review paper focuses on different aspects of perceived safety for a number of autonomous physical systems. This is a major aspect of robotics research, as more and more applications allow human and autonomous systems to share their space, with crucial implications both on safety and on its perception. The alternative terms used to express related concepts (e.g., psychological safety, trust, comfort, stress, fear, and anxiety) are listed and explained. Then, the available methods to assess perceived safety (i.e., questionnaires, physiological measurements, behavioral assessment, and direct input devices) are described. Six categories of autonomous systems are considered (industrial manipulators, mobile robots, mobile manipulators, humanoid robots, drones, and autonomous vehicles), providing an overview of the main themes related to perceived safety in the specific domain, a description of selected works, and an analysis of how motion and characteristics of the system influence the perception of safety. The survey also discusses experimental duration and location of the reviewed papers as well as identified trends over time.