2.1. Self-driving Vehicles as cyber-physical systems
intelligent cyber-physical systems (ICPSs) connect the physical and digital worlds by connecting sensors, actuators, and intelligent computing unit. These systems use advanced ML algorithms and approaches to interact with their surroundings, collect raw data, and process data in a simultaneously manner. As a result, ICPSs may use the data they receive to make decisions and take action [10]. Autonomous driving techniques have recently drawn a lot of interest from the academic and automotive communities due to significant advancements in computing technologies such as sensors, computer vision, machine learning, and hardware acceleration, as well as the wide adoption of communication mechanisms such as dedicated short-range communications (DSRCs), cellular vehicle-to-everything (C-V2X), and 5G [11]. When it comes to mobility, autonomous vehicles are a game-changer because they offer a convenient and safe driving experience without requiring human intervention. The perception sensor instruments (such as cameras and radars), intelligent processors, AI-based methodologies and approaches, and communication with other internet of things (IoT) devices have all improved as a result of these smart cars. The self-driving car is a type of cyber-physical system (CPS) as a result, and it travels on a very interactive and dynamic road [12]. One of the most popular CPS applications is autonomous vehicles, which can make transportation safer, more effective, and environmentally friendly. To enable autonomous vehicles in the transportation CPS, ICPSs are essential. A vehicle's ability to sense its surroundings and drive itself is referred to as self-driving technology. The term "intelligent vehicle" refers to a vehicle equipped with AD technology that can activate fully or partially automated driving functions. Eight categories are used to categorize autonomous driving operations according to how complex they are. The perception module helps intelligent vehicles collect environmental data and retrieve features from the outside world. The planning module aids intelligent vehicles in arriving at more palatable decisions. The self-driving vehicles can prevent barriers and constantly improve their methods of driving to assure passenger safety and comfort. The computing system architecture significantly contributes to ensuring communication effectiveness, energy efficiency, safety, and security in AD and IVs. The communication module is essential for communicating with the environment, which includes other cars, pedestrians, and intelligent transportation networks. The HD map, like the guidance programme for human drivers, serves as a guide for IVs and delivers enhanced scene and position information. The testing module is a necessary stage in the procedure before IVs can be used on public roadways. Human behaviors in IVs are an important part of current AD development since understanding human behaviors can aid in the creation of safe driving methods for autos.
Six levels of automation are defined by the Society of Automotive Engineers (SAE) which are from L0 to L6, no automation to fully automation. In level 0, human driver drives the vehicle without special automation. In Level 1, ADAS aids the human driver in the limited situations. In level 2, the steering and acceleration control of vehicle is performed by driving assistance in specific scenarios such as highway driving. In level 3, the entire functions are performed by the car, but there is yet human intervention in this level of automation. In level 4, we have fully autonomous system in which is responsible of all driving tasks. Level 5 is a level that the vehicle can drive like a human and handle all situations and driving conditions in a human manner [13–15].
As shown in Fig. 1, Perception, planning, control, system design, communication, HD map, testing, and human behaviors in IVs are the eight groups into which AD tasks are divided [16]. For IVs to guarantee safety, security, energy efficiency, and communication efficiency, the computing system is essential. A typical IV that has a wide variety of onboard sensors, such as Lidar, cameras, radar, a communication module, a global positioning system (GNSS), and others, may produce enormous amounts of data each minute. A computer system that can process the data in real time while maintaining safe AD must be developed to analyze and combine heterogeneous information effectively [13]. According to [11] the modular and end-to-end architectures are the two main types of self-driving vehicles model. modular approach is divided into the major modules (as illustrated in Fig. 2). end-to-end architecture is highly impacted by modern AI, it fundamentally depends on learning-based algorithms to map sensor inputs into control outputs [17].
Table 1 depicts the previous architectures in which based on CPSs and were designed for autonomous vehicles, vehicular networks, and security issues of cars.
Table 1
Previous architectures in the field of CPS + AV
Rows | source | architecture's name | purpose of architecture | architecture's main components |
1 | [12] | cyber security framework | address evolving cyber-physical threats, especially for autonomous driving applications | [analysis as CPS]: define cyber-physical system or subsystem of AV; identify (unacceptable losses, hazards/constraints); model functional control architecture, identify unsecured control actions; generate casual scenarios for unsecure control, transform casual scenarios to attack class with STRIDE; [Analysis in cybersecurity framework] |
2 | [13] | computing architecture for intelligent vehicles (IVs) | ensure efficiency in communication, energy, and safety | computing hardware; Operating System (OS); sensor systems [smart and non-smart sensors]; Fusion; Applications Communication layer; Security; Storage Management; Power and Thermal Management; |
3 | [20] | strategies for model-driven control-wise communication | enable adaptive cruise control in a cooperative manner | Networking module [C-V2X, preceding vehicles information, stochastic model regression, stochastic model estimator]; control module [vehicle, optimal predictive values, model predictive control (vehicle model, optimizer (costs and constraints] |
4 | [21] | Cyber-physical framework for multi-lane freeways' best CAV coordination | Group connected and automated vehicles together into smaller groups, then sequentially optimize their trajectories. | physical network [vehicles]; cyber network ([vehicle information, grouping, optimization, trajectory planning] |
5 | [22] | deep hybrid anomaly detection architecture | assures vehicle's onboard sensor systems security over a 6G network | Physical world over 6G network [AVs, the link between the AVs, authorized data, malicious data]; Cyber world [datasets, road side unit (RSU), multi-agent reinforcement learning, MaxEnt inverse reinforcement learning, reward function fine tuning] |
6 | [23] | PASTA framework | For a specific vehicle, PASTA co-optimizes the positions and orientations of sensors, deep learning-based object detectors, and sensor fusion algorithms. | an object detector algorithm, a sensor fusion algorithm, and a sensor configuration (with the deployment position and orientation of each sensor chosen for the vehicle). |
7 | [24] | path tracking control system based on CPS | increase the AVs' resilience to cyberattacks | Physical world [actuator (steering; driving, braking), vehicle model, sensor system (camera, GPS, LiDAR, IMU)], In-vehicle communication network; Cyber space [Path tracking control strategy based on CPS], Communication Network (Bluetooth, GPS, LTE, 4G/5G, WiFi, WiMAX, etc.) |
8 | [25] | context-aware navigation protocol (CNP) | to increase driving safety in vehicular cyber-physical systems (VCPS) | data sensing [locally sensed and remotely sensed]; Risk assessment [obstacle detection, collision risk assessment, collision avoidance modalities, maneuver control decision]; longitudinal maneuver [controls]; lateral maneuver [compute reference path, controls]; kinematic tracking [MCTA path optimization, posteriori kinematic planning]; cooperative control [path tracking, maneuver advertisement] |
9 | [26] | data-driven based distributed CPS (DDPS) | safe driving and cruise control | data-driven layer status information layer cloud service layer |
10 | [27] | Platform-Based design optimization of the electric vehicle | optimally adapt the automatic control of an intelligent electric vehicle to driving styles | high-level requirements [Ride comfort, Dynamic performance, and energy efficiency]; constraints [performance constraints and safety constraints]; design platform [library of components (models of vehicle, electric powertrain, brakes, and driver-style-based controller) |
11 | [28] | agent-based model for air traffic management | developing software (used the UML diagrams) | |
12 | [29] | smart camera architecture | intrusion detection in wireless visual sensor network | layer of software as a service (SAAS), layer of platform as a service (PAAS), layer of infrastructure as a service (IAAS), management layer, operators and cloud computation performance |
2.2. Perception system in self-driving vehicles
Perception system is the main part of a self-driving vehicle in which collects different sensor data and transform it to meaningful information to perform high-level tasks such as navigation and motion planning [30]. Perception system maps sensor data into world semantic model and consists of different subsystems like object detection, semantic segmentation, localization, and object tracking [31–33]. Different sensors are used in perception system to receive data from the environment. Visual-based perception detects visual objects and range-based perception with lidars and radars detects static and dynamic objects [34–36]. It is recommended that perception system must use multi-sensor system to precisely understand its surrounding in which it is vital for autonomous vehicle and Assistant Driving Autopilot Systems (ADAS) [37, 38]. Object detection is the most important task in the perception system. It detects surrounding objects by sensing data [39]. Semantic segmentation is another fundamental function for autonomous navigation of self-driving vehicles. It assigns a label to each part of an image or point cloud [40]. Object tracking is a task of tracking objects in the consecutive frames of images. It tracks dynamic motions of objects in different time intervals [41, 42]. Localization system detects the position of ego-vehicle. It depends on different locational sensors [43].
Using machine learning algorithms, received data from various sensors can be processed into semantic information in which other systems of autonomous vehicles take advantage of this information to perform reliably and safety [44].
2-2-1. Object Detection
Object detection is one of core task of perception system [45]. It detects static and dynamic surrounding objects by gathering data from sensors and processing that to meaningful information [46]. Object detection module detects the class of objects, position and orientation of objects [39]. The field of object detection tasks is divided in to 2D and 3D object detection [47]. 2D object detection architectures detect the objects in the 2D plane, the input data of DL algorithms are monocular images [48]. 3D object detection models detect the objects in 3D space, the input data are point clouds, monocular, stereo, and RGBD images [49]. Point cloud data is the most popular input for 3D object detection architectures. LiDAR generates point cloud data [50–52]. DNN algorithms are the baseline of many object detection architectures, especially CNN is the most applied algorithm in the object detection field [53].
2-2-2. Semantic Segmentation
Semantic segmentation is another fundamental task of perception system. It assigns a label into each pixel of an image or each point of a point cloud [54–56]. It is a technique of scene understanding. scene understanding for autonomous navigation in complex dynamic driving scenarios is crucial [57]. Semantic segmentation predicts the class, shape, and position of each element. Some of its applications are in medical image analysis, accurate road boundaries and obstacle detection [58]. By shifting from computer vision to DL based algorithms, semantic segmentation has become a trend in last decade. Semantic segmentation like object detection is divided in to 2D semantic segmentation and 3D semantic segmentation [59, 60]. CNN is the most efficiency algorithm for semantic segmentation task [61, 62].
2-2-3. Object Tracking
Multi-object tracking is a crucial module in the perception system to provide a robust understanding of the environment and enable cars to safely navigate in the driving scenes with the surrounding cars [63]. It produces a reliable representation from the world for planning system. Its goal is estimating orientation, position and size of all existing objects in all times [64]. The key challenges for multi-object tracking is three-fold: 1) ambiguity in the number of objects, 2) ambiguity in time and location in which objects appears and disappears, 3) ambiguity in the situation of objects [63]. Tracking based on detection is the most popular tracking paradigm in which firstly detects object in each frame, and then they are associated with in the frames [65].
2-2-4. Localization
Localization is an essential task in the perception system of self-driving vehicles [66]. Safely navigation needs identifying ego-vehicle's location, position and orientation in the driving scenes [67]. Different locational, visual, and inertial sensors use for localization and mapping tasks such as GPS, GNSS, LiDAR, IMU, RTK, etc. [68, 69]. Localization based on onboard sensors suffer from some main challenges such as weather condition such as fogging, snow, and dark, and also intersection driving scenes. It is suggested that V2V communications can improve the performance of localization system [70].
2.3. Unified Modeling Language
Object-oriented analysis and design boosts software productivity by improving quality, reducing development time, and reducing complexity. UML, an industry-standard notation, supports decomposition, abstraction, modularity, and system reusability across various domains. Although similar to Java and C++, UML is a modeling language. [71]. While programming languages are used during the execution phase, modeling languages can be used during the design phase. Software users and engineers must agree on the system requirements before the system's implementation. UML makes it possible for stakeholders to connect and communicate with one another, to prevent the development of systems that fall short of user requirements [72]. Software engineers must develop a system model after the requirements are specified. This model acts as a conduit for data transfer between software developers and system users [73].
UML is a powerful toolkit for analyzing system architectures and improving design quality. It utilizes object-oriented design and relational database models to describe a system using graphical notations [74]. UML is used throughout the software development process, including definition, analysis, design, production, testing, and maintenance [75]. It incorporates new technologies and methodologies to support object-oriented analysis and the entire process. UML is versatile and can be used at all stages of software development, allowing developers to focus on structures and product models rather than programming languages and algorithms. It can be used to decompose software systems layer by layer, enhancing embedded architecture in development plans. [76].
UML is a standardized language for software design and architecture, improving efficiency, cost-effectiveness, and accuracy through graphic representations and transparent design sharing. [77].
UML is a modeling language used for specifying, visualizing, constructing, and documenting software system artifacts. It helps users understand and communicate system requirements, construct new systems, and document system evolution. UML is versatile and can be used in any development environment, process model, life cycle stage, or application domain. It has three perspectives: functional, structural, and behavioral, with nine diagrams used in each [78].
UML is a visual modeling technique used for object-oriented system design. It involves requirements specification, use case modeling, and defining behavior and structure. UML can model static and dynamic systems, with the object-oriented software development process divided into four stages. [75]:
Use Case Modeling involves describing constraints, using scenarios to elicit user requirements, creating executable objects, and examining their behavior to create a system's use case model based on analysis findings.
Object Modeling: Object modeling simulates the internal organization and static viewpoint of a system, analyzing entities and relationships to determine functional constraints. Object diagrams and class diagrams are common logical building blocks for static system models.
Dynamic Modeling: Use cases are implemented using the Unified Modeling Language (UML) to analyze temporal states in system objects. Concurrent perspectives like sequence, collaboration, state, and activity diagrams are created for dynamic modeling, with only one or two diagrams used to describe issues.
Implementation and testing: A model environment is necessary to convert a unified modeling language into code, focusing on process analysis, consistency, and structure assurance. It includes code creation, automated documentation, dynamic implementation analysis, and reverse engineering. During the implementation phase, the model is mapped to programming languages and code-generation tools, with manual programming required. The code is tested in a simulation environment, and reports are generated to improve the model. Repeated code generation is necessary until an accurate and correct code is produced.[75].