🛠️Deployment Guide for 3D Vision-Based Obstacle Avoidance Solution

1. Application Overview

2. Camera installation and selection

2.1 Latent Mobile Robot

2.2 Forklift

3. Camera connection settings

1. Before connecting the camera, the following settings need to be operated. The camera's factory IP is 192.268.100.82. Before connecting, the local network port IP needs to be set to the same network segment, as shown in Figure 3.1.

1. The firewall is set to the off state, as shown in Figure 3.2.

1. If you need to modify the camera IP, you can open the " LxCameraViewer " PC software, click " Other Tools ", Network Related → IP Configuration Tool → Click IP Address to modify, press Enter, and then click Confirm Modification. After the modification is completed, the camera will automatically restart and reconnect.

1. You can observe whether the IP has been successfully modified through the status bar in the lower left corner. When different network segments are used, you need to modify the local IP before reconnecting to the camera.

1. Open the pre-installed upper computer software LxCameraViewer, and the upper computer software automatically retrieves the camera under the current network segment. Clicking to open the camera will automatically open the depth map and point cloud map, as shown in Figure 3.3.

1. After opening the camera, expand the 3d settings and filter settings for camera parameter adjustment, the recommended configuration parameters for obstacle avoidance applications, are high integration 650, low integration 200, filter settings low signal threshold recommended to be set to 10-20, installation and parameter configuration of different models, It is recommended to communicate with the technical support personnel of MRDVS.

4. Obstacle avoidance parameter configuration

1. After completing the above camera parameter settings, click the "Apply Algorithm" button. The algorithm settings will pop up in the right status bar. Set it to always-on mode or close it when disconnected according to the application settings. For the S2 camera application algorithm, select "Obstacle Avoidance Algorithm". For other cameras, select "Obstacle Avoidance Algorithm 2.0", and the algorithm version number will be displayed below. The S2 camera does not recommend using a fan-shaped obstacle avoidance area type, which will cause deviation in obstacle avoidance results.

1. The rectangular obstacle avoidance area and the fan-shaped obstacle avoidance area are shown in Figure 4.1.

1. As shown in Figure 4.2, the obstacle avoidance parameters are divided into the configuration parameters of the obstacle avoidance area outside the camera installation. After clicking the Settings button to complete the configuration of the camera's external parameters, the other four configuration areas are automatically updated to the current external parameters.

• Among them, the S2/S2 Max 2.0 camera can switch four groups of obstacle avoidance area configurations through IO: 0, 1, 2, and 3.

• Up to 20 sets of obstacle avoidance areas can be switched through SDK/UDP/RS485, which can be switched according to the running scenario of the robot. For example, when the robot moves to a narrow channel, it can be switched to a narrow channel configuration, and when the robot moves to a wide channel, it can be switched to a wide channel configuration.

5. External parameter settings

5.1 Parameter description

5.2 Parameter settings

6. Obstacle avoidance area settings

6.1 Regional parameter description

6.2 Obstacle avoidance area settings

1. The current obstacle avoidance application has 20 built-in obstacle avoidance areas. After the user completes the camera parameter configuration, the obstacle avoidance area can be configured. When the AMR moves to different obstacle avoidance ranges, the configuration area can be switched to control the obstacle avoidance range (IO: three groups of obstacle avoidance areas can be switched, UDP, 485, API, and CAN can be switched to 20 groups).

• Warning distance (X far distance): Indicates that the camera warning (deceleration) obstacle avoidance range is 2300, that is, the warning distance is 2300-camera position 430 = 1870mm, as shown in Figure 6.1.

• Alarm Distance (X medium distance): Indicates the camera alarm (stopping) obstacle avoidance range of 1200, that is, the alarm distance of 1200-camera position 430 = 770mm, as shown in Figure 6.1.

• Shielding distance (X short distance): Indicates the camera shielding range, that is, shielding distance 450-camera position 430 = 20mm, as shown in Figure 6.1.

• Obstacle avoidance range left (y left): Indicates that the camera detects obstacles in the range of 500mm to the left of the obstacle avoidance camera, as shown in Figure 6.2.

• Obstacle avoidance range left (y right): Indicates that the camera detects obstacles within a 500mm range to the right of the obstacle avoidance camera, as shown in Figure 6.2.

• Obstacle avoidance height (bottom Z): Indicates the detection of obstacles within a range of 10mm below the ground, as shown in Figure 6.3.

• Obstacle avoidance height (upper Z): Indicates that the camera detects obstacles within a range of 500mm above the ground, as shown in Figure 6.3.

1. Obstacle avoidance output:

• 0: Indicates normal without obstacles

• 1: Indicates that obstacles have been detected in the warning area (within the yellow box)

• 2: Indicates that an obstacle has been detected in the alarm area (within the red box)

1. After configuring the parameters according to the above process using the host computer, you can go to the software Installation path\ Tools\ params\ current camera ID\ avoidation.json Backup the avoidation.json file and then quickly synchronize the avoidance parameters for different cameras: After turning on the camera, execute "Load configuration from file", select the backed-up avoidation.json file and wait for the setting to complete.

7. Communication methods

1. Before understanding or using various communication methods, we must use the upper computer to configure the content of obstacle avoidance parameters one by one, in order to use non-API communication methods to switch different obstacle avoidance parameters.

1. All obstacle avoidance parameters of S2 Max can be set through the host computer and API provided by us, but for other communication methods (IO/UDP/CAN/485), it is limited to modifying the obstacle avoidance parameter index. The obstacle avoidance parameters of S2 can be set through the host computer provided by us, but for all communication methods, it is limited to modifying the obstacle avoidance parameter index.

1. Regarding the modification of obstacle avoidance parameters, the camera strictly follows the logic of priority internally.

2. The biggest difference between temporary parameters and default parameters is whether to save them when power is turned off. Which parameter to use depends on the last communication method used.

3. When using IO communication (IO is not suspended), using the communication method with priority 2 cannot modify the current parameters, but can modify the default parameters and temporary parameters. When IO communication is disconnected (IO is suspended), the specific use of default parameters or temporary parameters depends on the last setting method. When using 485 or UDP settings for the last time, temporary parameters are used. When using API calls or upper computer settings for the last time, default parameters are used. The logic is as follows

1. For the S2 camera, it takes 200~300ms to switch parameters until loading is completed. Therefore, S2 filters the switching parameter operation, and the last setting within 250ms will be ignored. It is not recommended to switch parameters too frequently for both S2 and S2Max, as it will increase the camera's resource usage and reduce the image frame rate. Because the refresh rate of obstacle avoidance results is related to the frame rate, it will also cause a decrease in the refresh rate of obstacle avoidance results. The refresh rate of obstacle avoidance results is equal to the image frame rate.

7.1 API calling method

1. The API calling method can support C++, C #, JAVA, ROS1, ROS2, and other environments such as Windows, Linux, and Arm. After installing the upper computer software, the SDK file and sample program code of the Windows environment are all under the installation path (such as D:\ Program Files\ Lanxin-MRDVS). Linux environment SDK can be provided by contacting MRDVS-related technical support or downloaded from the official website. It is not recommended to use this method to switch parameters for S2 cameras. It is recommended to use UDP or IO to switch parameters.
1. Document folder for the SDK and PC use documentation.
2. The FirmWare folder is the directory where the camera firmware packages are stored.
3. Sample folder for example code source code storage directory, you can choose according to the Development Environment.
4. The SDK folder is the directory where the SDK library files are stored, and environment variables can be configured according to the Development Environment.
5. Obstacle avoidance applications can parameter Sampl directory C/application_obstacled obstacle avoidance API source code program.

7.2 485 Communication Methods

1. 485 communication only supports the M4 Pro camera, using Modbus RTU transmission protocol. The following is the definition of the camera wiring harness.

1. The debugging method for 485 is as follows:

• When connecting via RS485, the baud rate is set to 9600 in the debugging assistant.

• Use the host computer to view the current Modbus address, read the two register addresses starting from address 108 (0x6C) in the input register (0x04), and the feedback register value is the current obstacle avoidance result.

• If you need to modify the obstacle avoidance parameters, you can write the 106 (0x6A) address of a single save register (0x06).

7.3 UDP communication method

1. When connecting via UDP, use the network debugging assistant to test the communication connection with the camera, and select hexadecimal to receive and send data. For the first communication, it is recommended to use instruction number 4. After establishing the UDP connection, the obstacle avoidance result will be automatically sent, and the sending frequency is related to the current image frame rate. After modifying the parameters, you can confirm whether the modification is successful in the message information sent by the camera to the body. XX is the current parameter in message number 1.

2. The parameters set by DP remain in effect until the camera restarts, unless modified by power outage or other communication methods. If you want them to remain in effect after a power outage, you can use message number 10 to save them as default parameters. The function of saving as default parameters is UDP communication-specific function, CAN, 485.

3. Set the local host address and local host port, and click to open; the remote host is the camera, fill in the camera IP port: 192.168.100.12: 6688, if the camera IP has changed, it is: camera IP: 6688; take the current as parameter 19 and modify it to parameter 0 as an example, click to send data number 5, and the flag of successful setting is that the 5th byte is changed from 0x13 (parameter 19) to 0x00 (parameter 0), as shown in the figure below.

7.4 CAN communication method

1. The communication protocol adopts CanOpen, which is an application layer protocol running on the standard CAN bus, and its communication mode is also the “master-slave” mode commonly used in industrial communication protocols, that is to say, there will be a master + multiple slaves in the network, and the slaves will not communicate with each other directly, and all the communication is between the master and the slaves. All communication is between the master and the slaves. The master is also called the “client” and the slaves are also called “servers”.

2. The underlying communication method uses the CAN standard frame format, which means the CAN ID is 11 bits (0x000~ 7FF) and the data is 8 bytes.

3. Regarding the usage convention of CAN ID: divide the 11-bit ID into a 4-bit function code and a 7-bit node ID, and CAN ID is also known as COB ID.

1. Can baud rate: default is 250K, can be configured according to the application.

2. Node number: Default 0x212, can be configured according to the application.

3. SDO's communication message is a basic protocol format in CanOpen. The 8-byte data in the CAN message is defined by the Communication Protocol as follows:

1. Message format sent by the ontology to the device.

• Reference message: 40 00 00 00 0B 00 00 00 00, 0B: i.e. MODECTRL, indicates that the obstacle avoidance parameter is set to area 11.

1. The format of the message sent by the device to the principal.

• Reference character: 43 00 00 00 0B 00 00 02, 0B: i.e. MODECTRL, indicating that the current obstacle avoidance parameter is region 11; 02: i.e. RESULT, indicating the current obstacle avoidance result.

1. Note: Setting the obstacle avoidance parameters does not take effect immediately, as shown below.

7.5 IO communication method

7.5.1 Corresponding Table of IO Input States and Obstacle Avoidance Parameters

• Note: The input IO is low when it is suspended, and the recommended voltage for a high level is 5~25V. - Note: The logic of using the default or temporary parameters refers to the preamble of Chapter 7 (Communication Methods), in short, when the camera is powered on and no other (except API) communication methods are used to configure the obstacle avoidance parameters, default parameters are used, and vice versa. In short, the default parameters are used when the camera is powered up and no other communication method (except API) is used to configure the obstacle avoidance parameters, and vice versa.

7.5.2Corresponding Table of IO Output States and Obstacle Avoidance Parameters

• Note: When the obstacle avoidance is turned on but the output IOs are all grounded, please make sure that the current obstacle avoidance parameters have been set by the host computer.

7.5.3S2 Max2.0 camera I/O wiring method

7.5.4S2 Camera I/O Wiring Method

1. Input: When the external input is high group state, low level, the LED in the optocoupler does not emit light, the photosensitive transistor in the optocoupler is not the same, the camera internal MCU receive level is high; when the external input is high, the LED in the optocoupler emits light, the photosensitive transistor in the optocoupler is conductive, and the camera insider MCU receive level is low.

2. Output: When the S2 camera MCU outputs a high level, the optocoupler LED does not emit light, the optocoupler phototransistor does not conduct, and OUT_* and OUT_G are open; when the S2 camera MCU outputs a low level, the optocoupler LED emits light, the optocoupler phototransistor conducts, and OUT_* and OUT_G are short-circuited, i.e., grounded (because the IN_G and OUT_G are connected). connected to the ground plane).

7.5.5 S2/S2 Max2.0 camera and I/O wiring method of industrial control machine

7.5.5.1Input IN_* wiring method

1. The following diagram shows how to wire the PNP-type output of the industrial controller to the input of the S2 camera (hereinafter abbreviated as IN_*, * stands for 1 and 2):

1. The following diagram shows how to wire the NPN-type output of the industrial controller to the input of the S2 camera:

1. Due to the different manufacturers of industrial control machines output circuit design is different, it needs to be based on the actual situation to determine whether the need to add pull resistance and the size of the resistance value.

2. It needs to be tested as follows:

• Use a multimeter to measure the impedance of the ICPC output to 24V, if <50kΩ, no pull-up resistor is needed; if ≥50kΩ, the additional pull-up resistor is needed, and it is recommended to prioritize the use of: plug-in metal film resistor (resistance value of 10kΩ, power ≥1W). Other resistors can be used for temporary testing, but long-term use affects reliability, and the resistance value can not be less than 3k ohms, not directly connected to the 24V.

• After the connection is confirmed correct, the output of the industrial computer will be pulled up and then use a multimeter to measure the voltage of the output to the ground, if the voltage is ≤ 2V, then disconnect the camera circuit connection and then check whether the industrial computer can be pulled up to the high level. If the voltage > 2V, then pull down the output to check whether there is a change in the S2 camera input, if there is a change, it means that it can be used normally; if there is no change, then pull down the output of the industrial computer and then measure the voltage of the output to the ground, if the voltage is greater than 1V, then it means that the pull-up strength is larger than that of the pull-down, and it is difficult to pull down, so it is necessary to increase the value of the pull-up resistor, and then measure the pull-down after the change in the input of the S2 camera; if the voltage is less than 1V, then it is necessary to increase the pull-up resistance. If the voltage is less than 1V, then disconnect the ICPC cable, connect the camera IN_* to 24V and GND through the resistor, and check whether there is any change in the camera input, and if there is any change, then check whether there is a correct connection between the ICPC and the camera; if there is no change, then contact the technical support to troubleshoot the problem.

1. The logic tree is shown below:

7.5.5.2 Output Out_* wiring method

1. As the S2 camera output for the ground/open signal, the S2 camera output and industrial computer input connection method need not distinguish between the industrial computer PNP, and NPN-type inputs, the main distinction for the normal high level or low level, that is, the industrial computer internal pull-up or not, and then determine whether the wiring needs to be added to the pull-up resistor.

2. The following figure shows the wiring method between the input of the industrial controller and the output of the S2 camera (hereinafter abbreviated as OUT_*, * stands for 1 and 2):

1. Whether to connect the pull-up resistor R judgment criteria for the industrial control machine power, the input side of the suspension (not connected to the camera), direct measurement of the input side of the voltage to ground, if the voltage to ground ≥ 5V, then do not need to add the pull-up resistor R; if the voltage to ground ≈ 0V, then you need to add the pull-up resistor R. Recommended priority to use: plug-in metal film resistor (resistance value of 10kΩ, power ≥ 1W). Other resistors can be used for temporary testing but long-term use affects the reliability, and the resistance value should not be less than 3k ohms, not directly connected to 24V.

8. Handling of common problems

8.1 black object

8.2 low lying object

• The point cloud map of the upper computer shows that the obstacles are not selected, you need to confirm that the height of obstacle avoidance (below) is a positive value, if it is positive, it is recommended to temporarily set it to 0 to determine whether the obstacles can be detected in the frame, and if it is still not recommended to adjust the height of the camera mounting.

• The point cloud map of the upper computer can select the obstacle objects but the output of obstacle avoidance results is 0. This phenomenon may be caused by the incorrect configuration of the camera installation height, and it is recommended that the user adjust the camera installation height.

8.3 Obstacle avoidance parameter does not take effect

• Configuration parameters were configured without clicking Setup Complete.

• Illegal parameters were modified, causing the setup to fail.

• An unstable camera connection appears disconnected, causing the downlink parameter to fail.

8.4 Mounting structure interference

• The presence of structures around the camera that block the camera's field of view at the time of installation results in abnormal camera data.

• The structural design of the camera was designed to interfere with the rear field of view resulting in abnormal camera data.

• The camera mirror protective film was not removed, resulting in abnormal camera data.

• The camera mirror is fitted with other protective glass causing data anomalies.

9. Others