Prerpints.org logo
Preprint
Data Descriptor

SaBi3d - A LiDAR Point Cloud Data Set of Car-to-Bicyle Overtaking Maneuvers

Altmetrics

Downloads

84

Views

38

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

04 June 2024

Posted:

05 June 2024

You are already at the latest version

Alerts
Abstract
While cycling presents environmental benefits and promotes a healthy lifestyle, the risks associated with overtaking maneuvers by motorized vehicles pose significant hindrances for many potential cyclists. Large-scale analysis of overtaking maneuvers could inform traffic researches and city planners on how to reduce these risks by better understanding these maneuvers. Drawing from the fields of sensor-based cycling research and of LiDAR-based traffic data sets, this paper provides a step towards addressing these safety concerns by introducing the Salzburg Bicycle 3d (SaBi3d) data set, consisting of LiDAR point clouds capturing car-to-bicycle overtaking maneuvers. The data set, collected using a LiDAR-equipped bicycle, facilitates detailed analysis of a large quantity of overtaking maneuvers without the need for manual annotation through enabling automatic labeling by a neural network. Additionally, a benchmark result for 3d object detection using a competitive neural network is provided as a baseline for future research. The SaBi3d data set is structured identically to the nuScenes data set, and therefore offers compatibility with numerous existing object detection systems. This work provides valuable resources for future researchers to better understand cycling infrastructure and mitigate risks, thus promoting cycling as a viable mode of transportation.
Keywords: 
Subject: Computer Science and Mathematics  -   Computer Vision and Graphics

1. Summary

Cycling has multiple advantages over other modes of transport, such as reducing emissions in city centers [1] and promoting a healthy lifestyle [2]. Therefore the European Union, and some of its member states like Austria, have declared the goal of increasing the bicycle modal share [3,4]. However, cycling is one of the most dangerous modes of transport [5]. Besides the positive effects of cycling, it also faces a higher risk of fatalities compared to other transport modes, especially in case of inadequate cycling infrastructure [6].
Longitudinal overtaking maneuvers by motorized vehicles contribute most to a feeling of subjective risk, thus presenting a barrier to cycling participation [7], and are likely to lead to severe injury in the event of an accident [8]. Therefore, many studies have tried to capture characteristics of overtaking maneuvers in recent years. Most of these capture limited data such as the minimum lateral clearance and overtaking velocity at a single point [9,10,11], providing only a snapshot of an overtaking maneuver without capturing data representing the whole maneuver, which is a complex process. On the other hand, works relying on more detailed data, such as LiDAR point clouds, often require manual annotation, which is time-consuming and expensive [12,13]. A third approach is to use test tracks and equipment with extensive sensor coverage [14,15]. However, naturalistic studies are often preferable to get results representative for real world interactions. Therefore, it would be desirable to have a system that records point cloud data from an instrumented bicycle, automatically detecting vehicles in the data and extracting rich metrics. This would allow for a very detailed description of overtaking maneuvers in naturalistic environments without the need for further manual annotation and thereby remove the drawbacks mentioned above.
To this end, we present a new data set, the Salzburg Bicycle 3d (SaBi3d) data set, which is a first step towards bridging this gap. It consists of 59 overtaking maneuvers manually annotated in a total of 4811 point clouds collected by a LiDAR-equipped bicycle. This data set can be seamlessly substituted for the nuScenes [16] data set for object detection tasks without any additional adaptations.

2. Materials and Methods

As to the best of the authors knowledge no LiDAR data sets collected by a smart bicycle exist, this materials chapter includes a short review of the related research area. This way, necessary context for the understanding and usage of this data set is provided. Subsequently, the data collection using a LiDAR-equipped bicycle and the data preparation are discussed followed by a description of the metadata. Last, the creation of the object detection benchmark results is outlined.

2.1. Related Work

There are two research areas related to this work: First, cycling research that uses LiDAR, and more generally laser based distance measurements. LiDAR scanners only recently became small, light, and cheap enough to be mounted on bicycles, yielding a limited number of works in this area. The second area are LiDAR based traffic data sets collected by LiDARs mounted to a moving platform, a motorized vehicles for all regarded works. This is a wide area, therefore only the most relevant works are listed. Last in this section, a short introduction to the 3d object detection system used for the benchmark results and its surrounding research will be provided.

2.1.1. Lasers and LiDARs on Bicycles

There are multiple recent works on using single beam laser range finders on bicycles. Most often they are used for measuring lateral clearance during overtaking maneuvers [7,9,10,11,15]. Another idea is to control a single laser based distance sensor to search for and track single cars. This approach has been developed and extensively investigated in a series of works by Jeon Woonsung and Rajesh Rajamani [17,18,19,20,21]. This lead to a LiDAR based collision warning system on a bicycle [22,23], an approach also pursued by other authors [24]. One work first collected maps of the infrastructure, detecting moving objects by comparing newly collected data to these maps [25]. This approach however is hardly scalable for widespread use in collision warning systems.
LiDARs have also been used for analyzing cycling infrastructure. One work uses LiDARs on a bicycle to differentiate the cycling path from the surroundings [26]. Another uses LiDARs found in current smartphones and tablets for detecting infrastructure damages [27]. So far however, there has been no detailed analysis of traffic situations using LiDARs on bicycle, although such works have been announced as well [28].

2.1.2. Autonomous Driving Data Sets

Within the area of autonomous driving, the task of locating vehicles in LiDAR data has been worked on extensively in recent years. Therefore, research and especially data sets from this area can be used as a basis for similar research regarding LiDARs on bicycles. Commonly used data sets in this area are KITTI [29], nuScenes [16] and Waymo [30]. Most recent well-performing approaches are implemented for nuScenes, which is why this data set is chosen as a kind of blueprint for the SaBi3d data set. However, major shortcomings of this data set for the task at hand are that (1) the data is recorded from a car, and (2) the sequences are short (approximately 20 s), which makes it unlikely to capture complete over taking maneuvers.
nuScenes uses their own nuScenes Detection Score (NDS) as evaluation metric, which is composed of different Mean Average Precision (mAP) scores and True Positive scores for the different classes [16]. While in the nuScenes data set multiple classes are annotated, this will not be necessary for the SaBi3d data set since only car-to-bicycle overtaking maneuvers are of interest. Therefore, the mAP of the class ’car’ is taken as evaluation metric, which was also usual in earlier data sets [29,31].

2.1.3. 3D Object Detection

3d object detection algorithms are almost exclusively based on neural networks [32,33]. Object detection has originally been a computer vision task performed on image data [34], but has recently been adapted for point cloud data [35].
A 3d object detection algorithm consists of three components, i.e., (1) point cloud representation,(2) featureextraction, and (3) core object detection [33,35,36].
As a benchmark algorithm, VISTA [37] was chosen since it scored highest on the nuScenes leaderboard at the time. It uses a voxelized point cloud representation, various convolution operations [38,39,40], projection into two different views and cross-view transformers to bring the projections back together [41]. The core object detector can be varied, but the Object as Hotspots detector [42] yielded the best results [37]. Therefore, the same setup was used for the SaBi3d benchmark algorithm.

2.2. Data Collection

The data was collected using a Holoscene Edge bike by Boréal Bikes1. While this bicycle is equipped with multiple cameras, LiDAR sensors, IMUs, GNSS, and further sensors to measure brake activation, humidity and temperature, only the LiDAR sensors were used for this work. This is done to gain a more nuanced picture of the overtaking maneuver while preserving accurate depth information. The LiDAR sensors are Horizon LiDAR sensors by Livox [43]. Their scanning pattern is different from the scanning pattern used in previous data sets such as KITTI and nuScenes, which use rotating sensors. The Livox Horizon uses a scanning pattern reminding of a recumbent eight instead. Point clouds are sampled with a frequency of 5 Hz. The bicycle is equipped with five LiDAR sensors of this type. Since each of them has a horizontal field of view (FOV) of 81.7   ∘ , a full 360   ∘ FOV can be achieved by positioning them appropriately.
The data was collected in the afternoons of weekdays in October 2022 in Salzburg, Austria. The data was taken from four different locations, which were selected in such a way that a variety of speed limits and types of bicycle infrastructure would be captured.

2.3. Data Preparation

After the data was recorded, the individual point clouds of each of the sensors were merged into one point cloud for each point in time [26]. The timestamps between each individual point cloud varied slightly but they were all mapped to the closest multiple of 200 ms. The point clouds were further transformed into the format of the nuScenes point clouds.
Using the open-source annotation tool SUSTechPOINTS2 [44], cars were annotated in four sequences, resulting in approximately 10,000 bounding boxes in 4,811 frames, which corresponds to approximately 17 min of recordings.
To allow for the evaluation of training results with this data set, the SaBi3d data set was split into a training split and validation split. The validation split was supposed to contain approximately 15-20% of the total data. One scene was chosen according to its length, resulting in a 82/18% training/validation split.

2.4. Metadata Generation

Furthermore, metadata in the format of nuScenes was generated. This allows for the seamless substitution of the nuScenes data set for the SaBi3d data set when using it for training 3d object detection algorithms. The nuScenes metadata is a database that consists of 13 tables which each have their primary key token and some are linked by foreign keys3. An overview of the tables in the original nuScenes data set and the changes made for the SaBi3d data set is displayed in Table 1. The tables are split into three sections. The first set of tables was not changed at all. The second set was manually adjusted since only minor adjustments were needed. The third set of tables was generated from the recorded point cloud data itself since it required iterating over the scenes, frames, or annotation files.
Some seemlingly unnecessary adjustments were made in order to guarantee the seamless substitution of the nuScenes and SaBi3d data set. This involved the inclusion of a solid black PNG as map data and the definition of an ego pose at zero for every frame. Leaving those out would not allow further processing for object detection through the nuScenes devkit4 which is used by many object detection algorithms.
In addition to the nuScenes-style annotation of cars in the individual point clouds, annotations for full vehicle trajectories were added as well. In particular, it was annotated whether a vehicle performed an overtaking maneuver and whether it constitutes oncoming traffic. This annotation is not needed for 3d object detection but will be useful for potential subsequent work on this data set focusing on the automatic detection of overtaking maneuvers.

2.5. 3D Object Detection Benchmark

Along with the data set itself, we provide 3d object detection benchmark results for comparison with future works using the data set. As a baseline, the VISTA [37] algorithm was chosen. It was hypothesized that the voxelized point cloud representation would minimize the effect of the different scanning pattern of the Livox Horizon Lidar sensors. Therefore, a model trained on the nuScenes data set should easily be transferable to the SaBi3d data.

3. Results

While reproducing the results of VISTA [37] on nuScenes [16], it was discovered that the detection performance of VISTA heavily relies on the class-balanced grouping and sampling (CBGS) preprocessing step [45]. Without the resampling step, poor results were achieved (Table 2). While the original algorithm yielded results between 85.0 and 85.5 mAP of a maximum of 100, the result without resampling was much lower, 53.6 mAP. It was in a similar range when training and evaluating only on the class ’car’, i.e., 58.1 mAP. This posed a problem since the SaBi3d data set only includes one single class, therefore resampling based on classes was not possible.
However, when evaluating the results on the SaBi3d evaluation split (Table 3), this proved not to be a problem. The evaluation on the model trained on nuScenes data with resampling yielded very poor results (0.3 mAP). When fine-tuning this model on only the class ’cars’ from the nuScenes data set, a slight improvement to 12.6 mAP was observed. However, when fine-tuning it on the SaBi3d training data, a mAP of 80.2 on the SaBi3d evaluation data was achieved. Noteably, training the model from uninitialized weights yielded results of 79.1 mAP, almost as high as the fine-tuned model. This shows that the expensive pre-training on the nuScenes data set is not necessary to achieve good results on the SaBi3d data set with current 3d object detection algorithms.

4. Discussion

To increase the modal share of cycling, desirable because of its environmental and health benefits, the characteristics of, objectively or perceived, dangerous overtaking maneuvers need to be understood. A deeper understanding will enable traffic planners to implement measures that will enhance safety. This topic has been studied using a variety of sensors and methods. LiDAR is an obvious choice for measuring the required accurate distance and speed of overtaking cars, however so far the costly manual annotation step presented a major obstacle to its usage. The presented SaBi3d data set is the first public, annotated data set of LiDAR point clouds of car-to-bicycle overtaking maneuvers collected by the bicycle itself, allowing for training and evaluation of a corresponding object detection system. To facilitate the use of our data set, we chose to structure the data almost identical to the popular nuScenes data set, allowing it to be used in various existing object detection systems. We also trained the, at the time of our evaluation, best system on the nuScenes 3d object detection benchmark on our data to provide strong baseline results. Any works using this data set can and should strive to achieve at least this level of object detection performance before pursuing any follow up applications.
Future work might include expanding this data set using different LiDAR setups for ease of usage by other research groups. Another consequent step could be adding a tracking component to the benchmark system and calculation of safety metrics to allow end-to-end assessments of overtaking maneuvers. And last, the data set and benchmark system could be expanded to cover other interesting maneuvers, e.g. at intersections.

Author Contributions

Conceptualization, C.O. and M.B.; methodology, C.O. and M.B.; software, C.O.; validation, C.O. and M.B.; formal analysis, C.O. and M.B.; investigation, C.O. and M.B.; resources, C.O. and M.B.; data curation, C.O. and M.B.; writing—original draft preparation, C.O.; writing—review and editing, C.O. and M.B.; visualization, C.O.; supervision, M.B.; project administration, M.B.; funding acquisition, M.B. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK) under Grant GZ 2021-0.641.557.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data set is available at https://osf.io/k7cg9.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
LiDAR Light Detection and Ranging
NDS nuScenes Detection Score
mAP Mean Average Precision
VISTA Dual Cross-VIew SpaTial Attention
FOV Field of View
CBGS Class-balanced Grouping and Sampling

Notes

1
https://www.borealbikes.com/
2
https://github.com/naurril/SUSTechPOINTS
3
For detailed description see: https://www.nuscenes.org/nuscenes#data-format
4
https://github.com/nutonomy/nuscenes-devkit

References

  1. Lindsay, G.; Macmillan, A.; Woodward, A. Moving urban trips from cars to bicycles: impact on health and emissions. Australian and New Zealand Journal of Public Health 2011, 35, 54–60. [Google Scholar] [CrossRef] [PubMed]
  2. Buehler, R.; Pucher, J.R. (Eds.) Cycling for sustainable cities; Urban and industrial environments, The MIT Press: Cambridge, Massachusetts London, England, 2021. [Google Scholar]
  3. European Union. European Declaration on Cycling, 2023.
  4. Bundesministerium für Klimaschutz, Umwelt, Energie, Mobilität, Innovation und Technologie. Radverkehrsförderung in Österreich. Technical report, 2022.
  5. Wegman, F.; Zhang, F.; Dijkstra, A. How to make more cycling good for road safety? Accident Analysis & Prevention 2012, 44, 19–29. [Google Scholar] [CrossRef] [PubMed]
  6. European Transport Safety Council. How safe is walking and cycling in Europe? PIN Flash Report 38. Technical report, 2020.
  7. Beck, B.; Perkins, M.; Olivier, J.; Chong, D.; Johnson, M. Subjective experiences of bicyclists being passed by motor vehicles: The relationship to motor vehicle passing distance. Accident Analysis & Prevention 2021, 155, 106102. [Google Scholar] [CrossRef] [PubMed]
  8. Merk, J.; Eckart, J.; Zeile, P. Subjektiven Verkehrsstress objektiv messen – ein EmoCycling-Mixed-Methods-Ansatz. CITIES 20.50 – Creating Habitats for the 3rd Millennium: Smart – Sustainable – Climate Neutral. Proceedings of REAL CORP 2021, 26th International Conference on Urban Development, Regional Planning and Information Society, 2021, pp. 767–778.
  9. Beck, B.; Chong, D.; Olivier, J.; Perkins, M.; Tsay, A.; Rushford, A.; Li, L.; Cameron, P.; Fry, R.; Johnson, M. How Much Space Do Drivers Provide When Passing Cyclists? Understanding the Impact of Motor Vehicle and Infrastructure Characteristics on Passing Distance. Accident Analysis & Prevention 2019, 128, 253–260. [Google Scholar] [CrossRef] [PubMed]
  10. López, G.; Pérez-Zuriaga, A.M.; Moll, S.; García, A. Analysis of Overtaking Maneuvers to Cycling Groups on Two-Lane Rural Roads using Objective and Subjective Risk. Transportation Research Record: Journal of the Transportation Research Board 2020, 2674, 148–160. [Google Scholar] [CrossRef]
  11. Moll, S.; López, G.; Rasch, A.; Dozza, M.; García, A. Modelling Duration of Car-Bicycles Overtaking Manoeuvres on Two-Lane Rural Roads Using Naturalistic Data. Accident Analysis & Prevention 2021, 160, 106317. [Google Scholar] [CrossRef] [PubMed]
  12. Dozza, M.; Schindler, R.; Bianchi-Piccinini, G.; Karlsson, J. How Do Drivers Overtake Cyclists? Accident Analysis & Prevention 2016, 88, 29–36. [Google Scholar] [CrossRef] [PubMed]
  13. Rasch, A. Modelling Driver Behaviour in Longitudinal Vehicle-Pedestrian Scenarios. Master’s Thesis, Chalmers University of Technology, Göteborg, 2018. [Google Scholar]
  14. Rasch, A.; Boda, C.N.; Thalya, P.; Aderum, T.; Knauss, A.; Dozza, M. How Do Oncoming Traffic and Cyclist Lane Position Influence Cyclist Overtaking by Drivers? Accident Analysis & Prevention 2020, 142, 105569. [Google Scholar] [CrossRef] [PubMed]
  15. Rasch, A.; Dozza, M. Modeling Drivers’ Strategy When Overtaking Cyclists in the Presence of Oncoming Traffic. IEEE Transactions on Intelligent Transportation Systems 2022, 23, 2180–2189. [Google Scholar] [CrossRef]
  16. Caesar, H.; Bankiti, V.; Lang, A.H.; Vora, S.; Liong, V.E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; Beijbom, O. nuScenes: A multimodal dataset for autonomous driving, 2020. arXiv:1903.11027 [cs, stat].
  17. Jeon, W.; Rajamani, R. A novel collision avoidance system for bicycles. 2016 American Control Conference (ACC); IEEE: Boston, MA, USA, 2016; pp. 3474–3479. [Google Scholar] [CrossRef]
  18. Jeon, W.; Rajamani, R. Active Sensing on a Bicycle for Accurate Tracking of Rear Vehicle Maneuvers. Volume 2: Mechatronics; Mechatronics and Controls in Advanced Manufacturing; Modeling and Control of Automotive Systems and Combustion Engines; Modeling and Validation; Motion and Vibration Control Applications; Multi-Agent and Networked Systems; Path Planning and Motion Control; Robot Manipulators; Sensors and Actuators; Tracking Control Systems; Uncertain Systems and Robustness; Unmanned, Ground and Surface Robotics; Vehicle Dynamic Controls; Vehicle Dynamics and Traffic Control. American Society of Mechanical Engineers, 2016, p. V002T31A004. event-place: Minneapolis, Minnesota, USA, doi:10.1115/DSCC2016-9772.
  19. Jeon, W.; Rajamani, R. Rear Vehicle Tracking on a Bicycle Using Active Sensor Orientation Control. IEEE Transactions on Intelligent Transportation Systems 2018, 19, 2638–2649. [Google Scholar] [CrossRef]
  20. Jeon, W.; Rajamani, R. Active Sensing on a Bicycle for Simultaneous Search and Tracking of Multiple Rear Vehicles. IEEE Transactions on Vehicular Technology 2019, 68, 5295–5308. [Google Scholar] [CrossRef]
  21. Jeon, W.; Xie, Z.; Craig, C.; Achtemeier, J.; Alexander, L.; Morris, N.; Donath, M.; Rajamani, R. A Smart Bicycle That Protects Itself: Active Sensing and Estimation for Car-Bicycle Collision Prevention. IEEE Control Systems Magazine 2021, 41, 28–57, Publisher:IEEE. [Google Scholar] [CrossRef]
  22. Xie, Z.; Rajamani, R. On-Bicycle Vehicle Tracking at Traffic Intersections Using Inexpensive Low-Density Lidar. 2019 American Control Conference (ACC). IEEE, 2019, pp. 593–598. Place: Philadelphia, PA, USA, doi:10.23919/ACC.2019.8814442.
  23. Xie, Z.; Jeon, W.; Rajamani, R. Low-Density Lidar Based Estimation System for Bicycle Protection. IEEE Transactions on Intelligent Vehicles 2021, 6, 67–77. [Google Scholar] [CrossRef]
  24. Van Brummelen, J.; Emran, B.; Yesilcimen, K.; Najjaran, H. Reliable and Low-Cost Cyclist Collision Warning System for Safer Commute on Urban Roads. 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2016, pp. 003731–003735.
  25. Muro, S.; Matsui, Y.; Hashimoto, M.; Takahashi, K. Moving-Object Tracking with Lidar Mounted on Two-wheeled Vehicle: Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics. SCITEPRESS—Science and Technology Publications, 2019, pp. 453–459. Place: Prague, Czech Republic, doi:10.5220/0007948304530459.
  26. Niedermüller, A.; Beeking, M. Transformer Based 3D Semantic Segmentation of Urban Bicycle Infrastructure. Journal of Location Based Services 2024, pp. 1–23. [CrossRef]
  27. Vogt, J.; Ilic, M.; Bogenberger, K. A Mobile Mapping Solution for VRU Infrastructure Monitoring via Low-Cost LiDAR-sensors. Journal of Location Based Services 2023, pp. 1–23. [CrossRef]
  28. Yoshida, N.; Yamanaka, H.; Matsumoto, S.; Hiraoka,T.; Kawai,Y.; Kojima, A.; Inagaki, T. Development of Safety Measures of Bicycle Traffic by Observation with Deep-Learning, Drive Recorder Data, Probe Bicycle with LiDAR, and Connected Simulators. 2022.
  29. Geiger, A.; Lenz, P.; Urtasun, R. Are we Ready for Autonomous Driving? The KITTI Vision Benchmark Suite. 2012 IEEE Conference on Computer Vision and Pattern Recognition; IEEE: Providence, RI, 2012; pp. 3354–3361. [Google Scholar] [CrossRef]
  30. Sun, P.; Kretzschmar, H.; Dotiwalla, X.; Chouard, A.; Patnaik, V.; Tsui, P.; Guo, J.; Zhou, Y.; Chai, Y.; Caine, B.; Vasudevan, V.; Han, W.; Ngiam, J.; Zhao, H.; Timofeev, A.; Ettinger, S.; Krivokon, M.; Gao, A.; Joshi, A.; Zhao, S.; Cheng, S.; Zhang, Y.; Shlens, J.; Chen, Z.; Anguelov, D. Scalability in Perception for Autonomous Driving: Waymo Open Dataset, 2020. arXiv:1912.04838 [cs, stat]. arXiv:1912.04838 [cs, stat].
  31. Everingham, M.; Van Gool, L.; Williams, C.K.I.; Winn, J.; Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision 2010, 88, 303–338. [Google Scholar] [CrossRef]
  32. Mao, J.; Shi, S.; Wang, X.; Li, H. 3D Object Detection for Autonomous Driving: A Review and New Outlooks, 2022. arXiv:2206.09474 [cs].
  33. Wu, Y.; Wang, Y.; Zhang, S.; Ogai, H. Deep 3D Object Detection Networks Using LiDAR Data: A Review. IEEE Sensors Journal 2021, 21, 1152–1171. [Google Scholar] [CrossRef]
  34. Liang, W.; Xu, P.; Guo, L.; Bai, H.; Zhou, Y.; Chen, F. A survey of 3D object detection. Multimedia Tools and Applications 2021, 80, 29617–29641. [Google Scholar] [CrossRef]
  35. Zamanakos, G.; Tsochatzidis, L.; Amanatiadis, A.; Pratikakis, I. A comprehensive survey of LIDAR-based 3D object detection methods with deep learning for autonomous driving. Computers & Graphics 2021, 99, 153–181. [Google Scholar] [CrossRef]
  36. Fernandes, D.; Silva, A.; Névoa, R.; Simões, C.; Gonzalez, D.; Guevara, M.; Novais, P.; Monteiro, J.; Melo-Pinto, P. Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy. Information Fusion 2021, 68, 161–191. [Google Scholar] [CrossRef]
  37. Deng, S.; Liang, Z.; Sun, L.; Jia, K. VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention, 2022. arXiv:2203.09704 [cs].
  38. Graham, B. Spatially-sparse convolutional neural networks, 2014. arXiv:1409.6070 [cs].
  39. Graham, B. Sparse 3D convolutional neural networks, 2015. arXiv:1505.02890 [cs].
  40. Graham, B.; van der Maaten,L. Submanifold Sparse Convolutional Networks, 2017. arXiv:1706.01307[cs].
  41. Chen, Q.; Sun, L.; Cheung, E.; Yuille, A.L. Every View Counts: Cross-View Consistency in 3D Object Detection with Hybrid-Cylindrical-Spherical Voxelization. Advancesin Neural Information Processing Systems; Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.F.; Lin, H., Eds. Curran Associates,Inc., 2020, Vol.33, pp.21224–21235.
  42. Chen, Q.; Sun, L.; Wang, Z.; Jia, K.; Yuille, A. Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots, 2020. arXiv:1912.12791 [cs].
  43. Livox Tech. Livox Horizon: User Manual v1.0. Technical report, 2019.
  44. Li, E.; Wang, S.; Li, C.; Li, D.; Wu, X.; Hao, Q. SUSTech POINTS: A Portable 3D Point Cloud Interactive Annotation Platform System. 2020 IEEE Intelligent Vehicles Symposium (IV); IEEE: Las Vegas, NV, USA, 2020; pp. 1108–1115. [Google Scholar] [CrossRef]
  45. Zhu, B.; Jiang, Z.; Zhou, X.; Li, Z.; Yu, G. Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection, 2019. arXiv:1908.09492 [cs].
Table 1. Description of the metadata of the nuScenes data set and the adjustments made in the SaBi3d data set.
Table 1. Description of the metadata of the nuScenes data set and the adjustments made in the SaBi3d data set.
Table Name Content in nuScenes Changes in SaBi3d
attribute Possible properties of instances, e.g. being parked or moving. None; only one line is used (vehicle.moving).
category Object categories and subcategories. None; only one line is used (vehicle.car).
visibility Fraction of annotation that is visible. None; only one line is used (4; corresponding to 80-100% visibility).
calibrated_sensor Definition of sensors, including their orientation. Position of sensor adjusted to correct height.
sensor List of sensors. Sensor name changed.
log Information about the log files of the recording. Empty file since no corresponding log files were recorded by the sensors.
map File paths to the respective map images. Corresponding file changed to solid black PNG since no map data was recorded. Was used only for rendering images and not needed for object detection.
ego_pose Ego vehicle positions with respect to a global coordinate system. All values set to zero since no ego location was recorded. Positions of detections are therefore relative to the bicycle and not to a global coordinate system.
sample References to the frames that were annotated at 2 Hz (sampling frequency: 10 Hz). Adjusted to the data. Every frame is annotated at a sampling frequency of 5 Hz.
sample_data Paths to data files of the samples (LiDAR, Radar, image). Adjusted to the data; only paths to LiDAR files.
scene One entry for every scene. Adjusted to the data.
instance One entry for every unique vehicle (a particular vehicle might appear in multiple frames). Adjusted to the data.
sample_annotation Cuboid bounding boxes indicating the position and properties of objects.  Adjusted to the data by transforming the output of the labeling tool appropriately. Coordinates were transformed from Euler angles to quaternions. Visibility was not annotated and set to 4 for every annotation. The number of LiDAR points contained in the cuboid was not annotated and set to a reasonable average of 1000.
Table 2. Evaluation results on nuScenes data set.
Table 2. Evaluation results on nuScenes data set.
Model Training mAP (car)
Provided checkpoint [37] 85.0
Replicated, without resampling 53.6
Replicated, with resampling 85.5
Cars only 58.1
Table 3. Evaluation results on SaBi3d data set.
Table 3. Evaluation results on SaBi3d data set.
Model Training mAP (car)
Provided checkpoint [37] 0.3
Fine-tuning checkpoint on nuScenes cars 12.6
Fine-tuning checkpoint on SaBi3d 80.2
Training on SaBi3d 79.1
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated