WAD 2018 Challenges

Berkeley DeepDrive is hosting three challenge tasks for CVPR 2018 Workshop on Autonomous Driving: Road Object Detection, Drivable Area Segmentation, and Domain Adaptation of Semantic Segmentation.

Submission Deadline June 11, 2018
Result Announcement June 16, 2018
Update Our challenges successfully concluded. Here are the Top 3 teams in each challenge:

Drivable Area

Team Members Organization Score
IBN_PSA/P Xingang Pan*, Hengshuang Zhao*, Jianping Shi, Jiaya Jia CUHK, SenseTime, Tencent 86.18
Mapillary Research Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder Mapillary Research 86.04
DiDi AI Labs Qiaoyi Li, Zhengping Che, Guangyu Li, Ying Lu, Bo Jiang, Yan Liu DiDi AI Labs 84.01

Road Object Detection

Team Members Organization Score
Sogou_MM Tao Wei, Bin Li, Xiaoyun Liang Sogou 33.10
VDIG0 Qijie Zhao,Tao Sheng,Feng Ni Institute of computer science & technology of Peking University, VDIG 29.69
seb Sebastian Bayer Karlsruhe Institute of Technology 20.66

Domain Adaption

Team Members Organization Score
NvDA Aysegul Dundar, Zhiding Yu, Ming-Yu Liu, Ting-Chun Wang, John Zedlewski, Jan Kautz Nvidia 62.40
DiDi AI Labs Zhengping Che, Qiaoyi Li, Ying Lu, Guangyu Li, Bo Jiang, Yan Liu DiDi AI Labs 57.67
CMU-GM Yang Zou, Vijayakumar Bhagavatula, Jinsong Wang Carnegie Mellon University and General Motors 54.59
If you have any questions, please email bdd-data-help@googlegroups.com.

BDD100K Dataset

BDD Dataset

The competition is based on BDD100K, the largest driving video dataset to date. It contains 100,000 videos representing more than 1000 hours of driving experience with more than 100 million frames. The videos comes with GPU/IMU data for trajectory information. They are manually tagged with weather, time of the day, and scene types. We also labeled bounding boxes of all the objects on the road, lane markings, drivable areas and detailed full-frame instance segmentation. To obtain the dataset, please log in and go to the download pages. The challenges include three tasks based on BDD100K: road object detection, drivable area segmentation and full-frame semantic segmentationThere are 70,000 training and 10,000 validation images for the first two tasks. 7,000 training and 1,000 validation images are provided for the third task.

Evaluation

In the our portal, you can upload your results of challenges in Submission tab in the same portal of data downloading. The evaluation metric is documented in our github repo. You can use our evaluation code to check your results on validation set. We will put up the leaderboard when the challenges conclude. For all tasks, it is a fair game to pre-train your network with ImageNet, but if other datasets are used, please note in the submission description. We will rank the methods without using external datasets except ImageNet.

Task 1: Road Object Detection

All the bounding boxes information is stored in the label files in .json format.

Training images: bdd100k/images/100k/train
Training labels: bdd100k/labels/100k/train
Validation images: bdd100k/images/100k/val
Validation labels: bdd100k/labels/100k/val

The evaluation will be based on testing results of the images in bdd100k/images/100k/test.

Task 2: Drivable Area Segmentation

All the drivable area annotations are also stored in the label files. But we also provide “Drivable Maps”, which provide the drivable area annotation in label map format. They can be used directly in semantic segmentation training.

Training images: bdd100k/images/100k/train
Training labels: bdd100k/drivable_maps/100k/train
Validation images: bdd100k/images/100k/val
Validation labels: bdd100k/drivable_maps/100k/val

The evaluation will be based on testing results of the images in bdd100k/images/100k/test.

Task 3: Domain adaptation of Semantic Segmentation

The training annotations for semantic segmentation is provided in label map format. The data name in the portal is Segmentation under BDD100K. We currently follow the same “train_id” with Cityscapes dataset. That is, each pixel has ids from 0-18 for training categories or 255 for ignored regions.

Training images: bdd100k/seg/images/train & Apollo training images.
Training labels: bdd100k/seg/labels/train

Validation: There is no official validation set for the video segmentation challenge.

Testing: Testing set of Video Segmentation Challenge.

Note: We evaluate the label maps based on Cityscapes train_id, even for the Apollo data. The mapping from Apollo categories to train ids is provided. To relieve your burden for evaluation and submission, we subsample the testing images for Camera_5 by every 5th frame. Also, the testing labels for road01_ins is problematic and they won't be used for evaluation in any of the challenges. Therefore, we evaluate on 179 images in this list. The list in md5 coding is also available. Please note that our evaluation server expects the file names to be in md5 coding.