Perelman School of Medicine at the University of Pennsylvania

Section for Biomedical Image Analysis (SBIA)

participating with CBICA

Multimodal Brain Tumor Segmentation Challenge 2017

ScopeRelevanceTasks • Data • Data Request EvaluationParticipation SummaryPrevious BraTSPeople


The datasets used in this year's challenge have been updated, since BraTS'16, with more routine clinically-acquired 3T multimodal MRI scans and all the ground truth labels have been manually-revised by expert board-certified neuroradiologists.

Ample multi-institutional routine clinically-acquired pre-operative multimodal MRI scans of glioblastoma (GBM/HGG) and lower grade glioma (LGG), with pathologically confirmed diagnosis and available OS, will be provided as the training, validation and testing data for this year’s BraTS challenge. These multimodal scans describe a) native (T1) and b) post-contrast T1-weighted (T1Gd), c) T2-weighted (T2), and d) T2 Fluid Attenuated Inversion Recovery (FLAIR) volumes, and were acquired with different clinical protocols and various scanners from multiple (n=19) institutions, mentioned as data contributors here. All the imaging datasets have been segmented manually, by one to four raters, following the same annotation protocol, and their annotations were approved by experienced neuro-radiologists. Annotations comprise the GD-enhancing tumor (ET — label 4), the peritumoral edema (ED — label 2), and the necrotic and non-enhancing tumor (NCR/NET — label 1), as described in the BraTS reference paper, published in IEEE Transactions for Medical Imaging (also see Fig.1). The provided data are distributed after their pre-processing, i.e. co-registered to the same anatomical template, interpolated to the same resolution (1 mm^3) and skull-stripped.

The data provided during BraTS'17 differs significantly from the data provided during the previous BraTS challenges. The only data that have been previously used and will be utilized again (during BraTS'17) are the images and annotations of BraTS'12-'13, which have been manually annotated by clinical experts in the past. The data used during BraTS'14-'16 (from TCIA) have been discarded, as they described a mixture of pre- and post-operative scans and their ground truth labels have been annotated by the fusion of segmentation results from algorithms that ranked highly during BraTS'12 and '13. This year, expert neuroradiologists have radiologically assessed the complete original TCIA glioma collections (TCGA-GBM, n=262 and TCGA-LGG, n=199) and categorized each scan as pre- or post-operative. Subsequently, all the pre-operative TCIA scans (135 GBM and 108 LGG) were annotated by experts for the various glioma sub-regions and included in this year's BraTS datasets.

Participants are not allowed to use additional private data (from their own institutions) for data augmentation, since our intentions are to provide a fair comparison among the participating methods. The only case that this will be considered as a valid contribution is if they also report results using only the BraTS'17 data and discuss any potential difference in the results.

The overall survival (OS) data, defined in days, will be included in a comma-separated value (.csv) file with correspondences to the pseudo-identifiers of the imaging data.

Validation data is released on June 30, through an email pointing to the accompanying leaderboard. This, will allow participants to obtain preliminary results in unseen data and also report it in their submitted papers, in addition to their cross-validated results on the training data. The ground truth of the validation data will not be provided to the participants, but multiple submissions to the online platform (CBICA's IPP) will be allowed.

Finally, all participants will be presented with the same test data, which will be made available through email during 1-21 August (Extended to 1-27) and for a limited controlled time-window (48h), before the participants are required to upload their final results in CBICA's IPP. The top-ranked participating teams will be invited before the end of August to prepare slides for a short oral presentation of their method during the BraTS challenge.


Data Usage Agreement / Citations

You are free to use and/or refer to the BraTS datasets in your own research, provided that you always cite the following three manuscripts:

[1] B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)", IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694

[2] S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J.S. Kirby, et al., "Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features", Nature Scientific Data, 4:170117 (2017) DOI: 10.1038/sdata.2017.117

[3] S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, et al., "Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge", arXiv preprint arXiv:1811.02629 (2018)

In addition, if there are no restrictions imposed from the journal/conference you submit your paper about citing "Data Citations", please be specific and also cite the following:

[4] S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., "Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-GBM collection", The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.KLXWJJ1Q

[5] S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., "Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-LGG collection", The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.GJQ7R0EF

Feel free to send any communication related to the BraTS challenge to