The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

Bjoern H. Menze; Andras Jakab; Stefan Bauer; Jayashree Kalpathy-Cramer; Keyvan Farahani; Justin Kirby; Yuliya Burren; Nicole Porz; Johannes Slotboom; Roland Wiest; Levente Lanczi; Elizabeth Gerstner; Marc-Andre Weber; Tal Arbel; Brian B. Avants; Nicholas Ayache; Patricia Buendia; D. Louis Collins; Nicolas Cordier; Jason J. Corso; Antonio Criminisi; Tilak Das; Herve Delingette; Cagatay Demiralp; Christopher R. Durst; Michel Dojat; Senan Doyle; Joana Festa; Florence Forbes; Ezequiel Geremia; Ben Glocker; Polina Golland; Xiaotao Guo; Andac Hamamci; Khan M. Iftekharuddin; Raj Jena; Nigel M. John; Ender Konukoglu; Danial Lashkari; Jose Antonio Mariz; Raphael Meier; Sergio Pereira; Doina Precup; Stephen J. Price; Tammy Riklin Raviv; Syed M. S. Reza; Michael Ryan; Duygu Sarikaya; Lawrence Schwartz; Hoo-Chang Shin; Jamie Shotton; Carlos A. Silva; Nuno Sousa; Nagesh K. Subbanna; Gabor Szekely; Thomas J. Taylor; Owen M. Thomas; Nicholas J. Tustison; Gozde Unal; Flor Vasseur; Max Wintermark; Dong Hye Ye; Liang Zhao; Binsheng Zhao; Darko Zikic; Marcel Prastawa; Mauricio Reyes; Koen Van Leemput

doi:10.1109/TMI.2014.2377694

Journal article

The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

Bjoern H. Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, …

IEEE transactions on medical imaging, Vol.34(10), pp.1993-2024

2015-10-01

DOI: https://doi.org/10.1109/TMI.2014.2377694

PMCID: PMC4833122

PMID: 25494501

Abstract

Computer Science

Computer Science, Interdisciplinary Applications

Engineering

Engineering, Biomedical

Engineering, Electrical & Electronic

Imaging Science & Photographic Technology

Life Sciences & Biomedicine

Radiology, Nuclear Medicine & Medical Imaging

Science & Technology

Technology

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low-and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

Metrics

4 Record Views

1733 Times Cited - Web of Science

See more details

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Highly Cited Paper

Collaboration types: Industry collaboration; Domestic collaboration; International collaboration
Citation topics: 4 Electrical Engineering, Electronics & Computer Science; 4.17 Computer Vision & Graphics; 4.17.128 Deep Learning
Web Of Science research areas: Computer Science, Interdisciplinary Applications; Engineering, Biomedical; Engineering, Electrical & Electronic; Imaging Science & Photographic Technology; Radiology, Nuclear Medicine & Medical Imaging
ESI research areas: Clinical Medicine

UN Sustainable Development Goals (SDGs)

This output has contributed to the advancement of the following goals:

Source: InCites

Details

Title: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)
Creators: Bjoern H. Menze - Technical University of Munich
Andras Jakab - ETH Zurich
Stefan Bauer - University of Bern
Jayashree Kalpathy-Cramer - Harvard University
Keyvan Farahani - National Institutes of Health
Justin Kirby - National Institutes of Health
Yuliya Burren - University of Bern
Nicole Porz - University of Bern
Johannes Slotboom - University of Bern
Roland Wiest - University of Bern
Levente Lanczi - University of Debrecen
Elizabeth Gerstner - Harvard University
Marc-Andre Weber - University Hospital Heidelberg
Tal Arbel - McGill University
Brian B. Avants - University of Pennsylvania
Nicholas Ayache - French Institute for Research in Computer Science and Automation
Patricia Buendia - Infotech Soft
D. Louis Collins - McGill University
Nicolas Cordier - French Institute for Research in Computer Science and Automation
Jason J. Corso - University at Buffalo, State University of New York
Antonio Criminisi - Microsoft
Tilak Das - University of Cambridge
Herve Delingette - French Institute for Research in Computer Science and Automation
Cagatay Demiralp - Stanford University
Christopher R. Durst - University of Virginia
Michel Dojat - French Institute for Research in Computer Science and Automation
Senan Doyle - French Institute for Research in Computer Science and Automation
Joana Festa - Univ Minho, Dept Elect, P-4719 Braga, Portugal
Florence Forbes - French Institute for Research in Computer Science and Automation
Ezequiel Geremia - French Institute for Research in Computer Science and Automation
Ben Glocker - Imperial College London
Polina Golland - Massachusetts Institute of Technology
Xiaotao Guo - Columbia University
Andac Hamamci - Sabancı Üniversitesi
Khan M. Iftekharuddin - Old Dominion University
Raj Jena - University of Cambridge
Nigel M. John - Infotech Soft
Ender Konukoglu - Harvard University
Danial Lashkari - Massachusetts Institute of Technology
Jose Antonio Mariz - RMIT University
Raphael Meier - University of Bern
Sergio Pereira - Univ Minho, Dept Elect, P-4719 Braga, Portugal
Doina Precup - McGill University
Stephen J. Price - University of Cambridge
Tammy Riklin Raviv - Massachusetts Institute of Technology
Syed M. S. Reza - Old Dominion University
Michael Ryan - Infotech Soft
Duygu Sarikaya - University at Buffalo, State University of New York
Lawrence Schwartz - Columbia University
Hoo-Chang Shin
Jamie Shotton - Microsoft
Carlos A. Silva - Univ Minho, Dept Elect, P-4719 Braga, Portugal
Nuno Sousa - RMIT University
Nagesh K. Subbanna - McGill University
Gabor Szekely - ETH Zurich
Thomas J. Taylor - Infotech Soft
Owen M. Thomas - University of Cambridge
Nicholas J. Tustison - University of Virginia
Gozde Unal - Sabancı Üniversitesi
Flor Vasseur - French Institute for Research in Computer Science and Automation
Max Wintermark - University of Virginia
Dong Hye Ye - Purdue University West Lafayette
Liang Zhao - University at Buffalo, State University of New York
Binsheng Zhao - Columbia University
Darko Zikic - Microsoft
Marcel Prastawa - General Electric
Mauricio Reyes - University of Bern
Koen Van Leemput - Harvard University
Publication Details: IEEE transactions on medical imaging, Vol.34(10), pp.1993-2024
Publisher: IEEE
Number of pages: 32
Grant note: P41-RR13218 / NIH NCRR NAC; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Center for Research Resources (NCRR) 600841 / European Union; European Commission TEKES (ComBrain) R15CA115464 / NIH NCI; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Cancer Institute (NCI) FCOM-01-0124-FEDER-022674 / COMPETE P41-RR14075 / NIH NCRR; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Center for Research Resources (NCRR) Swiss Cancer League Swiss NSF project Computer Aided and Image Guided Medical Interventions (NCCR CO-ME) MedYMA 2011-291080 / European Research Council through ERC; European Research Council (ERC) 291763 / Technische Universitat Munchen-Institute for Advanced Study - European Union P41RR014075 / NATIONAL CENTER FOR RESEARCH RESOURCES; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Center for Research Resources (NCRR) U54-EB005149 / NIH NIBIB NAMIC; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Institute of Biomedical Imaging & Bioengineering (NIBIB) PIRG-GA-2008-231052 / MICAT Project (EU) FCT; Portuguese Foundation for Science and Technology; European Commission R01EB013565 / NIH NIBIB; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Institute of Biomedical Imaging & Bioengineering (NIBIB) R141-2013-13117 / Lundbeck Foundation; Lundbeckfonden P41-EB-015902 / NIH NIBIB NAC U01CA154601 / NATIONAL CANCER INSTITUTE; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Cancer Institute (NCI) NIHR/CS/009/011 / National Institute for Health Research; National Institute for Health Research (NIHR) 133611 / Academy of Finland Swiss Institute for Computer Assisted Surgery (SICAS) Technische Universitat Munchen-Institute for Advanced Study - German Excellence Initiative R01EB013565 / NATIONAL INSTITUTE OF BIOMEDICAL IMAGING AND BIOENGINEERING; United States Department of Health & Human Services; National Institutes of Health (NIH) - USA; NIH National Institute of Biomedical Imaging & Bioengineering (NIBIB) Marie Curie COFUND program of the European Union
Academic Unit: CoE - Electrical & Computer Engineering; College of Engineering
Language: English
Resource Type: Journal article
PMID: 25494501
PMCID: PMC4833122
Record Identifier: 991031725826402976