Classification Contest

The Classification Contest

Keywords: MRI, heart, myocardial infarction, normal case, delayed-enhancement, classification

The goal of the Classification contest is to classify the exams in normal or pathological one, according to the clinical data with or without the DE-MRI exams.

The Classification contest is divided into two sub-challenges, the first one considering only the clinical information, and the second one considering both the clinical information and the DE-MRI.

Some cases could be sometimes ambiguous rendering this task not evident. Indeed patients coming in an emergency department could have other diseases, providing normal DE-MRI but abnormals clinical informations. For example, myocarditis could provide abnormal values for some clinical parameters, but normal DE-MRI. However, even if one parameter is ambiguous, considering the whole provided clinical parameters will prevent any big ambiguity.

The cohort consists of data extracted from 150 MRI exams (all from different patients) divided into 50 cases with normal MRI after the injection of a contrast agent and 100 cases with myocardial infarction (and then with a hyperenhanced area on DE-MRI), whatever their inclusion in the cardiac emergency department. The cases were randomly selected from our database. The inclusion criteria are patients received in the cardiac emergency department with acute disease (with symptoms of heart attack) and that undergo cardiac MRI. The exclusion criteria are patients with contraindications to the MRI and cardiac chronic diseases. There is an unbalanced distribution between normal and pathological cases, corresponding roughly to real life in managed exams in a MRI department.

The overall dataset was created from real clinical exams acquired from the MRI department at the University Hospital of Dijon (France). Each group was clearly defined according to physiological parameters and the presence or absence of a disease area on DE-MRI. The data are DE-MRI in short axis orientation, and a series of images covering the left ventricle.

Along with MRI, clinical characteristics are provided to distinguish normal and pathological cases. These characteristics are: sex, age, tobacco (Y/N/Former smoker), overweight (BMI > 25), arterial hypertension (Y/N), diabetes (Y/N), familial history of coronary artery disease (Y/N), ECG (ST+ (STEMI) or not), troponin (value), Killip max (between 1 and 4), ejection fraction of the left ventricle from echography (value), and NTproBNP (value).

Training and test case characteristics

The dataset provides 100 cases for the training and 50 cases for the testing.

Every training and test case represents a DE-MRI exam of the left ventricle. An exam (i.e. a case) consists of a series of 5 to 10 short-axis slices covering the left ventricle from the base to the apex. The ground-truths (contours of the relevant areas) will be provided with the training dataset. For each exam, associated clinical information will be provided in a text file. The images and the clinical information will be provided at the beginning of the challenge.

The training set with full ground-truth will comprise 100 cases (67 pathological cases, 33 normal cases) randomly selected among the 150 subjects.

The testing set is made of data from 50 subjects (33 pathological cases, 17 normal cases), all different from those in the training set.

For the training and the testing set, the unbalanced distribution between normal (1/3) and pathological (2/3) cases corresponds roughly to real life in managed exams in a MRI department.

Two sub-contests are designed, the first one consists in establishing the classification only with the clinical information, and the second one with the clinical information and the DE-MRI. In order to avoid any bias between the two contests, the order of the case is different from the segmentation contest, and moreover, news cases will replace randomly some of them for the classification contest.

Annotation characteristics and data pre processing

For both the training and test cases, the different areas are first outlined by an experienced user (a cardiologist with 10 years of experience in cardiology and MRI) and then the segmentation is verified by a second expert (a well-trained biophysicist with 20 years of experience).

There is no specific rule for the annotation of the clinical information. Data are recorded in a text file for each case with a simple layout.

To prevent the drawback of the displacement of the heart location between slices due to different breath-holds, the slices are realigned according to the gravity center of the area defined by the epicardial contour. The same process is applied for the training and test cases.

Assessment method - Metrics

As it is a binary classification, only the classification accuracy (normal vs pathological cases) is mandatory.

Submission method

During the on-site challenge, the participants will have one hour to run their methods on their own laptop in order to classify the exams between normal and pathologic cases.

They will provide a text file with the results of their classification (binary classification, the algorithms are not be requested). Then a ranking of the methods based on the classification accuracy will be provided immediately during the challenge.

Moreover, the participants will be requested to submit an article of four pages, following the MICCAI format, describing the methodology, with the opportunity to publish a long version of the article along with LNCS Challenge Proceedings. Articles will be reviewed by the organizing board and published on line if meeting the expected quality.

Important dates


Mid-April : Release of the training cases
October, 4: Online and on-site challenge (release of the testing cases).

Registration and challenge

Mid-April : Start of the registration process
September, 15: End of registration and deadline for submission of the papers.
October, 4: Online and on-site challenge.