Tasks

To participate, each participants will have to submit runs for the following two tasks:

• Detection The detection task stems from the requirement of the high detection accuracy needed to be viable for use in a clinical setting. Participants are asked to develop algorithms that achieve high classification scores on the 23 different classes presented in the development dataset. Each submission should consist of a single ".csv" file, which contains one line per image in the unlabeled dataset. Each line should start with the name of the predicted file, followed by the predicted label, and then end with the model's confidence of that prediction.
• Efficient Detection The efficient detection task focuses on the real-time analysis needed to deliver instant feedback to doctors performing endoscopies. To satisfy this requirement, the algorithm must achieve good classification scores while also being able to classify images as fast as they are put on screen, which is approximately 30 frames per second. Submissions to this task require the participants to submit their code in the form of a Docker image so that we, the organizers, can evaluate the submissions on the same hardware. The Docker image should produce a ".csv" file similar to that of the detection task, with the addition of the processing time being added to the end of each line.
• Segmentation In the segmentation task, we ask participants to use test images provided in the dataset to generate segmentations of polyps automatically. Polyps are clumps of cells that form on the mucosal wall of the GI tract. They are among the most important findings in an endoscopy procedure as they are a precursor to different cancer types, including colorectal cancer, which is one of the most lethal cancer types. For this task, we ask the participants to submit the zip file that contains predicted mask. The size of the predicted mask should be equal to the provided test dataset.

Sumbission Details

Submissions should be submitted in separated files with the following names, depending on the subtask:

 endotect20_GroupName_SubmissionName_SubmissionId.csv

• GroupName The name of the participating group, maximum 12 symbols (example: colonwizards)
• SubmissionName The id of the submission, maximum 9 symbols, one of the following ids: detection, speed, segmentation.
• SubmissionId The id of the submission, maximum 15 symbols (example: ver1)
• Example endotect20_colonwizards_detection_ver1.csv

To submit, please zip the submission files and send the zip archive via regular e-mail to the task organizers Pål Halvorsen (paalh at simula.no), Steven Hicks (steven at simula.no) and Michael Riegler (Michael at simula.no).

More details on how to submit for the specific tasks can be found found below.

Sumbission Format

Information on how to prepare the Docker image and submissions can be found here.

Detection (ID: detection)

Submissions to the detection task should be a .CSV file containing the the image name, classification label, and confidence value. The .CSV file should be generated using the test dataset.

Ranking Submissions to the detection task will be ranekd based on the achieved MCC on the test dataset. In all the submissions the confidence value/interestingness level/rank value refers to rank orders, expressing an expectation that some classified instances are more likely to be positive / correctly classified than others. The rank value is expected to be a floating-point value within [0.0;1.0] interval deciding for any pair of instances whether the first is more likely (>), equally likely (=), or less likely (<) to be positive / correctly classified than the second. If your algorithm does not provide any ranking for the classified instances, you must use 1.0 as the rank value.

Efficient Detection (ID: speed)

Submissions to the efficient detection task are similar to the detection task, but also include information about processing time. The .CSV file should contain the image name, classification label, confidence value, image processing time in seconds. The .CSV file should be generated by a Docker image containing your code and takes the test dataset as input.

Ranking Submissions ot the Efficient detection task will be ranked based on their processing time in seconds and achieved MCC. The image processing time in seconds value refers to a time interval from the moment when image has been completely loaded into memory to the moment of the final decision about the image class made by the classification algorithm. In other words, the full amount of time required to classify a particular image minus time spent to image loading and classification results saving.

Segmentation (ID: segmentation)

Submissions to the segmentation task should be a single zip file for all the predicted mask. The name of the predicted mask image should be the same as the input image.

Ranking Submissions to the segmentation task will be ranked based on their achieved DICE score on the provided test dataset.