Grand Challenge

---

The Grand Challenge (GC) platform is a digital research infrastructure providing a suite of services to support the development and benchmarking of AI algorithms in medical imaging and related fields. The platform provides three core interoperable services: challenges, reader studies, and controlled algorithm execution.

Challenges

Challenges are competitions designed to identify the best-performing AI algorithm for a specific task, evaluated against a defined set of metrics. Organizers provide a training dataset, a test dataset, a task description, and the evaluation metrics. Participants download the training data, develop their algorithm, and submit it as a solution.

 The GC platform supports challenge organizers by:

• Providing a customizable challenge website for documentation, training data distribution, and participant management
• Enabling communication with participants via direct messaging and forums
• Securely storing the test dataset used to evaluate participant submissions
• Validating and executing submitted solutions against the test dataset in a secure, scalable environment
• Running the evaluation pipeline to compute performance metrics for each submission
• Displaying a live leaderboard, updated with each successful submission
• Offering hands-on support throughout the setup and execution of the challenge
• Keeping the challenge publicly accessible on the platform for a minimum of five years

Reader Studies

Reader studies enable the collection of structured annotations or clinical assessments on a defined dataset. By presenting a set of questions to clinicians or annotators, organizers produce enriched datasets that can serve as training data, test data, or a clinical benchmark within a challenge.

The GC platform's reader study feature allows organizers to:

• Upload medical images and supporting data (video, text, PDF, etc.)
• Organize uploaded data into logical cases to define study content
• Configure study questions at the case, image, or annotation level. Question types range from simple yes/no responses and malignancy ratings to detailed segmentation tasks, such as lung nodule delineation on CT scans
• Invite readers from any location to participate, as the study is fully browser-based
• Download collected responses upon study completion

Algorithms

Algorithms developed by platform users or challenge participants can be shared with others through two access modes:

• Public access - The algorithm is available to all verified platform users, who can run it on their own data.
• Restricted access - The algorithm is made available to a specific set of users within a controlled environment.

In both cases, strict access controls are enforced: algorithm owners cannot access the data submitted by users, and users cannot access the underlying algorithm - only its output. This ensures data privacy and intellectual property protection for all parties.

Connecting Services

The three services on the GC platform are designed to work together, enabling end-to-end AI development and validation workflows. Algorithm outputs can be directly fed into a reader study, allowing clinicians or annotators to review, assess, or enrich the results produced by an algorithm. Reader studies, in turn, can be integrated into challenges — either as a source of training or test data, or as a mechanism for clinical benchmarking of submitted solutions. Finally, algorithms that emerge from a challenge can be published on the platform, making them available to the broader user community. This interconnected design allows organizers, developers, and clinicians to move seamlessly from data collection and annotation, through algorithm development and benchmarking, to deployment and shared access.

The services are aligned with FAIR data principles and support secure, privacy-preserving analysis workflows, enabling data to be accessed, processed, and evaluated. In this way, Grand-Challenge contributes to emerging European data space initiatives, such as the European Health Data Space (EHDS), by facilitating secondary use of health data for research and innovation under controlled conditions.