Skip to main content
Public Health

Evaluation of the European Reference Networks

The Commission launched in December 2022 the evaluation of the European Reference Networks (ERNs), which is an opportunity to learn and improve the existing system. It will also identify lessons learnt and areas for improvement, and ultimately if the ERNS are delivering the expected benefits for patients.

Legal Framework

According to the Commission Implementing Decision 2014/287/EU, all networks and their members must be periodically evaluated by a Commission-appointed body at least five years after their approval or last evaluation and every five years after that, if applicable. The evaluation process and methodology are based on the Implementing Decision 2014/287/EU provisions, as specified by the Evaluation Manual and the Toolbox.

The aim of the evaluation is to verify and assess:

  1. the fulfilment of the criteria and conditions set out in Delegated Decision 2014/286/EU;
  2. the accomplishment of the objectives set out in Article 12(2) of Directive 2011/24/EU;
  3. the outcomes and performance of the network and the contribution of each member.

Timeline

The evaluation process will take about a year, from the appointment of the Independent Evaluation Body to the issue of the final evaluation reports.

Key phases of the evaluation process:

  • Preparation for the set-up of the evaluation and information of ERNs members and organisation of tasks.
  • Self-evaluation of ERNs and their Health Care Provider members, which includes an internal validation of the self-evaluation results by the ERN Coordinator and verification and confirmation by the Independent Evaluation Body.
  • Technical evaluation by the Independent Evaluation Body:
    • Network Technical Evaluation, which includes a document review, a virtual interview with the ERNs, the Work-Package coordinators, and the Patient Representatives.
    • Health care providers technical evaluation, which includes a document review, and an on-site audit of a sample-of selected health care providers.
  • Evaluation results which includes drafting and finalising the Evaluation Reports, one per network and health care provider.  This process involves receiving feedback and making amendments and improvements.
Evaluation of the European Reference Networks
Timeline of the evaluation process: IEB (Independent Evaluation Body); ERN (European Reference Network for Rare Diseases); HCP (Health Care Provider)

The results of the evaluation will be made publicly accessible on this website.

Evaluation process

Independent Evaluation Body

The assessment will be carried out by an Independent Evaluation Body (IEB), an expert and independent assessment organisation appointed by the Commission following an open call for interest. The Independent Evaluation Body will evaluate networks and healthcare providers externally.

General description of the process

The process to evaluate ERNs was established in February 2022 employing the Independent Evaluation Body. All the relevant stakeholders (e.g. Board of Member States, ERNs, and patient organisations) were consulted on numerous occasions.

The evaluation process includes some preparatory tasks and 4 main steps:

  1. Preparatory tasks:
    • Done by the Independent Evaluation Body: organisational, managerial, and logistical arrangements, like training the evaluators, adapting the IT platform, etc.
    • Done by the ERNs coordinators: they inform the members and patient representatives about the procedure, and agree on responsibilities, activities, and deadlines.
  2. Self-evaluation: One self-evaluation is expected from each ERN and its members. These self-evaluations will help determine whether and to what extent the ERNs have met their original objectives.
  3. Technical evaluation: done by the Independent Evaluation Body, including interviews, a documentation review and on-site /online audits.
  4. Results: reports for each network, for each Health Care Provider (HCP) and for the ERNs system.

Documentation

Operational Criteria for the Evaluation of ERNs and HCPs

Each criterion is followed by a list of measurable elements, used to assess the specified aspect or level of performance.

Some of the measurable elements are considered “core” and should have been accomplished or implemented during the evaluation. The other measurable elements refer to important areas in which networks or their members should work and whose level of development indicate their maturity status.

The evaluation criteria are divided between:

  • European Reference Networks: 20 operational criteria with 52 measurable elements, of which 30 are core and classified in 7 areas; including governance and coordination, clinical care, quality and patient safety, patient-centred care, contributions to research, education and training, and networking and dissemination.
  • Healthcare Providers: 24 operational criteria with 64 measurable elements, of which 29 are core, and classified in 7 areas have; including patient -centred care, organisation and management, research and training, exchange of expertise and e-health, quality and safety, competence and outcomes of care, and human resources.

Evaluation Manual

The Evaluation Manual is the reference handbook, common for all parties involved, that includes both the evaluation of the networks and the healthcare providers.

Evaluation Toolbox

The Evaluation Toolbox is a collection of 18 items including: guidelines, forms, templates, references, and documents. Like the evaluation manual, the toolbox is accessible to everyone to comply with both principles of transparency and fairness to all parties.