Interview of Maria Papatzikou, Chair of the EA MLA Council, and Daniela Ionescu, Secretary of the EA MLA Council and manager of the peer-evaluation (PE) system
Accreditation developing in many new areas, digitalization booming in more and more sectors of daily operations, cybersecurity issues, Artificial Intelligence (AI) -related discussions, intensive development of internet-based processes, etc. These are a few of the major changes that organizations are faced with. EA and its members, the National Accreditation Bodies (NABs), are no exception. More specifically, changes in accreditation standards, but also changes in the processes in the NABs prompted EA to revisit its practices regarding peer evaluations.
“As members of EA, we continuously collect and analyse feedback from the market and national authorities, our main customers, and prescribers. It provides useful input for constant improvement and a real incentive to review our way of doing things. The same applies when we work in the EA MLA Council (MAC) to improve the peer-evaluation system of EA NABs”, says Maria.
The results of the survey made among the EA members highlighted five major improvement routes in the peer-evaluation system: how to ensure better coverage of the NAB activities, improve the way to determine the composition of the teams and options for evaluation, the reporting system, and the decision-making system to be able to respond quickly to NABs when they are required to develop a new scheme shortly? Revising the policy for planning witnessing activities is also a key point.
What measures to address the issues identified?
In general, the Secretariat’s support to the evaluation team has been considerably reinforced by transferring more tasks from the MAC Management Group to the Secretariat, while increasing resources and competencies in the Secretariat. Experience already shows that the change is a good one; it is very well received by the NABs and the PE teams.
“For example, the system set up by the Secretariat on the internet by which the PE teams can share information and interact about PE and reports with the Secretariat and the NAB under evaluation, has been reported to be a real improvement,” says Daniela.
More flexibility has been introduced in the planning of witnessing activities, which can be disconnected from the main evaluation week. It provides better coverage of the evaluation and makes it easier for the NAB to plan the relevant activities.
An online decision-making system has been set up. It can be used every time the peer-evaluation report does not raise critical issues. So far, it applies in 90% of the cases. The principle is that the Secretariat, with the MAC Management Group and the Task Force Group assigned to review the peer-evaluation report, come up with a recommendation submitted to the MAC members for voting through the EA intranet. There is no need to wait until the next MAC meeting: the decision can be made and published shortly after the closing of the ballot. Consequently, only critical cases remain on the MAC meeting agenda for discussions, leaving more room for other discussions e.g. policies, workshops, etc.
The risk-based approach in the peer evaluations – From in-the-field experience to a documented system
There is a risk aspect in peer evaluations. Then, the question is how the PE system should apprehend the risks in the best way for the NAB and the peer-evaluation system. How should the PE team take account of the NAB’s level of complexity, stability, maturity, experience, and capacity to adapt to new schemes and markets and for EA to be able to build the most suitable team and evaluation program? The objective is to maintain confidence in the MLA and trust in the PE system from the NAB themselves, the peer evaluators, the market, and regulators and bring added value to the NABs. The risk-based approach is the major tool that has been used. After several years of experience and analysis of feedback from PE team leaders, the approach has been discussed in detail and is now well documented. Specific training is given to team leaders and MAC members to ensure harmonization in its implementation.
“Some may argue it took a long time to complete the re-engineering of the PE system”, says Maria. “Yes, and it was necessary and good. I am happy to have participated in many discussions with the members. I am happy to see how much MAC and Management Group members were involved in the project and managed to finally have a consensus and common understanding between the members about the implementation of the risk-based approach in the peer evaluations. We wanted to allow everyone to contribute. We have managed several surveys and prepared several new documents and templates – a lot was made backstage. But now I am proud to see that all we have created is used, and more importantly, the peer evaluators and the Management Group members are satisfied. It makes the system work.” MAC members have gained ownership of the system” insists Daniela.
Continuous improvement is in accreditation DNA. “As NABs, we apply our processes and experiment with changes based on complaints and market feedback to improve and avoid dysfunctions. We test new ways and then document and implement them. This is how we have been working for this project and the reason why it is a success, thanks to the commitment of all, but also, openness and innovative spirit, the EA mark I believe in” concludes Maria.