Poland’s development assistance provided through the Polish MFA has been the subject of evaluation since 2012. The legal framework for the evaluation is set out in the Development Cooperation Act of 16 September 2011. Pursuant to its provisions, the objectives of development cooperation as defined in the Multiannual Development Cooperation Programme 2012-2015 (Art. 13.1.10) should become the subject of evaluation.
The MFA’s Department of Development Cooperation draws up an annual evaluation plan, which sets out the method applied to the organisation of the evaluation process and the main activities (the scope of evaluation, resource allocation, timetable). This document is later approved by an undersecretary of state for development. It is indicative and may be modified or adjusted to the changing conditions and needs.
The evaluation processes are focused on annual development cooperation programmes and projects or groups of projects. Initiatives implemented abroad and at home are analysed. Results of the evaluation are used mainly in decision-making processes, primarily to identify and programme the activities of Poland’s development cooperation in the short-term (annual) and medium-term (multiannual programme) perspectives.
Evaluation of programmes and projects implemented by Polish development cooperation is based on development evaluation standards and principles set out in DAC Quality Standards for Development Evaluation (2010), DAC Principles for Evaluation of Development Assistance (1991) and UNEG Norms and Standards for Evaluation (2016). Evaluation of Polish development cooperation also takes into account Europe Aid and European External Action Service guidelines, as set out in the document The Evaluation Policy for European Union Development Co-operation. Furthermore, it is consistent with the evaluation methodology drawn up by Evaluation Unit of the European Commission (DG DEVCO).
Evaluation is carried out ex post (after a project or a programme is completed). It is done by external companies selected through a public tender procedure, pursuant to the Public Procurement Act and the MFA’s internal laws and regulations.
Evaluation in 2015
In 2015, the evaluation of the development cooperation coordinated by the Minister of Foreign Affairs concerned selected measures conducted by the Solidarity Fund PL (FSM) and programmes and projects benefiting Moldova, completed in the period 2012-2014.
The evaluation was conducted in Poland as well as in Georgia, Tunisia, Ukraine and Moldova.
The main goal of the evaluation was to identify the factors that had the most influence on the efficiency of the Polish Development Assistance as well as to provide recommendations with respect to the measures planned for the upcoming years. The projects were analysed in terms of their appropriateness, usability, sustainability, efficiency and effectiveness.
The assessment was divided into two components. Component I covered the analysis of the initiatives conducted by the Fund under the tasks commissioned by the Minister of Foreign Affairs under the Act on Development Cooperation. Component II covered Polish Development Assistance in Moldova.
The analysis of the completed undertakings showed that:
- The assistance initiatives related to supporting democracy were usually addressed to persons who, due to their profession, would share the newly-acquired knowledge and skills further. The appropriateness of the implemented measures is confirmed first and foremost by the discernible change in the attitudes and the working style that arises from gaining the new skills and learning new patterns and models.
- Usability of the projects for the beneficiaries was also e.g. due to the high quality of the measures conducted, considering local character and the partner approach which was particularly appreciated by the beneficiaries. It is also important that due to the recent transformation experience, Poland is perceived as a more reliable country than other assistance donors.
- For the sustainability of the projects, it is of key significance to continue the project measures after the external financing is no longer provided, which requires continued organisational and financial involvement of local partners. In the case of some analysed initiatives, these conditions were not met. The annual system of financing has also an unfavourable impact on the sustainability of many projects.
- The institutional system within which the Fund realised the tasks commissioned by MFA within the territories of Georgia, Tunisia, and Ukraine facilitated the improvements in efficiency and effectiveness of Polish Development Assistance.
- For the upcoming years, it is e.g. recommended as follows:
- to introduce systemic solutions that facilitate the implementation of projects and cooperation with the project initiators, including the implementation of multi-annual projects and increasing their scale;
- to put more emphasis to the comprehensive and methodical needs analysis at the stage of preparing project concepts;
- to re-implement the supporting scheme for Polish entities in the area of obtaining financing for development projects from the sources other than the state budget of Poland by providing a part of the own contribution required by the grant donors;
- to increase the involvement of diplomatic posts in the implementation of the Polish Assistance measures, e.g. by creating dedicated positions in priority countries;
- to improve the presence of the Fund in priority countries of Polish Assistance, where the Fund’s operations will be carried out;
- to introduce a clear division of tasks between MFA and the Fund, which will allow you to avoid overlapping of measures financed from different sources;
- to improve the synergy among Polish Development Assistance projects implemented in the given country;
- to improve the relationship between the effects of the projects with the business benefits of Polish entrepreneurs.
- The implemented measures were consistent with the policy of introducing reforms by Moldova intended to bring the country closer to the European Union. The effects are most clearly visible in the projects related to the development of agriculture and entrepreneurship in rural areas. It concerns both the initiatives undertaken by the administration as well as by the Polish local governments and NGOs. The projects were mostly well-adapted to the needs of their beneficiaries, which resulted from a good knowledge of the conditions in Moldova and from the experiences of individual coordinators. The results are sustainable; which is mostly related to a high involvement of the Moldovan partners. From the viewpoint of the conducted measures, it was also important that Poland was perceived as a more reliable country than other assistance donors due to the recent transformation experience.
- Detailed analysis was conducted with respect to the results of the project that involved establishing and running an Information Center for the local authorities in Ialoveni. The evaluators assessed the operation of this facility very highly.
- The initiatives realised under the Small Grants Scheme were accurate, usable, effective and sustainable, yet consideration should be given in the future to improve their relation with the objectives of the Polish Development Policy.
- For the upcoming years, it is e.g. recommended as follows:
- as regards projects implemented by the administration, to put emphasis on granting support in drawing up acts of law or in implementing the existing law, while resigning from the projects that focus on improving the qualifications of the employee of the institution;
- to continue the business support projects in rural areas and the implementation of ecological technologies;
- to implement projects that facilitate combating corruption;
- to introduce systemic solutions that facilitate the implementation of projects, including the introduction of multi-annual financing and strengthening the cooperation with the project initiators;
- to improve the potential of Polish organisations operating in Moldova, e.g. by supporting them in the application process for external funds;
- to increase the involvement of the Polish Embassy in Kishinev in conducting Polish Assistance activities;
- to further support the operations of the Information Center for the local authorities in Ialoveni;
- to undertake actions intended to involve Polish business in development projects;
- to improve information-promotional activities concerning both the Polish Development Policy and individual projects;
- to consider ending the implementation of projects in the field of democracy support in Moldova;
- to improve the cooperation between MFA and project initiators.
Evaluation in 2014
In the year 2014, the Ministry of Foreign Affairs has conducted an evaluation of tasks of Polish development cooperation covered by the Polish Aid programme 2011 and the plans for development cooperation 2012 and 2013. The evaluation concerned specific projects financed from the funds provided by the MFA under the scholarship programme of the Specialist Eastern Studies at the University of Warsaw, and also development aid programmes for small and medium sized enterprises in Georgia and Ukraine as well as a co-production agreement between Belarusian Radio Racja and the MFA for the year 2013.
The evaluation was performed in Poland, Ukraine and Georgia.
The main aim of the review was to formulate recommendation allowing, among other things, the increase of Polish aid (i.e. to increase the influence of projects on the beneficiary’s situation), to strengthen the image of Poland as a country active in measures for sustainable development worldwide, and also to support the decision-making processes at the MFA.
Developmental initiatives implemented as part of four Polish aid components in the years 2011, 2012, 2013 were positively evaluated. The actions met the beneficiaries’ actual needs. The Polish Development Aid offer was seen as attractive, specifically tailored to attain the goals. Competent execution of project tasks positively influenced the image of Poland as a development cooperation donor.
The analysis of realised ventures shows that:
- The Scholarship Programme for Students of Specialist Eastern Studies at the University of Warsaw, managed by the Centre for East European Studies at the University of Warsaw, meets the needs and expectations of beneficiaries. Its graduates advance their scientific and job careers, thus proving the efficacy of the undertaking. The realization of the initiative contributes to the preparation of leaders of change in the participants’ home countries, and also brings mutual benefit for Polish image abroad. The programme, thanks to its long tradition and high quality (according to the students) of offered services, has become a recognised mark in participants’ home countries.
- The Georgian component of SME ought to be continued in the future. The analysed projects were properly linked with the goals of Multiannual Development Cooperation Programme for 2012-2015 and the Millennium Development Goals. Priority of help went to particularly worse off target groups and areas in deep socio-economic hardship. The most permanent effects resulted from on-site visits to Polish enterprises and agro-touristic farms accompanied by training, which furthered the competences gained thanks to the project participation. The main reason behind limiting the permanence was a lack of cooperation from local authorities, and a more difficult access to credit financing the development of newly formed enterprises.
- The Ukrainian component of SME was also highly valued by all the participating parties. The majority of organisations have been working in Ukraine for years, where they have their trusted partners. On the other hand, the problem of entrepreneurship development was not usually the main activity area for these organisations. The turning to SME stemmed from adjustment attempts by these organisations, per their statutory tasks, to adjust their actions to the priority within the framework of which it is possible to receive funding. At the same time, the projects were characterised by an accurate diagnosis of needs, while their execution by the professionalism and commitment of contractors.
- Belarusian Radio Racja (BRR) undertakes a wide range of actions to improve access to objective information for target groups. The radio is realising its aim mainly by broadcasting programmes to the territory of Belarus. The radio is complemented in its mission by a website, profiles on Facebook, Twitter, Contact, YouTube and Google+. The radio broadcasts almost exclusively in Belarusian. The scheduling and specific programming blocks are, in spite of limited funds, worked on and modified based on sociological research conducted on the territory of Poland among Belarussian expats. BRR cooperates with many local correspondents and institutions operating in Belarus as well as abroad.
Concurrently, in the process of evaluation, conventions and recommendations were formulated, the implementation of which may result in a strengthened impact of the Polish Aid Programme and even better adjustment of projects to the beneficiaries’ needs.
Evaluation in 2013
In 2013, the MFA continued its evaluation of Polish development cooperation. Projects carried out in 2010-2012 focusing on regional development, public and local administration capacity building in Ukraine and systemic global education in Poland were evaluated in terms of their effectiveness. The horizontal effect of human rights measures that were carried out as part of these initiatives was also evaluated. The projects selected for evaluation had been carried out with the participation of Ukrainian partners in the territory of the Autonomous Republic of Crimea and in the Lviv and Vinnytsia Oblasts (regional development support) and in Poland (educational projects).
The main purpose of evaluation was, on the one hand, to define the role played by selected development initiatives in enhancing the economic and social potential of partner countries and, on the other, to identify the factors that affect the implementation of Polish projects. Its results can also be used by potential applicants and other participants of projects implemented under Polish development cooperation.
The evaluation has reaffirmed Polish development cooperation’s positive image, identified elements which after minor adjustment would additionally boost this positive image and outlined a possible reorientation in the programming and implementing of development initiatives over the next years. It also identified possible changes in assistance initiatives which could effectively improve Polish development activities.
The evaluation has shown specifically that:
- Projects carried out in Ukraine
- responded to genuine needs. Their beneficiaries gained knowledge about regional development and rural areas, the development of associations, activation and self-organisation of local communities, brownfield regeneration and internal audit. They have also improved their skills in drafting sustainable development strategies and regeneration/renewal plans, project applications, as well as conducting internal audits or setting up associations.
- have blended well with measures implemented by the beneficiaries and also effectively complemented initiatives carried out by other donors. The applicants realized their planned objectives. They were able to do so because they were well prepared, had experience and tapped into the capacities of their Ukrainian partners. The implementation of projects has contributed to enhancing these capacities.
- provided a development impetus at the local level and have helped to create an image of Poland as a modern and open state.
- were diverse and were implemented on a wide scale and adjusted to the situation and the needs of the beneficiaries (teachers, students, academics). The projects achieved planned quality and quantity objectives. They have provided practical and useful knowledge that schools can apply. However, they did not bring about a structural change in the education system. It is up to the teachers to decide whether they want to introduce global education into the learning process.
- have contributed to a change of attitudes on the part of the participants of the implemented projects. However, this change is clearly visible only in the people who directly benefitted from the assistance. One-off interventions that lack mechanisms that would make the changes permanent risk quickly losing people’s interest and do not leave lasting imprints in the form of changed attitudes and motivations to act.
- have increasingly relied on modern technologies, but there is still room to tap into their full potential. E-learning trainings have proven the most popular. Thanks to such measures knowledge about global education can be developed on a wide scale.
Evaluation in 2012
In 2012 the evaluation of programmes and projects implemented by Polish development cooperation focused on two areas: SENSE (Strategic Economic Needs and Security Exercise), a training programme implemented in 2010–2011 and organised for seven countries: Afghanistan, Armenia, Azerbaijan, Georgia, Moldova, Tunisia and Ukraine, and a group of projects that in 2009-2011 supported a programme that helped to change Georgia’s state-owned child care institutions into a system of foster family care and led to the creation of innovative institutions resembling family institutions.
The main aim of the evaluation was to assess the impact of development assistance projects financed through the Polish MFA on selected elements of the situation in the country of the beneficiary (including good governance and the development of the social sector). Recommendations were formulated on the basis of collected and analysed information that enabled increasing the impact of Polish development initiatives. The evaluation results can be used by different stakeholders, including those who implement development cooperation to draw up and carry out projects and to cooperate in partner countries.
Evaluation feedback relating to examined areas has shown a positive image of Poland’s development cooperation. At the same time it helped to identify elements that after relatively little adjustment could additionally boost this positive image. In addition, a possible reorientation in planning assistance initiatives in the coming years has been outlined. In particular:
- SENSE has fulfilled its role as an effective assistance tool used to improve the quality of governance. However, due to the specific nature and the progress made by the EaP countries, the countries of this region will be excluded from the programme and will be included to a greater extent than before in initiatives carried out as part of the Eastern Partnership Academy of Public Administration.
- The model of new forms of work with children and the family in childcare institutions that was suggested in Georgia has been accepted and represents a lasting outcome of partnership cooperation. This shows Poland’s potential in this area.
- Both the SENSE programme and projects carried out in Georgia have significantly contributed to building a positive image of Poland, Poles and the Polish foreign aid programme.