In project evaluation, it is common to use techniques such as document reviews, semi-structured interviews, focus groups, and surveys. However, increasingly, development projects can leave their mark on the internet, leading to internet metrics becoming a new evaluation technique.
ECOPER regularly evaluates United Nations projects that aim to implement improvements in public policies in developing countries. In recent evaluations, we have used internet metrics to evaluate how the ideas generated by the projects leave their mark on the internet conversation. of the beneficiary country. This was the case during the evaluation of the UNCTAD project ‘Transparency in the regulation and facilitation of trade in the Pacific Agreement on Closer Economic Relations (PACER) Plus’, which aimed to improve the transparency of the trade policies of nine Pacific island microstates: Cook Islands, Kiribati, Nauru, Niue, Samoa, Solomon Islands, Tonga, Tuvalu, and Vanuatu. The project was aligned to various SDG targets, such as improving transparency in institutions (Target 16.6) and increasing exports from least developed countries (Target 17.11).
Using advanced online search tools, ECOPER measured the appearance of a series of keywords on the websites and the main online newspapers of the beneficiary countries. These keywords were related to the thematic focus of the project and included terms such as ‘tariff procedures’, ‘import licenses’ or ‘phytosanitary measures’. Measurements were made before and after national workshops and not only in beneficiary countries, but also in a control group consisting of Papua New Guinea and the Fiji Islands.
This exercise provided two very relevant pieces of evidence for the analysis of project relevance and impact. The first was that the appearance of the key terms was, in general, scarce. The second and more interesting finding was that in most beneficiary countries the number of references to these words increased after national workshops organized during the project, something that did not happen in very similar neighbouring countries which had not participated in the project.