All interested candidatesSHOULD APPLYthrough newUNDP Vendor Portal (Quantum), after registration viaregistration link
Log Notes:Guide for UNDP Suppliers using the Quantum Portal
DO NOT use the "APPLY" button on this ad; Applications received outside of the Quantum system will NOT be considered.
UNDP Procurement Notices:UNDP | replenishment notices
Technical support for registration issues:Quantum.firstname.lastname@example.orgIProcurement.email@example.com,Include the deal ID in the subject line.
1.BACKGROUND AND CONTEXT OF THE PROJECT
On January 1, 2021, UNDP launched its new project "Digital, Inclusive, Accessible: Supporting the Digitization of State Services in Ukraine" (Support DIA). With a total duration of three years, the project is made possible by the support of the Swedish government and is carried out in close collaboration with the Ministry of Digital Transformation of Ukraine (MDT) and other government actors.
The Your Status on Your Smartphone initiative, launched in Ukraine in 2019, would only be successful if engaging and accessible digital education for citizenship was developed and widely implemented to close the gaps in digital knowledge and skills and bridge the digital divide between generations. and groups to save Ukrainians from different social backgrounds.
Therefore, the project is based on the extensive experience of UNDP in two areas: reform of administrative services and redesign and digitization of public administration processes with the broad participation of citizens and experts. Under the project, UNDP is helping government partners to select various groups of services to be used by citizens from vulnerable groups. The digital services of the state are bundled for daily use and transferred to a fully digital format.
The DIA support project is based on three pillars:
- Support the development of capacities of government institutions in the design and implementation of an inclusive digital transformation, taking into account the EBDH and the incorporation of the gender perspective;
- Assist in launching or redesigning a range of digital public services and service-related digital tools for war-affected and vulnerable groups of Ukrainians;
- Communication campaigns to inform Ukrainians about the available state electronic services with the promotion of online resources that help to improve the digital skills of potential users of electronic services.
In particular, the DIA Support Project has contributed to the advancement of digital literacy to close the gaps in digital knowledge and skills and bridge the digital divide between generations and groups of Ukrainians from different social backgrounds; Improve the accessibility of digital solutions by designing relevant IT tools and creating relevant regulatory tools; and promote the development of effective, accountable and inclusive public institutions capable of responding to the needs of women and men from different groups in the transformation of public services.
The all-out invasion of Russia, which began in February 2022, has hampered Ukraine's drive to reform the state, but has failed to significantly slow the development of new digital solutions and innovations. To some extent, the field of digital transformation was even stimulated by the war, as the demand for online services needed to be accelerated by security considerations, not just the convenience of online delivery. The number of Diia users increased dramatically in 2022, with a total of more than 18 million users, as new digital services were developed to meet the needs of internally displaced persons (IDPs), those seeking social support from the government. and their Unfortunately, the number continues to grow.
During the two and a half years of its implementation, the DIA Support project has launched more than 30 relevant e-social services and digital tools to respond to the needs of Ukrainians who represent, among others, vulnerable groups such as internally displaced persons and the elderly. . More than 9 million applications were submitted through newly established services, with equal access for women and men. In addition, more than 90% of the applicants expressed their high level of satisfaction with the functionality of the online services. This proves that regardless of the COVID-19 pandemic and war-related difficulties, the public services introduced with the support of the project are effective and accessible to the Ukrainian people.
Currently, UNDP plans to analyze the lessons learned from the DIA support project and consult with key government stakeholders on the strategic priorities in the field of digitization of public administration for the coming years, in order to drive transformational reforms. digital and ensure inclusion and accessibility. of government digital solutions as well as the sustainable growth of users of electronic services and their satisfaction.
The main stakeholders in the project are (including but not limited to):
- Ministry of Digital Transformation of Ukraine (MDT);
- Ministry of Social Policy of Ukraine;
- Ministry of Justice of Ukraine;
- Ministry of Health of Ukraine;
- Ministry of Veterans Affairs of Ukraine;
- Ministry of Municipalities, Territories and Infrastructure Development of Ukraine;
- Vulnerable groups (internally displaced persons, the elderly, war veterans, etc.);
- Ukrainian citizens using public electronic services
Digital, inclusive, accessible: supporting the digitization of public services in Ukraine (support DIA)
Quantum Project ID
Company Results and Performance
Contributory result (UNDAF/CPD, RPD or GPD):
UNDAF OUTCOME: 3. By 2022, women and men, girls and boys participate in decision-making and enjoy human rights, gender equality and effective, transparent and non-discriminatory public services
Indicative problem(s) with gender marking:
CPD Outcome: 2.2.1 Use of digital technologies and big data to improve public services and other government functions. GEN2
Cost Sharing Agreement Sign Date
11. November 2020
PAC Meeting Date
December 15, 2020
1 January 2021
December 31, 2023
The project budget
Project effort at the time of evaluation
3.365.651 US-Dolar (Stand 1. Enero 2023)
government of sweden
UNDP in Ukraine
Human rights-based approach
The HRBA framework is an essential part of the project's theory of change, as the digitization of public services should only be done according to the principle of "leaving no one behind" by envisioning different groups of users of these services. Essentially, HRBA considerations will be integrated into the core of the project by linking the service re-engineering process with “human rights and digitization” experiences. In particular, HRBA will be integrated into project components, incorporating human rights approaches into policy making and officer capacity building, and building new digitized/mobile-based digital services.
Cross-cutting issues of gender and sustainability are addressed in all three project outputs: capacity building of civil servants, government technical support with e-services, and awareness of existing e-services among the Ukrainian population. The project strives to ensure that women and men are equally represented in all interventions and have equal access to the electronic services produced. UNDP takes an equal opportunity approach in hiring and procurement practices and strives to employ a workforce that reflects diversity and gender balance.
Gender balance is measured by collecting (wherever possible) gender-disaggregated data at the activity and outcome level. The project launched two educational programs specifically for women, namely the leadership program for female innovators in the field of digital transformation and the internship program for women aspiring to STEM careers. While the internship program will be successfully completed in early 2023, the leadership program will continue through mid-2023.
Also within the framework of the project's communication campaign to raise awareness among citizens about electronic state services, all media materials developed are usually directed at the needs, preferences and challenges of women and men from specific target groups. Prior to broadcast, materials are pre-tested and modified to reflect the genre preferences of the target audience.
More detailed information on the background and context, rationale and theory of change, monitoring plans, specific objectives and indicators are available in the project document.. The results achieved so far since the start of the project according to the logical framework will be provided together with the project mid-term report and other project documentation.
2. PURPOSE, SCOPE AND OBJECTIVES OF THE CONTRACT
According to the article The Nature of PE is largely a management tool to provide the project team and stakeholders with a report on the results compared to the original plans, the project document and the financing agreement, make recommendations and admit further interventions from Sweden and UNDP. lead.
The main purpose of the assignment is to carry out the final evaluation of the project "Digital, Inclusive, Accessible: Support for the Digitization of Public Services in Ukraine (Support DIA)". The purpose of the evaluation is threefold and aims to (1) analyze the implementation of the project in the period 2021-2023, its relevance, effectiveness, efficiency, sustainability and coherence, especially in view of the large-scale invasion of Ukraine by of the Russian Federation; (2) extract lessons learned, including actionable suggestions on operational directions that could be further refined and improved in the follow-up phase of the project; (3) Make recommendations and inform the development of future interventions by Sweden and UNDP to take into account the ongoing war and its impact on government partners and the need for further UNDP support. This will not only support accountability, inform decision-making and allow UNDP and the project board to better manage development results, but will also pave the way for the development of the follow-up phase of the Swedish intervention. and UNDP. The scope of the PE covers the activities of the project from its inception until the end of June 2023.
Given the nature of the assessment, the International Assessor will work with the National Assessor to:
- Evaluate the progress of the project towards the achievement of the objectives, expected products and results as specified in the project document against a set of parameters established in the evaluation questions in the following sections. Compare the planned results of the project with the actual results. Evaluate the relevance of the objectives, the expected results and the established results, considering the impact of the ongoing war. Evaluate indicators of project success or failure with the goal of identifying necessary changes that need to be made to align project activities to meet the emerging needs of government partners at the end of the project. Review of the project strategy and its sustainability risks;
- review progress indicators and goals set in a logical framework in terms of their relevance and feasibility considering the country context and provide feedback on the achievement of goals;
- Build on lessons learned that can improve the sustainability of project benefits achieved before the outbreak of large-scale war and tailor UNDP's response to the needs and priorities of project beneficiaries, which are changing somewhat due to the war.
- Provide clear recommendations for adaptive management to improve the project for the remaining time of its life, guide future interventions by Sweden and UNDP and, after consultation with project beneficiaries, report on the development of the monitoring phase of the project .
This PE evaluates the performance of the project with respect to the verification criteria established in the UNDP Evaluation Guidelines, based on the OECD-DAC Evaluation Criteria:Relevance, effectiveness, efficiency, sustainability and coherence. The international consultant should also consider how the project has applied the human rights-based approach and gender mainstreaming in development efforts. The evaluation will take place between July 1 and August 31, 2023 (at home). The PD must be conducted in accordance with the guidelines, rules and procedures established by UNDP and reflected in the UNDP Evaluation Guide.
3. JUDGMENT CRITERIA AND QUESTIONS
When evaluators propose a methodology for conducting the evaluation, they should use an evaluation matrix to translate the theory of change and its established framework of direct and indirect results into a comprehensive set of measurable categories for evaluation analysis. This matrix must be designed to effectively cover the dimensions of gender equality and human rights.
Below is a set of preliminary questions for reviewers to consider in their proposed approach based on the OECD-DAC evaluation criteria. The final list of assessment questions and tools should be proposed by the International Advisor in consultation with the National Advisor and agreed with UNDP in an inception report. All evaluation questions must be gender sensitive and will be reviewed by the UNDP Gender Specialist.
- Country context: How relevant is the project to the target groups of the interventions, including the needs and priorities of the government and the current evolving country context?
- Target groups: How relevant is the project to address the needs of government partners, particularly emerging priorities in crisis situations? What kind of needs and priorities cannot be satisfied by the project (if any)? To what extent does the project address vulnerable groups (IDPs, older people and others according to the project document) and gender issues in the digitization of e-government services? To what extent do the project interventions match the needs of the partner ministries of Ukraine as the main beneficiaries of the project?
- Is the project still relevant given the changing environment given the ongoing war of the Russian Federation against Ukraine on February 24, 2022 and considering the risk/challenges mitigation strategy? Are the indicators of project goals, impact, results and outputs still relevant in the current context? What more can be done to better understand the needs of the target groups relevant to the project approach? Suggest strategies for adapting the project to the crisis situation and possible directions for expansion, focusing on war response and reconstruction.
- Does the project design take into account the results of the mid-term review (MTR)? Given the changes in the country context, including those caused by the large-scale Russian invasion of Ukraine and its impact on the operational context, is it necessary to reformulate the project design and the framework for project results? What actions can be taken to improve the relevance of the project?
- How might the project objectives be reviewed, adjusted and scaled to shape the development of the potential follow-up phase of the project and help government partners meet their changing needs in light of the ongoing war and post-war recovery?
- To what extent has UNDP adopted gender-sensitive, human rights-based and conflict-sensitive approaches?
- To what extent was the project aligned with government policies and strategies, the United Nations 2030 Agenda for Sustainable Development and the UNDP Country Program Document/United Nations Partnership Framework/United Nations Transition Framework? United? Is the project in line with the Ukrainian Recovery and Development Plan?and UNDP Recovery and Resilience Building Framework for Ukrainedeveloped after the outbreak of full-scale war?
- How does the project contribute to the SDGs, UNDP Strategic Plan, Country Program Document and UNDP Ukraine Recovery and Resilience Building Framework?
- Was there sufficient cooperation and exchange of information between the project partners? How do they correspond to each other and how do they contribute to the realization of the United Nations 2030 Agenda for Sustainable Development?
- Does the project build on or seek synergies with existing programmes, projects and strategies to maximize impact, allocate resources efficiently and avoid duplication?
- Assess overall project performance by reference to the project document/cost-sharing agreement, strategy, objectives (or lack thereof), and relevant indicators, and identify key issues and constraints affecting achievement of project objectives. project, including each component of the project, have impacted, taking into account crisis situations. Were the planned goals and results achieved in accordance with the results framework? Can the proposed goals be implemented in the current context of the country? To what extent have results at the output and output level led to results for gender equality, empowerment of women involved in Ukraine's digital transformation and promotion of sustainable practices? What results are achieved beyond the logical framework? How can the project build on or expand on the achievements, particularly with a focus on the government's response to war?
- How effective was the personalization and administration of the Low Value Grant (LVG) contests in the current context of the country? To what extent were the eight (8) grant projects supported through the three competitions effective in meeting the changing needs of the government and people of Ukraine in the context of war?
- Has the collaboration with the project beneficiaries and key project partners, including but not limited to the Ministry of Digital Transformation, the Ministry of Justice and the Ministry of Social Policy, been carried out successfully and contributed to the achievement of the objectives of the project? How could the role of partner countries in the project be strengthened?
- How were stakeholders involved in the implementation of the project? Has the project effectively collaborated with the Ukrainian Librarians Association in establishing digital centers for the general population of Ukraine? To what extent was the project successful in promoting digital skills and resources to improve them?
- How effective is the project in providing capacity building initiatives (training, educational series) for officials involved in digital transformation? What measures/interventions could be implemented to increase the impact of the project or expand its reach?
- Has the project proven to be profitable? Have resources (funds, human resources, time, expertise, etc.) been strategically allocated to achieve the expected outputs and results and address inequalities and gender issues? Was the use of the resources assigned to the GLVs efficient to achieve the expected results?
- Are the project activities aligned with the schedule established by the project team and the annual work plans? Are project disbursements and expenses in line with budgets? Do adjustments to project activities and budget revisions adequately reflect changes in operating circumstances and the program environment? Were project activities coordinated and implemented in a timely manner?
- Is the management, coordination and monitoring of the project efficient and adequate?
- Does the monitoring take into account issues of gender equality and women's empowerment? To what extent are project monitoring results and customer feedback incorporated into project design and implementation?
- Do the measures applied by the project ensure that the results of the project (if any, impacts and results) are likely to persist after the end of the project? Define the most promising areas that require further support and expansion through future interventions, taking into account the current evolution of the country context.
- Is there sufficient public/stakeholder awareness to support the long-term objectives of the project?
- What financial, social or political risks may jeopardize the sustainability of the project results? Define potential risk/challenge mitigation approaches.
- Are the grant activities or initiatives supported by the project likely to continue, scale up and replicate after project completion and increasingly contribute to gender-responsive and inclusive socio-economic development at the national and/or local level?
- To what extent do the interventions have well-designed and planned exit strategies?
- Has the project contributed to gender equality, the empowerment of women, the promotion of human rights and social inclusion? To what extent have capacity development initiatives been sufficient to ensure sustainable improvements for women, men and vulnerable groups and to promote responsible practices and HRBA-oriented policies among civil servants? What else can be done to improve the sustainability of the project?
- Given the current evolving country context, has the project contributed or is it likely to contribute to long-term social and economic improvements?
- What could be the possible priority areas of engagement and related recommendations for the remaining part of the project and further interventions by Sweden and UNDP in terms of digital transformation, taking into account the emerging needs and priorities of project beneficiaries? Findings, conclusions and recommendations should reflect gender equality, women's empowerment and social inclusion.
4. EVALUATION APPROACH AND METHOD
In cooperation with the national consultant, the international consultant should facilitate the use of various methods to ensure that data collection and analysis provide evidence-based qualitative and quantitative information based on various sources, including project reports, survey results, meetings individual etc This approach will allow the evaluation of the Allow project to not only assess causality using qualitative and quantitative means, but also provide reasons why certain results were or were not achieved and triangulate the information for greater reliability of the results. The specific mixed method approach is detailed in the initial report and presented in the final report. All data provided in the report should be disaggregated by gender and other social variables whenever possible. In addition, the evaluation methods and sampling frame must take into account the diversity of stakeholders affected by the project. Ethical standards are required throughout the evaluation and all interested parties must be treated with integrity and respect for confidentiality.
The international consultant is expected to promote the use of a participatory and consultative approach and to work closely with government officials, civil society partners, the Swedish Embassy in Ukraine (SE), UNDP Country Office (CO). and the project team to ensure planning and implementation. The evaluation will assess the extent to which the project has been successfully integrated into the UNDP Strategic Plan given the current country context.
Project performance is assessed against the expectations set out in the Project Results Framework, which provides performance and impact indicators for project implementation and appropriate means of verification. All indicators in the results framework should be assessed individually and the final results noted. An assessment of the project's design, implementation, and overall M&E quality should be conducted. The evaluation assesses the main financial aspects of the project, including the revision of the project budget. Project cost and financing data is required for the project, including annual expenses. Deviations between planned and actual expenses must be evaluated and explained. The evaluation should also take into account the aspect of value for money: the minimum purchase price (economy), but also the maximum efficiency and effectiveness of the purchase.
The conclusions on the implementation of the project from the beginning to the end of June 2023 should be comprehensive and balanced, highlighting the strengths, weaknesses, challenges and results of the project. They should be well supported by evidence and logically linked to the results of the progress assessment. They must answer key evaluation questions and provide information to identify and/or resolve key problems or issues relevant to project beneficiaries, UNDP and Sweden.
Recommendations for adaptive management to improve the project for the rest of its life and to guide future interventions by Sweden and UNDP in the area of digital transformation with a focus on war response, recovery and building resilience should identify how best practices and successes are achieved. The project may be scaled up or expanded to better respond to the changing needs and priorities of government partners and to adapt/reinforce the project's theory of change, based on interviews with project partners and beneficiaries and desktop analysis (see below). The recommendations must be conclusive, credible, practical and action-oriented, and define who is responsible for the action so that they can be used in decision-making. The international consultant is expected to help produce up to five recommendations that will address the execution of the remainder of the current phase of the project and define the development of the possible follow-up phase of the project.
The international consultant should also provide detailed information on a proposed draft and evaluation methodology (methods, approaches to be used, with particular emphasis on addressing gender issues and the inclusion of relevant SDGs, evaluation criteria to assess each proposed component ), work plan and reporting structure. to UNDP before starting the field work; These documents and the list of stakeholders to meet must be agreed with UNDP. Evaluation products must take into account gender and human rights aspects. In proposing the methodology, the international consultant should be guided by the UNDP approach to project evaluation..
The methodology is based on the following:
1. Document review of the documents listed below (including, but not limited to):
- Original project documents, progress reports, annual work plans, M&E frameworks, and financial documents;
- Project-related meeting notes (for example, board meeting minutes);
- Other project-related materials produced by the project (for example, publications, audiovisual materials, and consultancy reports, including the mid-term evaluation of the project).
2.Group interviews/discussions with key partners and stakeholders, which can be conditionally divided into the following groups:
- relevant UNDP country office representatives and project management and staff;
- representatives of the SE or Sida;
- national government institutions;
- international development actors active in the field of intervention.
3. Independent surveys are needed to determine outcome-level results among specific beneficiary groups, including Ukrainian citizens with vulnerabilities, officials involved in project capacity building activities, etc.
The aim of the interviews is to examine how the partners and stakeholders evaluate the project, what concerns and comments they have. The consultant should collect and analyze the needs and suggestions and make recommendations/visions on how future interventions by Sweden and UNDP could address them. In addition, a briefing will be held to discuss the findings, conclusions, and recommendations of the evaluation.
Since the assessment will largely be conducted virtually, attention should be paid to the availability of stakeholders and their ability or willingness to conduct remote surveys. Because the mission is remote, observation and field visits are not possible; Meanwhile, focus group discussions could be organized online. All kinds of limitations (if any) related to data collection should be considered in the final report.
5. Evaluation products (key results)
In cooperation with the national consultant, the international consultant must provide the following services:
days and hours
Delivery item n.o
The initial report is expected to be up to 10 pages with no attachments, single spaced, Myriad Pro font, and size 11, and will include, but is not limited to, the following components:
Salida: The start report is prepared with a detailed description of the methodology and scoring matrix; The commented structure of the report is developed. A set of data collection tools will be developed. All documents are sent to UNDP for final approval.
4 business days after signing the contract
Delivery item n.o
Salida:Minutes of the first results discussed in a final meeting with the project team and the UNDP Country Office.
5 business days after shipment #1 is completed
Delivery item n.o
Salida: Draft report prepared and submitted for UNDP comments. UNDP will review the draft evaluation report and provide the evaluator with a set of comments that address the required content (as agreed in the inception report) and the quality criteria set out in the UNDP Evaluation Guidelines.
10 working days
after shipment #2 is complete
Delivery item n.o
A full evaluation report (up to 40 pages with no attachments, single spaced, Myriad Pro font, size 11) is expected as the main product, including the following components, among others:
The detailed structure of the final report should be agreed with UNDP and reflect all important aspects in the approach.
Salida: Final evaluation report with all the annexes required according to section 3 of these specifications. The report must be submitted to UNDP, SE for review and approval.
8 business days
after delivery #3 is completed
Delivery item n.o
Prior to the event, consultation will take place with the contractor regarding UNDP's expectations for the presentation.
Salida: PowerPoint presentation (in PPT format) prepared and presented during the joint stakeholder meeting.
3 business days
after shipment #4 is complete
Payment will be made based on satisfactory completion of Services after review and acceptance by UNDP. 100% of the total amount is paid upon completion of Deliverables 1-5.
6. APPLICATION RULES
A team of two (2) independent consultants will conduct the assessment: one (1) international team leader (with experience and practice participating in projects and assessments in other regions of the world) and one (1) national expert who will work closely cooperation and under the direction of the international consultant.
Working closely with the National Adviser, the International Adviser will interface with UNDP project and CO staff to provide any clarification and guidance that may be required. UNDP project and CO staff provide the consultant with administrative and logistical support, as well as the necessary data and documentation. The UNDP Program Analyst will also connect the consultant with the broader program entity, senior management and key evaluation stakeholders, review and approve inception reports, including evaluation questions and methodology, review and comment on draft evaluation reports, distribute, compile and consolidate the draft and the final evaluation. report feedback Assessment reports and sent to consultant to finalize assessment report. Successful completion of each deliverable requires the approval of the UNDP Evaluation Manager.
The Consultant will inform UNDP of any issues, problems or delays that arise during the execution of the task and will take the necessary measures to correct them.
A comprehensive evaluation report (with the above parameters) is expected as the main product. The report should be as free of technical jargon as possible to ensure accessibility for a wide and diverse audience. The report must be in line with the UNDP principles of gender sensitive communication and be written in English.
All reports and results will be submitted to UNDP in electronic format (*.docx, *.xlsx, *.pptx and *.pdf or other formats accepted by UNDP).
This evaluation will be carried out in accordance with the principles established in the "Ethical Guidelines for Evaluation" of the UNEG.. The consultant must protect the rights and confidentiality of information providers, respondents and interested parties through measures to ensure compliance with legal and other relevant data collection and reporting requirements. The counselor must respect differences and give respondents equal space and dignity regardless of their gender, race, sexual preference, ethnicity, ability, or other markers of identity. The consultant must also ensure the security of information collected before and after the assessment and records to ensure anonymity and confidentiality of information sources where expected. Information and data collected in the evaluation process may also be used solely for the evaluation and not for any other purpose with the express permission of UNDP and partners.
All information and products produced by the consultant under this assignment remain the property of UNDP Ukraine.
UNEG, “Ethical Guidelines for Evaluation”, June 2008. Available athttp://www.uneval.org/search/index.jsp?q=ethical+guidelines