Evaluation on beneficiary application of knowledge and skills: 2016 learning-related programming

The Evaluation on beneficiary application of knowledge and skills – 2016 learning-related programming provides important insights on the effectiveness of UNITAR’s training, including factors that enable/deter the application of knowledge/skills acquired. The evaluation is based on the results of a desk review of learning-related events, the deployment of a survey to randomly sampled participants, and interviews with 40 survey respondents who confirmed application or non-application of knowledge and skills. The report is a product of the Planning, Performance and Results Section of the UNITAR and records the section’s periodic independent assessment of outcomes of learning-related programming.

What are the evaluation’s principal findings?

  • 82 per cent of respondents confirmed application of knowledge and skills acquired through UNITAR’s learning events. The rate of application does not vary significantly across gender.
  • Opportunity to apply knowledge/skills and the importance for job success are the most cited factors influencing on application.
  • Governments and international organization (non-UN) sector respondents reported the highest rate of application among sectors.
  • Higher rates of application were reported for blended and face-to-face training, as opposed to e-Learning, at 92 per cent, 90 per cent and 79 per cent, respectively.
  • The results also found the application rate to be higher (83 per cent) for those respondents who completed an event in comparison to those who did not complete (70 per cent).
  • For those respondents who confirmed application/use of knowledge/skills from learning events, the rate of application varies from 50 to 90 per cent of their time at work.
  • Participant self-assessment of learning continues to be the primary tool to measure knowledge/skills development in half of all UNITAR learning-related events. When combined with the relatively low course completion certification rate (at 20 per cent) for learning-related events with objective assessment, the linkage between application (or performance improvement) and UNITAR training is less clear and largely assumption based.   

Duration and Behavioral Changes

Three additional considerations are considered after this analysis:

  • The evaluation points out that the application is strongly linked to the duration of the learning event, i.e. the longer event lasts, the more time for exercises, simulation games and application of learning. Shorter events, such as those lasting one day or less, contain the lowest application rate, at 67 per cent; in contrast, longer events present more than 80 per cent of application rate.
  • Moreover, respondents reported various changes in work-related performance from acquiring knowledge and skills, including:
    • Disseminating knowledge/skills in different social and professional circles e.g. between family members, community members, colleagues, supervisors, scholars, government officials and civil society organizations;
    • Developing new projects and funding proposals;
    • Assuming leadership roles;
    • Shifting mindsets following the acquisition of relevant knowledge/skills; and
    • Using different skills, knowledge and terminology specific to the training, such as enhanced teamwork, project management, communication and public speaking, time management, reporting writing, leadership, presentations, needs assessments.

“I have used the knowledge gained from the gender course to assist in the design of a training module on gender for my organization. We are now currently working on a tool for gender mainstreaming and the same information and knowledge will inform how we structure and tailor our tool”.

Recommendations

Based on the evaluation’s findings and conclusions, the report includes four recommendations to optimize the delivery and effectiveness of the UNITAR’s learning-related programming:

  • Programmes should increase the number and proportion of learning events that include objective assessment of learning.
  • Programmes should create incentives to help ensure that participants successfully complete learning events and obtain certification of completion if relevant.
  • Programmes should take concrete steps to integrate follow-up actions into knowledge and skills-based training, such as learning retention plans, communities of practice, or post-training coaching or mentoring.
  • Programmes should provide post-training letters to participants to encourage supervisor support for the application of knowledge and skills.

You can read the full report here.