21 Apr Results of the final piloting phase
Phase 4 of the Public Pilot Implementation has been carried out between September 2021 and February 2022. As described in the General Agreement, it has aimed at piloting the universal MICADO solution with end-users in each of the partner cities/regions, to validate the applicability, and add value and impact of the MICADO solution. It has differed from the usability test (Phase 3), in which the MICADO solution had been tested with a selected number of participants and in a controlled environment (one-by-one test), because the target groups (public administrations, civil society organizations, migrations) have used the MICADO solution in remote, in an uncontrolled environment. This allowed us see how users interacted with MICADO without any guidance. To collect their feedback we have asked them to fill an online questionnaire. The results from the evaluative research have been used for a second development iteration of the technical components. UNIBO as the task leader of Pilot Monitoring and Evaluation carried out the management and the evaluative research and monitoring of the local pilots.
The official release of the newest MICADO solution took place on 10th of November 2021. Afterwards, local groups applied the new release to the local environments. The new release also included a link to an external survey tool, which users were asked to use to complete an online questionnaire. In order to obtain a quantitatively higher set of feedback than in the previous phase (Phase 3: Usability Test), during this fourth phase three online questionnaires were designed for the 3 target groups, in the local languages and in the selected migrants’ languages per city, to submit before the end of the public pilot implementation. Participants were asked to navigate around individually in the corresponding interfaces and then fill a short survey including empirical data about themselves, their impression of the usefulness of the application as well as ideas for improvement.
Every piloting city has tested the application in their local language (Spanish, German, Dutch and Italian), English and two other languages selected according to the local context in each city. Antwerp tested it in Arabic and Spanish; Bologna chose Arabic and Urdu; Hamburg chose Arabic and Dari; Madrid chose Arabic and Chinese. The application of Micado for every target group was already translated in the languages reported above, however for the survey, after the WP5 team leaders decided the content of the surveys, every pilot city translated it from English to the local language and the other two selected languages.
The public pilot phase was slowed down by the COVID-pandemic and associated measures, which made it more difficult to reach migrants, in particular illiterate migrants and recently arrived migrants with lower local language skills. Despite these hurdles, 402 responses to the online questionnaire were collected by the end of Public Piloting Phase, thereby reaching the agreed number of target groups for the pilot cities. The charts below provide an overview of the number of test users per pilot city and target group.
Chart 1 – Questionnaires collected per target group