Evaluating the Personal Transitions Service WeiHsi Hu, the Director of Logical Thinking discusses their research looking into the effectiveness and impact of the Mayday Trust's Proof of Concept for Personal Transitions Service. Logical Thinking is a consultancy supporting charities to undertake research and evaluation to better understand the needs of their communities and the impact of their work. We have worked with Mayday Trust from the very beginning in designing the theory of change and the evaluation framework. We work closely with Mayday’s in house analyst to evaluate both quantitative and qualitative data. On our end, our focus were mainly on the qualitative data interviewing people they work with, stakeholders, and reviewing case notes of 113 individuals. Our evaluation examined prioritised outcomes in 5 phases of the Theory of Change, particularly: • Individual’s initial engagement and relationships with Mayday• The degree to which there is trust and understanding between the individual and their coach• Individual’s own learning and understanding of their assets and strengths• Individual’s own development of their assets• Individuals’ utilisation/application of their assets to achieve the hard outcomes You can read our evaluation in full detail in the report – but for now I would like to share some highlights from this research. Evidence base & Key messages The Proof of Concept had set out to deliver the new Model to a minimum of 220 people, consisting of various cohorts of people including those from the existing homelessness pathway, young people from the young people’s pathway, probation referrals and care leavers. Referrals have mostly been received from the homelessness pathway, which amounted to a total of 113, as opposed to the initial 220 target. Due to the limited numbers, the volunteers were not engaged as part of the Model so the full extent of the approach has not been tested. With the limitation of numbers and quantitative data collection, we are unable to establish quantitative evidence on the outcomes of asset development. Additionally, we have not been able to model an effective staff:people ratio as originally designed—that a Coach could work with between 35 and 50 people at a time. However, the data (interviews and full review of the case notes) we collected evidence that what Mayday is doing is new and that people are welcoming and engaging with it. In terms of engagement, we have found that Seeing PTS as new approach makes better engagement People living or working with Mayday talked about wanting to try something different. People who knew about the Mayday programme felt that the programme offered a real alternative to the current homeless pathway in Oxford, and the fact that it was different was enticing to people to engage. It shows how much people desired an alternative approach in the pathway. The feedback from people who took part in the Mayday programme commented on their ability to have ample time and space to have conversations with their coaches. They felt they were in control of their support, when and how they would like to progress. It was particularly interesting how individuals often talked about having the space to explore their assets and had the choice to withdraw and rethink and not being thrusted upon when they were working with Mayday. Voluntary engagement is key In the early stages of the programme, stakeholders’ views highlighted the potential challenge of voluntary engagement and that Mayday seemed to have difficulties engaging people when support was voluntary. The assumption goes 'if support is mandatory, people will have to and therefore will engage'. Interestingly, we found statistically significant quantitative evidence for the contrary to this assumption: When coaching became a mandatory part of the accommodation service, the active engagement dropped significantly. This highlights the crucial role of voluntary engagement – and how the theory of change got this right. Clearly, when individuals have an active choice and control over their engagement with their coaches, they are more likely to trust their coaches and participate meaningfully. Conversely, when coaching is made mandatory, individuals feel far less in control of their progress and see coaching as yet another ‘box to tick’ to move out of homelessness, and therefore engage less enthusiastically, or not at all. Persistent and positive approach brings people back in Coaches’ persistence and ever-present presence was the single utmost factor in re-establishing engagement. In many occasions, coaches found innovative ways to reach out to people. Having positive conversations about things people are interested in, their hobbies, their passions were a big part of re-engagement. For example, one person started to engage, because they wanted to pursue their interest for music. As a result of this persistent and positive approach, people felt the coaching offer was sincere and people felt they were valued as individuals – not just as clients. It is important to emphasise how the two previous findings tie together. It is a fine balance between voluntary engagement &/vs. outreaching to keep people engaged. When we talk about voluntary engagement, it is not just leaving clients be and letting them fall through the crack when people don’t engage. The Mayday approach let people have a say if and when they want to engage, but at the same time, coaches persistently reach out to let the clients know that people should get in touch when they are ready. As a result, people felt the sincerity and became more genuinely engaged as time goes by. Relationships and Trust People felt coaches were non-judgemental and always accessible for them. The coaches’ behaviours was a prominent factor in building trust and forging good relationships. People who work or live with Mayday said they had a good relationship with their coaches, they trusted them and felt free to talk to them about anything The conversations were not just their typical ‘needs’. From fixing a bike to their football team, from keeping fit to their relationship with their children. Self-reliance and independence People felt comfortable asking for help and indeed they did for various reasons. But as they progressed, people started to seek less help, and not because they were not engaged, or they thought they could not be helped, but because they developed new skills and networks to overcome problems. Coaches were very successful at promoting independence. We found that coaches consistently enforced a sense of ownership, by handing over the responsibility to people. Coaches should be commended in their initiative to take on a supportive role providing information and advice as requested, and empowering people to take self-action to make things happen. Asset utilisation and hard outcomes The next stage of research will look into the relationship between asset utilisation and hard outcomes more closely. For our discussion on Mayday's Theory of Change and how asset utilisation and hard outcomes relate, please see the presentation from 28 June.