Dear Scott, I really enjoyed reading your blog summarizing the great work of the EVALSDGs insights. Very insightful! thank you.

Great post and recommendations!

Great tips for presenters and emerging evaluatos

Hi, this sounds great! Is there a way to access the equity tracer tool?

Keep up the good work !

Totally agree with creating space for emerging evaluators/researchers1 and EViE is a great place to start!

Awesome insightful tips! Way to Go Gabriel!
GO EVIE!

In reply to Tatiana.

Thanks for your question, Tatiana. We use the huddles with staff to evaluate Fellows progress towards program goals. However, Fellows do something similar to assess progress towards the goals they set for themselves. One of the key components of our program is practitioner inquiry. Fellows regularly collect data from their classrooms and meet in small groups, using a protocol, to support each other as critical friends to improve their teaching and teacher leadership.

This is great! You mention doing this with “program staff” – does that also include the fellows? Or just folks implementing the program?

In reply to Rafiqah Mustafaa.

Hi Rafiqah! Feel free to send AEA365 a blog post that shares some of those takeaways. We would love to incorporate that into one of our upcoming individual weeks.

In reply to Jane Fields.

Hi Jane! Sorry for my late reply, I just saw your comment. EvalAcademy actually just developed a template that is similar (but better) than what I used. You can find it here https://www.evalacademy.com/resources/credentialed-evaluator-competencies-template

This was a great read! I appreciate Hot-tip #3, there’s been many times where I look back at the design of one of my previous reports and re-introduce those elements into a current project! This saves ton of time. Thank you for linking to this post in your latest blog post!

In reply to Ali Holstein.

Thanks, Ali! I’m glad it resonates! My colleague and I just presented at EERS on how we’re leveraging SEL in our ECB work. If it can be helpful, I’m happy to share our slides. Feel free to email me if so: rmustafaa@casel.org.

Thanks for sharing! Love the idea of connecting evaluation to SEL

Thank you so much for sharing this experience, Barbara! I have equally experienced the power of story-telling especially in Outcome Harvesting. Storytelling helps to capture the full complexity of the outcome and the context in which it occurred as well as the challenges faced. Story-telling can help capture both the intended as well as the unintended outcomes, especially in the details of the story and probing questions. There are some outcomes that may not even have been thought of as outcomes, but during the story-telling they emerge clearly and the contribution of the change agents can be discerned. Apart from providing the context and cultural underpinnings, story-telling may also help to identify some of the actors in the context, and this can go a long way through the force-field analysis to determine the degree of contribution (influence) of the change agent to the outcome. This comes in particularly handy, when you are using Podio or other applications that may require categorisation of the degree of influence, whether as a percentage or in a Likert scale.

Hello, none of the embedded links work! Can this be repaired? Thanks.

The pace of AI is impossible to keep up with. The new Chat4 GPT can take one of your pictures of the food in your refrigerator and generate recipes for you, write SAS code, and transform a sketch on a napkin into a web page. The risk of fraud is also there because it can be used to hijack Facebook pages. However, I still focus on how it can be used “for good” and am pulling a session together with John Baek (chair of the Integrating Technology in Evaluation TIG) for the next AEA meetings. Susan Wolfe will serve as a discussant. Feel free to post or email your comments and examples of how you have used it in evaluation in preparation for the session.

So well said Donna. Thank you.

In reply to Jeremy.

Thanks again Jeremy. I appreciate the dialogue and discussion on the topic. There is a conference on the topic of AI and Evaluation coming up on March 24th that might interest you. In any case, thanks again for your candid comments concerning these AI tools. Best wishes. – David

In reply to David M Fetterman.

Thanks, David. I hope my comments are understood as “critical” in that I’ve read the post and reflected on the issues. In raising these points, I’m trusting the experts that know/have the technical background and are asking relational and evaluative questions.

In making an alternate suggestion, I think you’re misunderstanding my point. There’s no evidence that any of these platforms address the foundational concerns I referenced. With regards to accuracy, they will confidentially produce garbage data and non-existent references. If I have to verify each one, what’s the point of using them?

What is their corpus? How was it generated? How was it trained? On what data? Was that data stolen, as many of the creators themselves admit? There’s no way to know the extent of these questions. To me, this is the very opposite of personal empowerment, though I’m sure it’s (further) empowering some people.

Thinking of the environmental effects it takes to run powerful computers to generate all that text/content, it’s crypto all over again on that front.

To your point of SAS code, I’ve also seen them generate completely unusable code in a wide variety of programming languages, but I’m sure they spit out good code sometimes! It’s bound to happen, just as these mistakes. Yet, I’d just rather write it myself or go to trusted, transparent sources than have to check everything.

For what it’s worth, I don’t accept that any of them are here to stay, unless you’re referring to a very broad understanding of both “stay” and these salami “AIs.” As it stands, it’s difficult for me to envision anything but harm based on the systems and history at work. Then again, that has been historically the type of thing that stays…so maybe you’re right. But it has no place in my practice.