This is probably the most isolated focus group I have ever done. A four hour flight from NZ to Samoa, then a 24-hour boat trip to the tiny Tokelauan atolls to undertake interviews for a Ministry of Foreign Affairs & Trade (MFAT) evaluation. Absolutely a highlight of my evaluation career, and so valuable to experience the isolation first-hand as it impacts on every aspect of Tokelauan life. Mamalu o na Taupelega e Tolu, thank you for your wonderfully warm hospitality and for sharing your views and experiences.
New Zealand introduced the Recognised Seasonal Employer (RSE) policy in 2007. The RSE scheme has proven popular, with some 8,000 Pacific Islanders working in New Zealand under the scheme in 2009. Findings from the evaluation of the implementation of the RSE policy were presented at the Making Pacific Migration Work conference hosted by the Development Policy Centre, ANU on 3 April 2012. Presentations can be downloaded here.
Every now and then you come across people who are natural story tellers. Here’s a person (Hans Rosling) who can tell amazing ‘statistics’ stories with graphics: http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html.
During a recent AEA discussion people wrote in with their top ‘must read’ books for evaluators. Here is the list, for your information. My personal favourite is Jane Davidson’s ‘Evaluation methodology basics’ – it’s the only evaluation text I’ve read cover to cover. Needless to say it’s well thumbed. Let me know if you have books you’d like to add to the list.
Alkin, M. C. (2004). Evaluation Roots: Tracing Theorists’ Views and Influences (1st ed.). Sage Publications, Inc.
Davidson, Jane. ‘Evaluation Methodology Basics: The nuts and bolts of sound evaluation” (Sage Publications, 2004). It’s simple, practical and user friendly and addresses real practical issues in the field.
Fairweather and Tornatzky. Experimental Methods for Social Policy Research
Fetterman, David. Empowerment Evaluation Principles in Practice. (For some communities and initiatives, empowerment evaluation is a powerful model.) http://www.davidfetterman.com/
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2003). Program Evaluation: Alternative Approaches and Practical Guidelines (3rd ed.). Allyn & Bacon.
Frechtling, J. A. (2007). Logic Modeling Methods in Program Evaluation. Jossey-Bass.
House, Ernie. “Professional Evaluation.”
House, Ernie and Ken Howe. Values in Evaluation. (deals with issues, ethics, values, role, etc.)
Glesne, Corrine and Peshkin, Alan. (1992). Becoming Qualitative Researchers: An Introduction. (White Plains, NY: Longman). A terrific starting place for researchers and writers gathering, analyzing, and reporting about qualitative data.
Greene, Jennifer. Mixed Methods in Social Inquiry
Issel, Michele. Health program planning and evaluation. A good, although dense, reference book for public health and nursing program evaluators.
Kelly, James Becoming Ecological: An Expedition into Community Psychology
Light, Dick, Judy Singer, and John Willett. By Design. Not an evaluation book per se, but I consult it a lot when designing evaluations.
Mathison, S (ed.) 2005, Encyclopedia of evaluation, Sage, Thousand Oaks, Calif.
McDavid, James C. and Hawthorn, Laura R.L. (2006). Program Evaluation and Performance Measurement: An Introduction to Practice. (Thousand Oaks, CA: SAGE Publications, Inc.). Theory and practical application about evaluating programs and measuring performance in public and non-profit organizations. An introductory-level text that covers quantitative and qualitative measurement and focuses on using findings for evidence-based decision-making.
Miller, M. D., Linn, R. L., & Gronlund, N. E. (2008). Measurement and Assessment in Teaching (10th ed.). Prentice Hall.
Miranda, Roger. Eva the Evaluator (just for fun)
Morra Imas, Linda G. and Ray Rist. The Road to Results – Designing and Conducting Effective Development Evaluation
Morris, Lynn Lyons et al. (1987). How to Communicate Evaluation Findings. (Thousand Oaks, CA: SAGE Publications, Inc.). One of only a few books dedicated to this aspect of research and evaluation. Practical, step-by-step guidelines for reporting findings of evaluation studies. The book is easy to follow, practical, and non-technical in its language.
Owen, John M. (2007). Program Evaluation: Forms and Approaches. Third Edition. (New York: The Guilford Press). Very useful introductory book about definitions of and approaches to evaluation that elevates the discussion of “good practices.” Discusses frameworks for different forms of inquiry and how to move from evaluation to application.
Patton, Michael Quinn. (1997). Utilization-Focused Evaluation: The New Century Text, Third Edition. The author is reputed to be the foremost expert in the realm of applied research. This book teaches how and why to conduct program evaluation, reviews relevant literature, and provides case studies to elucidate the discussions. He has a book on Development evaluation as well.
Preskill, H., & Russ-Eft, D. (2004). Building Evaluation Capacity: 72 Activities for Teaching and Training (1st ed.). Sage Publications, Inc.
Preskill, H., & Torres, R. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage.
Rossi, PH, Lipsey, MW & Freeman, HE 2004, Evaluation: a systematic approach, 7th edn, Sage, Thousand Oaks, CA.
Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. 2nd Ed. New York: Perseus.
Schwandt, Tom. – anything by him.
Seidman, I. (2006). Interviewing As Qualitative Research: A Guide for Researchers in Education And the Social Sciences (3rd ed.). Teachers College Press.
Sylvan, Donald A. and Amanda K. Metskas. (2009). “Trade-offs in Measuring Identities: A Comparison of Five Approaches,” in Measuring Identity: A Guide for Social Scientists. London: Cambridge University Press. (pp. 72-112). This chapter focuses on Israeli and Palestinian identities and analyzes five previous research projects, each of which used a different methodological approach to measure identities: experimental approaches, surveys, interviews, narrative analysis, and text-based data. It weighs the pros and cons of these alternative approaches, and argues that many real world identity-related puzzles require a multi-method approach.
Torres, Rosalie T. et al. (2005). Evaluation Strategies for Communicating and Reporting: Enhancing Learning in Organizations. Texts about reporting evaluation findings are hard to come by and this one is a helpful and comprehensive resource. It contains helpful real-life cases and is an engaging read. The authors focus both on traditional and creative forms of communicating evaluation findings.
Viola & McMahon. Consulting and Evaluation with Non-Profits and Community Based Organizations
Weiss, Carol. Evaluation.
Wholey, Hatry and Newcomer. Handbook of Practical Program Evaluation.
Wiggin, Grant. “Assessing Student Performance.” For higher education.
Wolcott, Harry F. (2001). Writing Up Qualitative Research. Second Edition. Excellent handbook for all levels of students and professionals writing research and evaluation reports. Contains practical tips, examples, and stories – very readable.