Home
Introduction
Current
Issues
Projects
Publications
We
Provide
Links
- We provide...
Answers to Questions About the Evaluation Papers PHSOCSCI 700 and PHSOCSCI 701, 2002
What Evaluation Model or Approach Is Used in the Papers?
There are many different models and approaches to evaluation. For instance Goal-Free Evaluation, Utilisation Focused Evaluation, Stakeholder Evaluation, Empowerment Evaluation, Kaupapa Maori Evaluation models, and Strategic Evaluation to name a few. The evaluation papers at the University of Auckland (PHSOCSCI 700 and PHSOCSCI 701) draw on elements from all of these approaches.
The idea is that by the end of the course students will have developed an evaluation philosophy and model that suits them. A general evaluation planning model is taught which can accommodate a number of key elements from the various different evaluation approaches listed above. The model incorporates evaluation right across a programme’s lifecycle, including: developmental/formative, process, impact/outcome and summative evaluation. Students can adapt this general model by emphasising elements that reflect their overall approach to evaluation.
The papers outline some of the key issues in evaluation theory. At the same time they draw on the lecturers’ practical experience in designing and undertaking evaluations and real world evaluation experience with evaluating complex programmes. Doing real world programme evaluations in the New Zealand context can be very challenging and exciting. Health and social programmes all need to take account of the Treaty; they often use emerging community strategies that present interesting challenges for evaluation; and they involved a variety of stakeholders with different expectations and understandings of evaluation.
Our hope is that when students leave the course they will be equipped to plan and undertaken evaluations of real world social, health and community programmes. They should also have a sufficient background in evaluation concepts and models to enable them to further develop their evaluation practice as they get more practical experience in undertaking evaluations.
Why are these Courses Labeled Public Health Social Science (PHSOCSCI)?
The evaluation papers are labeled PHSOCSCI. This stands for Public Health Social Science. They draw on evaluation models that have been developed within social science over the last forty years. The papers cover programme evaluation for programmes right across the territory from public health through to broader social and community public sector programmes. Public health and health promotion over the last two decades have been experimenting with a wide range of different strategies for improving people’s health and wellbeing. During this time there has been a move from just looking at individual factors to social and cultural determinants of health. At the same time there has started to be increasing mainstream recognition of the importance and relevance of the Treaty of Waitangi to health and social issues in New Zealand. In response to this, public health and health promotion programmes have been trying to develop programmes that respect Maori and community autonomy, rights and perspectives. These methods present fascinating challenges for evaluation and some evaluation models and approaches have been developed which can respond to these new types of programmes. What we are now seeing is that within the wider social policy area, there is a growing emphasis on exactly this sort of programme - multi-strategy, community orientated and attempting to be more responsive to Maori rights. This means that evaluation models originally developed for public health programmes are now being seen as being particularly well suited for evaluating the new social and community programmes which are being developed in New Zealand across a range of social policy areas.
At the same time, it is also now being realised that good programmes should aim to achieve a range of outcomes, not just restrict themselves to particular areas such as health, education, employment, housing or justice. This is currently described as the search for “joined-up solutions”. Therefore health promotion and other social programmes are currently tending to merge, or at least be complementary, by aiming to achieve a broad range of health and other social objectives. Evaluation models need to be able to deal with programmes that have a wide range of objectives from both the health and social domains and evaluation models that can deal with this diversity of objectives and outcomes are taught in the courses.
How Much Emphasis is there on Evaluation for Maori Programmes?
The course is in mainstream evaluation, not specifically Kaupapa Maori evaluation models. However, it seeks to equip students with the ability to plan and undertake (where appropriate) the evaluation of mainstream programmes taking into account Treaty responsibilities. Helen Moewaka Barnes (Ngati Wai, Ngati Hine), Associate Director of Whariki, a Maori Research Unit working in conjunction with the Alcohol and Public Health Research Unit, is involved in lecturing students during the course. For Maori students, their assignments are often on programmes involving Maori. The course can also provide them with information on different mainstream evaluation models and issues. This may give them some useful background for involvement in the ongoing discussion amongst Maori evaluators regarding the development of Kaupapa Maori models of evaluation practice.
How Much Time is Involved in Doing the Papers?
Each of the papers involves three one-day workshops. These run from 9am till 6pm. In addition there is reading to do, online work and preparing an assignment. The intention of the online work is for students to discuss issues with other students and the lecturer - like traditional tutorials. The online work uses the World Wide Web and we are working to develop an increasingly user-friendly environment for this networking. The online work can be done at any time that suits you, it does not require all students to be connected at the same time. Students do need to have access to a computer connected to the Internet. Students in Auckland can use the computer facilities at the University.
What do the Assignments Consist of?
The assignments have been designed to enable students to grapple with a real world evaluation problem. In the assignment for PHSOCSCI 700, Planning for Programme Evaluation, the students develop a comprehensive evaluation plan which covers all aspects of evaluation objective setting, stakeholder planning, programme logic, overview of methodology, risk management, reporting etc. In the assignment for PHSOCSCI 701 Methods for Programme Evaluation, students develop detailed methods to be used in the evaluation. This includes why the methods were selected, data collection, data analysis, preparation of all evaluation forms and materials, details of how the methods results will be written up, etc. The assignment can focus on evaluating a project of the student’s choice (given that there is enough in the project to provide adequate opportunity to provide a rich learning opportunity) that can be in a health promotion, community, social services or a related area.
Who is PHSOCSCI 700 Planning for Programme Evaluation for?
Planning for Programme Evaluation is for anyone who wants to be able to develop a comprehensive evaluation plan or to be able to critically assess evaluation plans. This would include those who are, or want to be, evaluators, programme planners, programme staff in large programmes, managers, researchers, and policy analysts from Maori and mainstream programmes. At the end of the course participants should be able to develop a comprehensive evaluation plan for a project. It also should help students to be able to assess evaluation plans if they are involved in commissioning or contracting for evaluation work. People from a range of different social policy backgrounds have attended both the courses.
Who is PHSOCSCI 701 Methods for Programme Evaluation for?
Methods for Programme Evaluation is for anyone who wants to be able to further develop their skills in designing methods for programme evaluation. It is relevant for those who are, or want to be, evaluators, programme planners, programme staff in large programmes, managers, researchers, and policy analysts from Maori and mainstream programmes. At the end of the course students should have a deeper understanding of the details of evaluation methods, how to design specific evaluation forms and instruments, practical issues in implementing evaluation methods, data analysis of evaluation results, and reporting on evaluation findings. Evaluation methods such as key informant interviews, questionnaires and surveys, records and before and after data collection are covered.
Who Lectures in the Papers?
The main course lecturer is Dr Paul Duignan he is a Senior Lecturer with the Alcohol and Public Health Research Unit at the University of Auckland. He has been involved in evaluation for well over a decade, working at the Alcohol and Public Health Research Unit at the University of Auckland and consulting in evaluation methodology, design and public policy in the health and social sectors. He did his doctorate on evaluation frameworks and methodologies. He was a methodology consultant for the team from the Ministry of Social Policy that won the 2000 Australian Evaluation Society (AES) Award for Excellence for the most outstanding nomination for a Public Sector Evaluation Framework/ Methodology. One of his recent papers won the 2001 American Evaluation Association Conference President’s Prize. Helen Moewaka Barnes (Ngati Wai, Ngati Hine), is Director of Whariki, the Maori Research Unit working alongside the Alcohol and Public Health Research Unit. She works in the evaluation area and in particular on issues related to evaluation and Maori. Professor Sally Casswell is the Director of the Alcohol and Public Health Research Unit, and is nationally and internationally recognised for her innovative work in community programme evaluation, and has been involved in a wide range of evaluation projects using a variety of different evaluation methodologies.
How Much Does it Cost to do the Evaluation Papers?
The course costs in 2001 were $578 per paper ($550 for a 2 point course + $8 for building levy + $20 for resources levy) plus a $167 student services fee. Half of this fee can be refunded for anyone only enrolled for one semester.
What are some of the Qualifications the Papers Can Contribute to?
If students are wanting to just do the papers on their own, they can be undertaken as Certificates of Proficiency (COP) without having to contribute towards a particular qualification. Qualifications that the papers can contribute towards include the Postgraduate Certificate in Public Health (Evaluation), a Postgraduate Certificate in Health Sciences, a Diploma in Public Health or the Masters of Public Health. It may be possible for them to contribute towards other Social Science, Public Policy, Education or Management qualifications. Note: students should refer to the 2002 University Calendar which sets out the requirements for these qualifications. Each of the papers is equivalent to one Victoria University 15 point Masters level paper.
Why do these evaluation papers?
Evaluation is a fascinating area in which to work. Evaluators get involved in a wide range of different programmes and develop the skills to help improve and assess programmes. Evaluation skills are increasingly in demand in the health and social sectors in New Zealand. As evaluation is still an emerging art, it is important to study in an environment where the issues of dealing with real world evaluations are dealt with in addition to evaluation theory. This courses emphasises taking a broad approach to evaluation right across a programme’s lifecycle and puts emphasis on the importance of using all available qualitative and quantitative techniques available to the evaluator.
This set of questions and answers has been prepared to provide background information on the courses, it should be noted that the 2002 Calendar is the official statement of conditions regarding courses at the University of Auckland and should be checked by students wishing to undertake these or any other courses.
Contact us now for further information: Dr Paul Duignan (p.duignan@auckland.ac.nz <mailto:p.duignan@auckland.ac.nz>) (09 3737 524 or 04 4999 729)