Evaluation Standards B

In an ever increasing attempt ensure that my evaluation meets all industry standards I have created a second explanation of how this evaluation adheres to a different set of program evaluation standards taken from: 

The Joint Committee on Standards for Educational Evaluation, James R. Sanders, Chair (ed.): The Program evaluation Standards, 2nd edition. Sage Publication, Thousand Oaks, USA, p.23-24; 63; 81-82,125-126 (see www.wmich.edu/evalctr/jc/) 

Program Evaluation Standards

Utility Standards

Evaluators will ensure that the needs of all intended users will be met by being credible, having an appropriate timeline of one school calendar year and ensuring the that evaluation is meaningful to all stakeholders

-       * by using a Beneficiary Assessment approach the evaluator will ensure that the needs of all intended users will be met and that it addresses questions meaningful to all stakeholders involved

Feasibility Standards

Evaluators will be practical, realistic, and apply results with efficiency and sensitivity

-       * as the evaluators will be working with all intended users and will be implemented with their needs and interests in mind, the recommendations made will be such that they will meet the needs of all interested groups

-       * the evaluation will receive funding from the PVNCCDSB and as such operate with an appropriate and realistic budget, thus meeting the needs of those involved in the evaluation

-       * all intended users at various levels will participate meaningfully (providing feedback throughout the evaluation) in the evaluation and as such all interests will be represented in the reported findings

Propriety Standards

The evaluation will be conducted legally, ethically and take into account the level of sensitivity of the issue being evaluated by working closely with all stakeholders and intended users by adhering to the Canadian Charter of Rights and Freedoms, gaining consent for participation by all participants and disclosing findings diverse meaningful and sensitive manners

-       * evaluators will be working hand-in-hand with all stakeholders including The PVNCCDSB, educators, students and with The Ministry of Education. As such the evaluation will meet all legal and ethical standards of the organizing bodies involved.

-       * consent forms will be handed out and collected for all those participating in the evaluation (some required to be signed by parents)

-       * all people involved will be treated with dignity and respect through the dissemination of accurate information by trained experts

Accuracy Standards

The evaluation will reveal and convey accurate information worth merit of the program being evaluated

-       * the program contain a variety of different types of qualitative and quantitative data that maintains the context (purpose) of the evaluation

-       * through working with intended users the evaluation will ensure the information is as accurate as possible, systematically collected using experts and efficient data collection methods and will be disseminated meaningfully through a diverse set of methods designed to communicate results with a maximum understanding of all intended users

-       * by working with (having checks and balances) all levels of administration, the PVNCCDSB, parents, and students, avoidance of stakeholder biases will be ensured and as such the results will be communicated clearly and accurately

     Also Please visit the following Link to see a model of how steps in the evaluation process will take place.

Evaluation Standards A

“Principles should proactively guide the behaviors of professionals in everyday practice.” (American Evaluation Association) 

An evaluation should adhere to a set of principles or standards of practice. My evaluation of the Peterborough, Victoria, Northumberland and Clarington Catholic District School Board’s Bullying and Cyber – Bullying Prevention Plan as it relates to the treatment of the LGTBQ community adheres to the five guiding principles as set forth by the American Evaluation Association. Those five principles are as follows: Systematic Inquiry, Competence, Integrity/Honesty, Respect for People, and Responsibilities for General and Public Welfare. Find the Link to the Principles as set forth by the American Evaluation Association Here. Better Evaluation also explains evaluation principles and their importance Here.

“A. Systematic Inquiry: Evaluators conduct systematic, data-based inquiries.” (American Evaluation Association)

My evaluation has a variety of manners in which to collect data. There are qualitative and quantitative assessments. Within said types of assessments are: focus groups, expert and participatory observations, surveys, and evaluations of policy. Most importantly, throughout the evaluation, intended users participate and provide feedback in each step of the process. They are actively involved in everything from the focus groups to website creation and finally the delivery of results.

“B. Competence: Evaluators provide competent performance to stakeholders.” (American Evaluation Association)

The evaluation will be undertaken with the greatest of care over a reasonable amount of time (one school year) with regular and irregular reporting of data and delivering of feedback. Experts, as well as the evaluator, will provide support for all stakeholders involved. Also, I understand that some of the topics concerned with the evaluation are sensitive in nature and that dealing with young people with such delicate subjects that surround diversity is challenging. As such, I enlist the help of experts in the field, teachers and school staff when examining and implementing programs. 

“C. Integrity/Honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process.” (American Evaluation Association)

It is very important to conduct an evaluation with the utmost desire to improve the program and that is what this evaluation has set out to do. Involving stakeholders at every level provides me with insight otherwise unattainable into both the policy and the affect it has on the LGTBQ community. Furthermore, the evaluation has a number of manners to collect a range of data. This ensures that there will be a more balanced approach to the evaluation, which is significant when it comes to topics such as treatment of those in the LGTBQ community.

“D. Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders.” (American Evaluation Association)

The purpose of the evaluation itself is to, “attempt to foster social equity.” (American Evaluation Association) The goals of this evaluation are to disseminate information that leads to a decrease in reported cases of bullying and cyber-bullying of those in the LGTBQ community and to examine the effectiveness of policy that leads to a decrease in reported cases of bullying and cyber-bullying of those in the LGTBQ community. Understanding the sensitive nature of the subject, I will seek to involve intended users and experts in the evaluation so to create a higher level of understanding and comfort among stakeholders, as such fostering a heighted sense of dignity and self-worth among respondents, clients, program participants and other stakeholders.

“E. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.” (American Evaluation Association)

This evaluation’s foremost goal is to improve the overall welfare of the interests of the general public. Its desire is to enhance both respect for diversity and the public’s understanding of a marginalized group of young people. Specifically, being that the primary organization to which the evaluation is a being performed for (client) is the PVNCCDSB, a government organization, the evaluation has the greater good of the public in mind and will work hand-in-hand with stakeholders to be in a position to make decisions that take into consideration, “the welfare of society as a whole.” (American Evaluation Association)

Reporting The Data

Evaluation Use

Utilization

Focused Evaluations are evaluations that are designed to enhance, “the utility and actual use of evaluations.” Furthermore, evaluators must design their evaluation with, “careful consideration” of how the evaluation will be conducted from start to finish.  This evaluation’s usability was at the forefront of its creation and purpose. The goal of the evaluation is to decrease instances of bullying and cyber-bullying among those in the LGTBQ community and as such, in its conception the interests of stakeholders and the improvement of the manner in which is a heart the ability to “take action.”

Intended Use by Intended Users

This evaluation’s purpose stems from its ability to enhance the utility of the primary intended users to which the evaluation involves.  The success of the evaluation is based upon the stakeholder’s interaction with the evaluator throughout the evaluation process. It is essential for the evaluator to work with the stakeholder so that the evaluator can gain the necessary insight into the program that they require in order to assist in the improvement of the treatment of those primary intended users.

The primary intended (Schools and LGTBQ Students) are directly involved in the evaluation as participants in focus groups, surveys, and organizing many events to help inform the larger community of LGTBQ rights and issues and in doing so decrease instances of LGTBQ bullying and cyber-bullying. 

This evaluation has three main purposes (though one may argue there are others, but I have intended to identify 3).

1. Improving –

a)Desire to improve the treatment of those in the LGTBQ Community

b) To decrease (improve) the instances of bullying and cyber-bullying among those in the LGTBQ 

Community

2. Ongoing Development – To improve the understanding of LGTBQ Rights and Issues within the PVNCCDSB

3. Knowledge Building – To further develop the understanding of the community at large of issues surrounding the LGTBQ Community

Reporting The Evaluation

Primary Intended Users – Students, Teachers, Administration, Parents, Community, Board of Education

Update:

In order to improve the reporting of the data and to eliminate/minimize bias, I will have the data reporting vetting by colleagues. This vetting will ensure that the data and reports are presented in a neutral manner. Moreover, when the time comes to present the reports to the intended users, I will ensure that they are delivered with the input of stakeholders. For example when data / reports are being delivered to students, I will encourage input and feedback on the manner in which the reporting takes place from teachers and administration. This will ensure not only a lack of bias, but the best manner in which to communicate results as those who have the best understanding of the stakeholders will provide meaningful feedback and suggestions on how to best conduct the reporting process. 

Reporting Strategies

1.     Email/Pamphlet – Parents

2.     Assembly/TAP – Students

3.     Information Night – Students and Parents

4.     Website – Local Community

5.     Board Meeting – Administration and Board of Education

Questions to Consider (Better Evaluation)

1.     What do you need to include in different kinds of reports?

2.     At what point do you need to get feedback on your findings - and from whom?

3.     Will your findings be presented in draft form?

4.     Are you willing to share draft findings?

5.     Will you have any influence over the way the findings are re-presented?

Enhancing Use

The evaluation will have enhanced use through its dissemination. All intended users of the evaluation are not only addressed, but addressed by appropriate parties and in a manner that is conducive to their understanding and the evaluation’s success. Moreover, the evaluation is intended such that even though the reporting period has ended, the programs set forth in the evaluation (focus groups, TAP, the Website/YouTube Channel and so on) will be still in use and monitored well after the evaluation. Furthermore, changes in policy will take affect and drive the success of the evaluation further. With the chances in policy, those in the LGTBQ community will further be able to express themselves and actively participate in their school community. As such, this will build a positive learning community within schools leading to further change in the future. Lastly, Board policy is reviewed every two years and keeping this practice in place, but making necessary changes to existing policy, will allow for reporting to occur beyond the intended evaluation period of 1 school year (September 1^st to June 30^th).

Update:

Understanding that evaluation data must be not only presented and useful, it also must be implemented. In order ensure that the data and subsequent findings/recommendations are implemented effectively, after 3, 6, and 12 months a checklist/rubric will be completed by intended users and the client. Teachers will conduct a survey with students and participate in a survey themselves at 3 and 6 months that asks questions regarding the success of the study. After 12 months, administration will participate in different survey regarding the overall implemettion of the results. All three surveys and their results will be submitted to the PVNCCDSB to review. These surveys will provide a degree of responsibility and accountability to intneded users, thus encouraging them to implement the findings/recommendations. Moreover, the implementation of the recommendation will be evaluated every two years by the PVNCCDSB in a Process Evaluation. This is another further manner in which to ensure that the evaluation is being implemented effectively.  

Find the Below Chart Here.