Designing and Preparing of Evaluation Plan Implementation - Assignment Example

Designing and Preparing to Implement an Evaluation Designing and Preparing to Implement an Evaluation Evaluation Plan The following evaluation plan is for a workplace training program that impacts learning, performance and intervention outcomes. The plan is for a follow-up evaluation regarding the training program and the learner outcomes. The training program was for 950 diverse employees with the following breakdown; 50 Upper level managers 150 Supervisors 750 Hourly workers Rationale and Purpose of the Plan Programs are implemented within organizations to achieve certain goals and objectives. It is important to evaluate the implemented programs in order to check if these have been achieved. The purpose of the plan is to determine what impact the workplace training program has achieved in terms of learning, performance and changes on the employees. The evaluation will reveal the extent to which observed changes in employee performance is due to the training program (Bogdan & Biklen, 1992). Stakeholders Stakeholders are the people that will be affected or involved in the workplace training program in the organization. The evaluation plan should take into account who the stakeholders are and what they want to know. The following are the stakeholders for this program. Stakeholders What they need to know Owners of the organization Impact on profitability, dividends, share value. Management of the organization Impact on sales, profitability Human Resource Department Employee performance, motivation, change Employees Performance Evaluation Questions These are the questions that need to be asked during the evaluation. Questions will be tailored around what impact the workplace training program has had in terms of learning, performance and change. The questions will help draw conclusions about the training program and determine any problems that may have arisen in order to address those in future training programs. Questions will be tailored to each specific objective of the program. The questions will be based on outcome objectives and should answer questions to do with changes, effects, impacts and results of the training program (Durrance & Karen, 2005). The questions for this evaluation will focus on allocation of resources such as allocation of the training budget for employees. Example, Objective of the workplace training program; The training will improve the performance of employees and increase profitability of the organization. Evaluation Questions: By what percentage has there been a change in the profitability of the organization. How many employees are still on the job 3 months after the implementation of the training? Evaluation Design The selection of an appropriate evaluation design is important. The evaluation must be credible and attribute the outcome changes to the implementation of the program. This is because other changes may be attributed to the outcome changes and not to only the training program. These could include economic and social changes, the predisposition of the employees which may lead to section bias. There are three types of evaluation designs; Randomized experiments, quasi experiments and non-experimental designs. The evaluation will use the randomized experiment. This will involve the use of a control group or a comparison group. The randomized experiment will use two groups of employees; those that participate in the training program and those that do not. The purpose of the control group is to provide an estimate of what would have happened had the training program not been implemented. The control group must have similar characteristics to those that participated in the program. The evaluation will involve collection of data before, during and after the implementation of the training program (Denzin & Lincoln, 1994). Validity of the program is also key to the credibility of the evaluation. The validity is the ability of the study to measure the true impact of the training program. Randomized experiments are the least vulnerable to threats of internal and external validity and have the highest degree of validity amongst other evaluation designs. There are normally two significant threats; extraneous events and selection bias. The following are some of the threats to validity and how they will be dealt with. Threat to Internal Validity How to minimize the Threat 1. Selection Bias. This is where the control and program participants are selected with different characteristics. Selecting a random sample matching both groups with as many similar characteristics as possible. 2. Maturation. This occurs as time passes and may produce outcomes in participants that are unrelated to the effect of the training program. The training may become tiring and boring. The program ahs to ensure that the trainers are aware of the maturation effects. They should make the training lively, diverse and interesting. The evaluation should also use the control group to account for the maturation effects Threat to External Validity Extraneous events. These are caused by economic, social and political factors that are beyond the organization. How to deal with it The events that may affect the outcomes need to be monitored. Data Collection Methods The data collection methods will be through surveys, interviews, focus group discussions and observation. This data collected from this will be both quantitative and qualitative. Quantitative methods convert data into numerical form whereas qualitative data converts data into words and images. The evaluation plan will use both methods as this will provide more conclusive and rich answers to the evaluation questions. The following methods will be used. Data Collection method Source Archival data Data from previous workplace training programs Observation Observing employees behavior after the training was implemented Surveys Questionnaires administered online or on paper with structured questionnaires to employees Individual and Focus Group Interviews Individual managers and owners of the organization. Focus groups for the 3 different levels of employees; upper level managers, supervisors and hourly workers Sampling Sampling will involve choosing a group of the employees who will provide information for the evaluation. Sampling will be done for this evaluation in order to have a fair representation of the employees, the upper level managers, supervisors and hourly workers. This is to enable comparisons to be made within all the employees and the different categories of employees. Simple random sampling and further using stratified random sampling will be used. Random sampling will be used because the entire population for the survey is known, the 950 employees. Random sampling ensures that all the employees have an equal chance of being selected. The random sampling will use stratified sampling to further ensure that the 3 groups of employees are equally represented in the sample (Russ-Eft & Preskill, 2001). Data Analysis There are different methods of analyzing the quantitative data and the qualitative data. The quantitative data will be analyzed using frequencies, percentages, mean and mode. Sophisticated statistical analysis such as multiple regressions will not be used due to the nature of the evaluation. The quantitative analysis will use content analysis. Qualitative data will be in the form of words and will need reading through, sorting, categorizing the content and interpreting the words. Content analysis will involve assigning alpha numeric symbols to phrases, topics and patterns. The following is a summary of how the data will be analyzed. Data collected Type of Data Data Analysis Method Archival data Qualitative Content Analysis Observation Qualitative Content Analysis Individual Interviews and Group Focus Groups Qualitative Content Analysis Surveys using questionnaires Quantitative Frequency, Percentage, Mean, Mode, Standard Deviation Calculation and Estimation/Forecasting return on investment The economic analysis will be base on the cost of the workplace training program and the benefits or return on investment. The figures used will be of the actual number of employees trained. This assumes 46 employees were trained. PROGRAM BENEFITS Increase in sales Average weekly sales for trained groups 12,075 Average weekly Sales for untrained groups 10,449 Increase 1,626 (15.5%) Profit contribution 2 % of sales 2% of 1626 32.5 Total profit contribution for 46 participants 46* 32.5 1,495 Annual Profit benefits 1495 * 48 weeks 71,760 Staff Retention after training 46 participants still on the job after 3 months COSTS OF THE TRAINING PROGRAM 32,984 Cost Benefit Ration (CBR) 71760 / 32,984 = 2.2: 1 Return on Investment = 71,760 – 32,984 / 32, 984 * 100 = 118% References Bogdan, R.C., & Biklen, S.K. (1992). Qualitative research for education: An introduction to theory and methods. Needham Heights, MA: Allyn and Bacon. Denzin, N. K., & Lincoln, Y.S. (1994). Handbook of qualitative research. Thousand Oaks, CA: Sage Publications. Durrance, J. C., & Karen, E. F. (2005). How Libraries and Librarians Help: A Guide to Identifying User- Centered Outcome. Chicago: American Library Association. Russ-Eft, D., & Preskill, H. (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books. Read More