POLICY AND PRACTICE Making it work


Cost-benefi t analysis questions Direct Cost


How much do the materials cost, and do they duplicate other materials that need to be purchased?


How much does the training and ongoing support cost? How much time will it require?


Will there be a need for additional personnel?


Are there future hiring decisions related to staff skills that will have a fi scal impact?


What is the cost of the new technology or equipment required to implement the approach?


Benefit


What are the actual academic gains that can be expected? Are these gains practically signifi cant in addition to being “statistically signifi cant”?


What non-academic gains in attendance, discipline, parent engagement, or reduced academic disruptions can be expected?


What are the direct savings of teaching time associated with implementation?


What are the expected improvements in staff climate and morale?


What are the expected benefi ts in retaining teachers and staff?


What are the expected benefi ts from the general skill development of teachers?


What are the direct savings associated with staffi ng? What are the expected benefi ts for leadership development?


to the entire cost (direct and indirect) puts the fi delity of implementation at risk and negates any claims that the approach is evidence-based.


Analysis of the benefi ts of a specifi c evidence-based approach should always begin with a complete understanding of the expected impact on pupil learning. Potential adopters should demand more than just statistically signifi cant evidence that a strategy “works”. Educators must explore the practical signifi cance of a given approach in terms of actual gains that can be expected. For example, how many children can be expected to be reading at year level within a specifi c time period, given the high fi delity implementation of a specifi c literacy approach?


In addition, it is important to understand other benefi ts that can be expected from the adoption of a specifi c approach. Two substantial considerations are a) the overall impact on non-academic factors (like attendance and discipline) that predict academic performance, and b) the overall impact on the amount of available teaching time. Educators will be likely to note that the impact of a specifi c approach on the amount of teaching time available in a given day can be positive or negative. Some benefi ts are even less direct and cannot be assessed for a longer period of time. Increased staff retention rates due to improved job satisfaction are more diffi cult


What we know


● Evidence-based approaches are not being widely adopted in classrooms.


● A cost-benefi t analysis should be the starting point when deciding whether to adopt an evidence-based approach.


● Adopting and implementing an evidence-based approach requires faith in the evidence, and an ongoing commitment to delivering it with fi delity.


● Teachers’ enthusiasm must be maintained.


to assess than increased pupil scores on standardised tests. Nonetheless, retention of high quality teachers is essential to the success of longer term reform, and improvements in these rates as a result of implementing a specifi c practice should be explored.


A cost-benefi t analysis can be completed


by simply comparing the list of costs to the list of benefi ts. Sophisticated methods of weighting the costs and benefi ts could be constructed, but are probably not necessary. Rather, educators should qualitatively compare the two lists and determine which warrants action. When benefi ts outweigh the costs, adoption should follow. If unclear patterns emerge from the cost-benefi t analysis, the approach being considered might well be avoided.


Sustaining adoption Indirect


How much teaching time will be required from personnel and pupils?


How much time will be expended in gaining teacher and staff support?


How much prep time should teachers expect outside the classroom?


What are the hidden costs in ongoing support, adoption of pre-requisite programmes, support for struggling staff, etc?


How much political capital will be expended in changing to a new approach?


Sustaining adoption of an evidence-based approach is as important as the decision to adopt and the initial implementation. Furthermore, sustaining the adoption can be more diffi cult than choosing and getting started. Initial adoption is sometimes accompanied by the excitement of innovation and the motivational energy associated with the novelty. As the novelty wears off, educators must be cognisant of the demands of sustaining the approach. Plans for adopting an evidence-based


approach should include a plan for sustaining the approach. How will the continuation funds be generated? How will new personnel be trained and assimilated into the approach. Will there be costs associated with upgrading to newer versions of the approach? All of these questions should be addressed in the sustainability plan. It is also important to provide enough implementation time to get the expected results. Educators can be impatient, especially when the stakes are high. Some approaches take time to be implemented with the appropriate fi delity. Others have an inherent delay in the timeline for getting results (eg, a focus on GCSE results with Year 7 pupils won’t yield pure outcome results for four years). Realistic timelines should be articulated such that evidence-based programmes are given enough time to demonstrate that they are achieving the desired impact. Sustaining programmes can also be


achieved by celebrating “early wins”. Educators are more likely to sustain a given effort if they can see the immediate benefi ts of the effort. Sometimes, it is necessary to be creative in identifying early wins so that educators do not lose their excitement and commitment to a specifi c approach. If educators do lose their enthusiasm, a booster session of training or some other incentive may be necessary to reinvigorate implementation.


Whether adopting or sustaining an evidence-based approach, the key to success is constantly promoting and celebrating feasible fi delity. If the fi delity requirements of a given approach are not feasible, there is little likelihood that the approach will have the desired impact.


About the author David Andrews is dean of the Johns Hopkins University School of Education, in Baltimore, and was the founding dean of Ohio State University’s College of Education and Human Ecology. Throughout his career he has been committed to improving academic and behavioural outcomes for at-risk children and young people.


winter 2012 Better: Evidence-based Education 23


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28