Spencer Foundation: “Evidence for the Classroom” Request for Proposals

Posted on April 11, 2013 by coreeducation

The Spencer Foundation, an organization that investigates “ways in which education, broadly conceived, can be improved around the world,” is seeking research studies to inform the current push for data-based educational reforms.

Here is more from the press release:

We announce a second-round Request for Proposals (RFP) to promote research that examines the assumptions underlying these reforms by investigating whether, when, and how student performance data informs instruction in K-8 classrooms. In an educational environment in which all kinds of decisions are expected to be “driven” by data, we are eager to learn more about how K-8 teachers use student performance data for instructional decisions and how organizational and individual factors affect that use. We take a broad view of the types of student performance data that could form the basis of fundable studies, and we think that a collection of well-designed research studies can advance current conceptions of the role data can play in efforts to improve education. In our first round of funding under this call, we awarded grants to five projects; project summaries can be found here. We plan to fund up to five more projects in this round.

The Spencer Foundation has always “been dedicated to the belief that research is necessary to improvement in education. The Foundation is thus committed to supporting high-quality investigation of education through its research programs and to strengthening and renewing the educational research community through its fellowship and training programs and related activities.”

Currently, the Spencer Foundation is accepting preliminary proposals for the second round of the “Evidence for the Classroom” project. The deadline is June 11, 2013.

The current Evidence for the Classroom RFP includes information on eligibility criteria and how to apply. For remaining questions on the RFP or for other Data Use related questions, please email datause@spencer.org.

For investigators interested in data use questions that fall outside the scope of the RFP, the Foundation continues to welcome proposals within the broader focus area, Organizational Learning in Schools, School Systems, and Higher Education Institutions.

Further information about the RFP and the process to apply can be found at:

http://www.spencer.org/content.cfm/data-use-and-educational-improvement

Education Department releases Forum Guide on education data

Posted on January 23, 2013 by coreeducation

The National Center for Education Statistics, in conjunction with the Institute of Education Sciences and the U.S. Department of Education, has recently released the Forum Guide to Taking Action with Education Data. The goal of the new Forum Guide is to give educators practical ideas about how they can process data and implement concrete changes based on what they learn.

The movement to make efficient use of data has grown in recent years, with other organizations such as the Center for Education Policy Research setting out to be a “unique partnership among districts, states, foundations, and university-based researchers designed to leverage the overwhelming amount of newly available school-, teacher-, and student-level data to address previously intractable policy questions in education and improve educational outcomes for all students.”

The National Center for Education Statistics provides the following description for the Forum Guide:

The Forum Guide to Taking Action with Education Data provides stakeholders with practical information about the knowledge, skills, and abilities needed to more effectively access, interpret, and use education data to inform action. The Briefs that accompany the Introduction are written for three key education audiences: Educators, School and District Leaders, and State Program Staff. The document includes an overview of the evolving nature of data use, basic data use concepts, and a list of skills necessary for effectively using data. The Guide recommends a question-driven approach to data use, in which the following questions can help guide readers who need to use data to take action:

What do I want to know?
What data might be relevant?
How will I access relevant data?
What skills and tools do I need to analyze the data?
What do the data tell me?
What are my conclusions?
What will I do?
What effects did my actions have?
What are my next steps?

The introduction to the document itself adds, regarding the need for the Forum Guide:

In an era of diminishing public resources, the effective use of data is about being efficient: once an education organization has gone to the effort of collecting data, failing to use the information to inform instructional, administrative, and policy-related activities is uneconomical, unwise, and a waste of a valuable information resource. Conversely, appropriate action, based on the right data at the right time, can lead education organizations to greater efficiency, educators to greater effectiveness, and students to greater academic achievement.

The Appendix to the guide also provides links to helpful online resources and articles.

To access the guide, please visit the following website:

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2013801

Five Recommendations For Reporting On (Or Just Interpreting) State Test Scores

Posted on September 26, 2012 by coreeducation

Matthew DiCarlo of the Albert Shanker Institute has developed five recommendations for reporters who write stories on state test scores. But these recommendations are not only useful for reporters – educators also would benefit from understanding the nuances of these numbers that are increasingly used to judge worth.

1. Look at both scale scores and proficiency rates: Some states (including D.C.) don’t release scale scores, but most of them do. Scores and rates often move in opposite directions. If you don’t look at both, you risk misleading your readers (frankly, it’s best to rely on the scores for presenting trends). In addition, check whether any changes (in scores or rates) are disproportionately concentrated in a small number of grades or student subgroups.

2. Changes in proficiency rates, especially small changes, should not be taken at face value: In general, rate changes can tell you whether a larger proportion of tested students scored above the (often somewhat-arbitrarily-defined) proficiency cutoff in one year compared with another, but that’s a very different statement from saying that the average student improved. Because most of the data are cross-sectional, states’ annual test score results entail a lot of sampling error (differences in the students being compared), not to mention all the other issues with proficiency rates (e.g., changes in rates depend a great deal on the clustering of students around the cutoff point) and the tests themselves. As a result, it’s often difficult to know, based on rate changes, whether there was “real” improvement. If you must report on the rates, exercise caution. For instance, it’s best to regard very small changes between years (say, 1-2 percentage points, depending on sample size) as essentially flat (i.e., insufficient for conclusions about improvement). Also keep in mind that rates tend to fluctuate – up one year, down the next. Finally, once again, the scores themselves, rather than the rates, are much better for presenting trends.

3. Changes in rates or scores are not necessarily due to school improvements (and they certainly cannot be used as evidence for or against any policy or individual): Albeit imperfectly, test scores by themselves measure student performance, not school performance. Changes might be caused by any number of factors, many of which have nothing to do with schools (e.g., error, parental involvement, economic circumstances, etc.). If a given change in rates/scores is substantial in magnitude and shared across grades and student subgroups, it is plausible that some (but not all) of it was due to an increase in school effectiveness. It is almost never valid, however, to attribute a change to particular policies or individuals. Districts and elected officials will inevitably try to make these causal claims. They should be ignored, or the claims should at least be identified as pure speculation.

4. Comparing average scores/rates between schools or districts is not comparing their “performance”: Unlike the other “guidelines” above, this one is about the scores/rates themselves, rather than changes in them. If one school or district has a higher average score or proficiency rate than another, this doesn’t mean it is higher-performing, nor does a lower score/rate signal lower effectiveness. The variation in average scores or rates is largely a function of student characteristics, and schools/districts vary widely in the students they serve. These comparisons – for example, comparing a district’s results to the state average – can be useful, but be careful to frame them in terms of student and not school performance.

5. If you want get an approximate idea of schools’ relative performance, wait until states release their value-added or growth model results: Many states employ value-added or other growth models that, interpreted cautiously, provide defensible approximations of actual school effectiveness, in that they make some attempt to control for extraneous variables, such as student characteristics. When possible, it’s much better to wait for these results, which, unlike raw state testing data, are at least designed to measure school performance (relative to comparable schools in the state or district).

Featured Education Week ‘Spotlights’

Posted on September 4, 2012 by coreeducation

Education Week is again offering new “Spotlights on Education” for free. Spotlights are collections of recent articles grouped by theme to give readers an in-depth look at education issues. Featured Spotlights include:

The Education Week Spotlight on Personalized Learning – a collection of articles hand-picked by Ed Week editors for their insights on:

Digital badges as an alternative to traditional grading
How Race to the Top districts are embracing individualized instruction
Implementing competency-based learning
Replacing seat-time mandates with requirements for demonstrated competency
The promises and challenges of using digital tools to personalize learning

The Education Week Spotlight on Data-Driven Decisionmaking - a collection of articles on:

Districts using data-driven decisionmaking as part of school reforms
Putting longitudinal data to use in schools
Issues surrounding student data tied to teacher performance
Using data to prepare students for college and career opportunities
Developing an early warning data-system to identify and prevent school dropouts

The Education Week Spotlight on Math Instruction – a collection of articles on:

The potential impacts of the Common Core Standards on math instruction
Using smartphones as math learning tools
New multimedia programs tailored to math students
Distinguishing students who occasionally struggle in math from those with a genuine disability
Professional development needs of teachers preparing for common standards in math
Early math skills as a predictor of school success

The Education Week Spotlight on English Language Arts and the Common Core focuses on:

Incorporating language arts across the subject areas to meet the expectations of the common standards
Adapting to the increased focus on nonfiction
Understanding how the common core could impact popular reading instruction techniques, like prereading exercises
Efforts to build a free, online repository of text-dependent questions and tasks aligned to popular basal-readers.
Preparing English-language learners for the complex challenges of the common core

To access these free Spotlights, please visit http://www.edweek.org/ew/marketplace/products/edweek_spotlights.html

Dear Data, Please Make Yourself More Useful

Posted on June 7, 2012 by coreeducation

In a recent commentary piece for Education Week, Brad C. Phillips and Jay J. Pfieffer reflect on data and how it is (mis)used in education. “Factions are setting up camp at two extremes: one for those who believe data is the Holy Grail, and the other for those who shun it,” they write. Phillips and Pfieffer’s view of data takes the middle ground: data is only useful if people can access and use it.

“Mountains” of data exist, but “there is little that busy people can use to make good decisions…the fixed and standardized ways that data are reported often do not strike educators as relevant or useful.” To win teachers over to the belief that data can be useful in better understanding their students at the individual level, the authors offer several guidelines to help data meet what they call the “usefulness standard”:

Engage teachers and decision makers in the design of the tools used to collect data. They observe that only 28 states make longitudinal student data available to teachers, and though 40 states offer feedback to teachers based on student performance data, few ask whether the data included are what teachers want and need. Currently, the emphasis is on collecting summative test-score data, which only measures what has been learned at the end of a course of study—it does not help teachers make midcourse corrections or revisions to help students as they learn.
Create regular opportunities to huddle around the data. Only 8 states require teachers and principals to be “data literate.” Statewide longitudinal data systems should create regularly scheduled opportunities “for teachers to gather and strategize about particular students who are struggling.”
Tailor reports to your audience. Data systems need to have the capacity to create multiple types of reports that can be pulled at different points of time—each stakeholder is going to want different information. The authors believe that part of the reason some educators are skeptical about the utility of data is because the tools they have available are not the right ones for the job.
“Useful” means many things and has many audiences. Currently, most data collection and reporting is narrowly confined to what is required by NCLB. Adding other types of data, such as prior coursework and grades, writing samples, participation in tutoring programs, etc. will improve the usefulness of data to classroom teachers, who can use it to tailor their lessons.
Continuously hone validity and accuracy. The focus on “accountability” and the way data is used for this purpose is often viewed as punitive and unfair. On the other hand, statewide longitudinal data systems “have the opportunity to become highly developed instrument panels that guide teaching with a host of information about students, not just test scores.” The daily practice of using data has been acknowledged by classroom teachers to improve their effectiveness, and thus the data produced.

In the end, the authors conclude that “the time is nigh for education data systems to make themselves much more useful. Just as electronic health records and disease registries are fueling greater discoveries and personalized patient care, education data must become a necessity of teaching.”

To read the full article, please visit http://www.edweek.org/ew/articles/2012/05/23/32phillips.h31.html?qs=dear+data

Movin’ It and Improvin’ It

Posted on February 1, 2012 by coreeducation

The Center for American Progress (CAP) has released a report that analyzes how states can use the results of their teacher evaluation systems in a meaningful way. In other words, “what kinds of strategies should they adopt to increase the amount of measured effectiveness in the teacher workforce over time?”

In October 2011, the National Council on Teacher Quality reported that 25 states have adopted policies that require annual evaluations of teachers, and 23 require evaluations to consider student achievement data (you can read our summary of this report here). This amounts to two-thirds of all states changing their teacher evaluation policies just in the past three years, a “stunning amount of policy activity in an area that had remained nearly stagnant for decades.”

However, rather than interpreting teacher evaluation as a way to provide teachers with meaningful feedback and professional development opportunities, in many districts and states poor evaluations have resulted in punitive actions, such as dismissal. In many cases, evaluation results are not being tied to professional development plans unless the results are bad. In this way, teachers who receive good results are not being given opportunities to improve their practice further.

So how can this be addressed? CAP has identified two strategies: movin’ it, which treats teacher effectiveness as fixed and uses this information for selective recruitment, retention, and deselection to attract and keep teachers with higher effectiveness and remove those with lower effectiveness. States who are making personnel decisions based on new evaluations results are “movin’ it.”

In contrast, “improvin’ it” treats effectiveness as a “mutable trait that can be improved with time.” When there is discussion about providing useful feedback to teachers or using evaluation results to tailor professional development for individual teachers, these are “improvin’ it” policies.

These policy choices do not present an either/or. “Smart” school systems would combine the policies to maximize increases in teacher effectiveness, and research shows that high-improving and high-performing schools do just this.

The report urges policymakers at all levels to “move beyond the false choice at the heart of this debate” and come to consensus on the fact that the PD system as it now stands in most districts is a waste of billions of dollars annually. Replace this system with proven models and support districts in implementing them. Encourage districts to anticipate potential hurdles in rolling out the PD system, and work with them to overcome barriers.

For this to happen, districts need to perform comprehensive audits of their current PD investments and determine whether or not these investments actually help teachers to improve, regardless of their current level of effectiveness. Additionally, states implementing new evaluation policies need to “take every step possible” to ensure that the feedback given to teachers is “as valuable as teachers have been promised.”

For information on Core Education’s professional development services, see http://www.coreeducationllc.com/PD.php

To read the full report, please visit http://www.americanprogress.org/issues/2012/01/movin_it_improvin_it.html

Lightening the Load: How Community Schools Can Support Effective Teaching

Posted on January 30, 2012 by coreeducation

A new paper from the Center for American Progress takes a look at the increasingly talked about “wraparound services” for children in low-income communities, and the connections to teacher efficiency. Wraparound services generally refer to non-classroom services such as health care, family involvement programs, and food assistance. There is research on the potential benefits such services provide for students, but how teacher efficacy relates to these services has yet to be determined.

The authors of the paper interviewed faculty and staff from 14 schools around the country that integrate wraparound services, called “community schools” by education professionals. From these conversations, the authors identified four main trends:

1. Providing wraparound services at school helped reduce the health-related issues that would otherwise cost students instructional time.

2. These services help students and families stay in the community by meeting basic needs, and the resulting decrease in mobility benefits teachers by creating classroom stability.

3. Family programs, such as ESL classes, encourages parents to communicate more with teachers and empowers them to help their children with homework and generally support the work the teacher does in the classroom.

4. Enlisting the help of community partners and services providers, such as onsite health professionals, can free teachers to concentrate on instruction with fewer worries about nonacademic student needs—thus reducing their stress and burnout tendencies.

Taking these trends together, the researchers make several recommendations to schools, districts and states to help them maximize the benefits of wraparound services for teachers:

A. Creatively combine multiple funding streams and align school services with any existing commitments to provide wraparound services.

B. Incorporate teacher input when aligning instructional strategies with wraparound student services.

C. Include strategies for data collection and analysis whenever possible.

D. Explore the impact of wraparound services on teacher effectiveness to see whether there is an optimal mix of service to provide at high-poverty schools, and whether the presence of such services makes these schools more attractive to teacher candidates.

To read the full report, please visit http://www.americanprogress.org/issues/2012/01/chang_wraparound.html

Three Ways to Improve America’s Teachers

Posted on January 17, 2012 by coreeducation

Last month, Wendy Kopp of Teach for America and Dennis Van Roekel of the National Education Association discussed their thoughts on ways to improve the teaching force in the US in an article that appeared in USa Today. “As [education] leaders…we know from experience that great teachers are made, not born…Unfortunately, not all teachers are getting the high-quality preparation they need to excel…”

Applauding Secretary of Education Arne Duncan’s recent blueprint for teacher education reform, they yet have three things they would like policymakers and reformers to keep in mind as they go about the work of overhauling teacher preparation systems.

1. Use data to improve teacher prep. In Louisiana, statistics have shown that there are significant differences in student outcomes depending on where their teachers trained. The state is using a three-tiered system to assess its teacher prep programs; similar systems are being used in California and Maryland.

2. Bring new talent to the teaching profession. It is critical for all people entering the career, whether recent college grads or career changers, to receive high quality training. One viable path for quality preparation is the proposed Presidential Teaching Fellows program, supported by the federal government that would give states committed to improving their teacher prep funding to do so, as well as merit-based scholarships for teacher candidates. The scholarships would favor candidates from low-income backgrounds as well as those of color.

3. Give teachers opportunities for continuous professional development. Training should not stop once the teacher enters the classroom. Mentoring, leadership opportunities, etc. should be a part of every teacher prep program.

In short, Kopp and Van Roekel want reformers and policymakers to remember that one third of all K-12 students will, at some point, have a teacher who is in his or her first three years of teaching. In order to ensure that these students have the same instructional quality as their peers assigned to veteran teachers, “we must commit to giving [teachers] the best preparation possible.”

To read the full article, please visit http://www.usatoday.com/news/opinion/forum/story/2011-12-20/teachers-education-public-schools/52121868/1?AID=4992781&PID=4166869&SID=y2200ify8yar

Designing High Quality Evaluation Systems for High School Teachers

Posted on December 12, 2011 by coreeducation

John Tyler of the Center for American Progress released a report last month focused on the challenges of developing evaluation systems for high school teachers, and possible solutions. Tyler believes that special attention should be paid to high school educators for three reasons: 1) high school student performance lags behind performance of students with similar demographics at the elementary and middle school level; 2) the decision to drop out is made in a student’s high school years, which means that improving the average teacher quality could be one avenue for scaling back the dropout rate; and 3) high school is the “last line of defense” for preparing students to enter the workforce or college.

As we know, the only thing that everyone seems to agree on when it comes to teacher evaluation systems is that we need them. But depending on one’s “camp,” the shape, purpose, and outcomes of any evaluation system can vary widely. However, Tyler asserts that in order to accomplish anyone’s goals, whatever they may be, the information needed to design an effective evaluation system needs to come from two sources: teacher-related inputs (observations, classroom artifacts, etc.), and outputs from the teaching-learning process (student performance). This seems simple enough, but at the high school level it is complicated.

First and foremost is the issue of time: given the number of course options at most high schools, there is simply not enough time for all teachers to be effectively observed and evaluated by someone who is trained to do so with current staffing. “This not only compromises the validity and reliability of the evaluation results; it decreases the likelihood that teachers will buy into and support the evaluation system.”

Second, it is hard to use student performance data at the high school level because so many subjects are untested, and thus there is no baseline and final data to compare to determine a teacher’s contribution to learning. Additionally, at the high school level students in an 11^th grade English course may have taken different paths to get there, which can affect outcomes.

Tyler’s potential solutions to these challenges include:

1. Developing new and enhancing existing assessments that test high school teachers’ content-based pedagogical knowledge.

2. Exploring and testing the increased use of technology, such as classroom recording, as a means for generating efficiency and productivity gains in practice-based evaluation.

3. Conducting more research on the properties and use of Student Learning Objectives (SLOs), as a measure of effective teaching based on student performance.

4. Continuing to study how value-added measures can be used at the high school level.

5. Finding the best way to incorporate all available information from both teacher inputs and outputs into the ultimate evaluation of teachers.

To read the full report, please visit http://www.americanprogress.org/issues/2011/11/high_school_evaluation.html

State of the States: Teacher Evaluation & Effectiveness Policies

Posted on November 1, 2011 by coreeducation

The National Council on Teacher Quality (NCTQ) released its report State of the States: Trends and Early Lessons on Teacher Evaluation and Effectiveness Policies late last month that offers a closer look at what is being done across the nation with regard to teacher assessments. While admitting that it is too early to assess which states have been successful, the report attempts to provide a detailed picture of what is currently happening and preliminary observations of potential best practices in this area. A summary of their findings are:

a. 32 states plus DC have made some change to their state teacher evaluation policy in the last three years.

b. 24 states and DC now require an annual evaluation for all teachers, nearly double the number of states that had such requirements two years ago.

c. In 2009, only 15 states used student achievement data as part of teacher evaluations, four of which used these measures as the primary criterion. Now, 23 states require “objective evidence of student learning in the form of student growth and/or value-added models.”

d. 17 states plus DC have adopted legislation that requires student achievement or growth be a “significant” portion of teacher evaluations.

e. 18 states plus DC allow for teacher dismissal based on teacher evaluation results, though only 13 of those states have evaluation measures explicitly tied to student performance.

The “early lessons” identified by NCTQ tend to push for quick action rather than a slower pace of planning advocated by some groups. However, there is some concern expressed for avoiding evaluation systems that use a “scarlet letter” to brand teachers. Among the dozen or so recommendations are:

1. Teacher effectiveness measures don’t have to be perfect to be useful.

2. Insistence on comparability of measures for all teachers could cripple evaluation efforts.

3. The most useful initial capacity of new evaluations will be to discern the most and least effective teachers.

4. Stakeholder input is important, but bold leadership is more so.

5. State review and approval of district evaluations may not be an adequate approach to ensuring quality and rigor.

6. A scarlet letter, in the sense that parents should be privy to the evaluation scores of teachers, isn’t appropriate teacher effectiveness policy..

7. Teacher evaluation policy should reflect the purpose of helping all teachers improve, not just low-performers.

8. States need to attend to potential bias with systemic checks of their evaluation system and maintain flexibility to make adjustments as needed.

To read the full report, visit http://www.nctq.org/p/publications/docs/nctq_stateOfTheStates.pdf

The "Core" of Education

Tag Archives: data use