A large part of Senator Michael Bennet’s campaign has revolved around his efforts to improve the Denver Public Schools during his time as superintendent there. He was in charge of the schools from the summer before the 2005/2006 school year and left halfway through the 2008/2009 school year. So 3½ years running DPS. The CSAP tests are given in the spring of each year so the 2005/2006 tests occurred after Michael Bennet had been on the job for ¾ths of a school year.
It takes some time for a superintendent to get up to speed and then have their efforts move through the system to then have an effect. So his impact on the difference between 2004/2005 and 2005/2006 was probably minimal. On the flip side the grades for 2008/2009 are totally under his efforts and the 2009/2010 are partially due to his efforts, especially as Boasberg has continued in a similar vein.
So, how did he do?
The above graph (Excel file: DPS_CSAP.xlsx) is the percentage of students, across all grades tested, getting proficient or above on the CSAP scores. For comparison purposes I included the composite score for BVSD to show the validity of this measure (BVSD has done nothing to improve its results and the composite score reflects that). The Composite line is the average of the 6 tests.
First off, DPS is in terrible shape. But they have also shown steady improvement over the last decade – and that is gigantic. The composite has moved from 30% to 40% and that means 10% of the children going through the schools are now graduating proficient where before they were not. Granted, at that rate it will be another 30 years to get up to Boulder’s pathetic 70% – but 30 years is much better than never.
Now as to the impact Michael Bennet had. Looking at the composite, 2000/2001 – 2003/2004 was the best run for DPS. (Also this last year shows a very promising bump – if that is a true change in the trend line they we may have a real winner with Boasberg.) But Michael Bennet did keep it on an upward run. So by the measure of first do no harm, Michael Bennet did succeed by continuing the upward trend.
But to the bigger question, did Michael Bennet do a significantly better job than other superintendents? This is important because his efforts, his policies, his approach have been help up as a national model for the country. Before his example is repeated elsewhere, we should determine if it is worth of emulation. And the thing is, I don’t see any measurable difference in the results from his tenure and prior to his arrival.
On the flip side, this can be very valuable information – if used properly. The vast majority of scientific knowledge is things that we know are wrong. We can look at the specific changes Michael Bennet proglamated at DPS and let other districts know that they have no impact. They don’t help, and neither do they hurt. By not wasting time on initiatives similar to those Michael Bennet instituted, districts can instead try different models, where some will find great success.
What can also be of great value is to find the changes DPS instituted from 2000 – 2004 and identify the ones that set it on that upward trajectory. Those are the efforts that we want to publicize as a route to improvement in a public school system. And by focusing on what has worked at DPS, we should be able to improve those efforts to increase the rate of success.
Finally Michael Bennet should not be denigrated for not bringing about a change in improvement. First, DPS continued on its upward trend at the same rate and continuing that upward trend is major. Second, he tried. The only way we will improve our schools is to try new approaches. He tried many, and did so in a way that did not hurt the district (as a whole – some schools and kids did suffer under some projects).
ps – I asked if I could talk to Senator Bennet for 5 – 10 minutes to get his thoughts on this in case I missed something. The Bennet campaign declined to respond.
I’d say the evidence strongly suggests Bennett was nothing more than a caretaker. What’s interesting is if you correlate the test scores chart above with a chart showing DPS annual budget which has grown (dare I say it) at a faster rate than the increase in test scores which could be interpreted to mean that Bennett spent more money than his predecessors to achieve the same slightly upward trending test score results.
Bennett seems like a genuinely nice, intelligent man but to say he is some kind of public education miracle-worker would be disingenuous. He’s an experienced chief administrator and experienced chief administrators are always good at one thing: justifying their jobs and salaries with endless reports containing facts and figures that support their claims of being invaluable.
“miracle worker.” The question posed was simply did DPS improve under Sen. Bennet. The answer, based on the data provided by David, is yes.
Of course, education in this state is so poor that it’s a reasonable question to ask if it matters at all.
There’s a lot of conjecture in your (sloanslake’s) post that is presented as fact. How much did per pupil funding increase during Bennet’s tenure? (constant dollars, please).
Then, how much improvement would you expect for a given budget increase. Say the budget increased 10%. What improvement would that buy?
I don’t know the answers, but your post basically says the Bennet should have done better, while offering no rationale to support that assertion.
Exactly right SL!
To make a judgment I’d have to really dig through their School Improvement Plans, especially from 2003 to the present and see what was changed. I’m tempted to make conclusions based on the chart, but I’m feeling a little sane today, so I won’t.
It IS an interesting starting point in research and kudos to you for putting this info together.
The other thing I would add is that it is my understanding that most districts around the state have experienced a stall in their CSAP scores over the last five or six years. It’s a very frustrating stat, but education doesn’t respond very quickly to such a situation. As you said, it’s first do no harm.
I’m not sure that I would agree that a “large” part of the Bennet campaign revolves around his tenure at DPS as the diary suggests. I’ve seen him focus more on Federal issues, and K-12 education is hardly a federal issue. So, I would argue that it doesn’t really matter what his record at DPS is since the individual states and municipalities determine the success of a school district; not a U.S. Senator.
That being said, I personally have a hard time judging his performance based on CSAP scores because I don’t agree that the CSAP is an accurate measure of student learning.
I want to know what each primary candidate is going to do to address national and international issues. And so far, Bennet appears to be the only Democratic candidate knowledgeable of those issues.
Romanoff is well versed on state issues and should have run for Governor. But what is his solution for immigration reform, off-shore drilling, Iran, Afghanistan, etc.?
If you’ve got a better measure, get it to David and maybe he’ll graph it. Graduation rates are probably worth something, but are also subject to numerous external factors beyond any school district’s control (economy, influx of non-ESL students).
David asks some good questions and it’s refreshing to see a conversation that’s actually about accomplishments, not about how someone’s voice sounds or a handful of cherry picked votes (on both sides).
I don’t know. Everyone has made a big deal about the budget vs. test scores but how much of that budget was supposed to go towards the students to begin with? Was he handed a giant pile of cash and told to do whatever he wanted with it as long as CSAP scores improve?
I also think that the role of U.S. Senator is unique in that they must bridge state and national issues as well as being knowledgeable regarding international issues. Too often Senatorial campaigns become about one issue.
I do not have a better answer to quantitatively measuring learning than the CSAP. What I would argue is that we are trying to apply a quantitative measurement to qualitative data. I see CSAP as a poorly constructed measurement tool that was created for the sake of having a measurement tool, regardless of how accurate it was.
Rather than focus on arbitrary test standards, we should enforce and strengthen teacher license requirements and require teachers to continuously work on their own education. Teachers are not robots and you cannot force them all to teach the same way. At some point you have to trust that the teachers are doing their jobs correctly, so long as you provide them the proper tools.
But I do think it does a very good job of measuring differences year over year. In other words, if CSAP says 40% of the students are proficient or above, that probably means between 36 & 44% are. Very fair point.
But if the CSAP scores go from 20% to 40%, I think it is fair to say that DPS has double the number of students they are preparing adequately, or better.
What will be very interesting is if 3 years from now Boasberg continues the growth rate of this last year, which is really good. If he does that over 3 years, it’s real. And in that case they should look for the differences between Boasberg and Bennet and let other districts know of the initiatives unique to Boasberg.
Someone needs to talk to the good Senator about how DPS skipped out on their bill to Microsoft under his tenure.
Also, it said in the paper today that he put an amendment into the latest Financial Reform Bill that any money received back would go to pay down the deficit instead of reinvestment, is totally moronic. He should just join the GOP and be done with it.
Why don’t you go to the story earlier this week in the (dreaded) Denver Post that highlights Senator Bennet’s accomplishments as DPS superintendent:
As he says himself: “I’ve always said we got it to a starting line. Compare it to any other urban school district in America, there was more accomplished in Denver with less conflict. You could prove me wrong, but I doubt it.”
Compared to Romanoff constantly taking credit for single-handedly turning Colorado blue, and Michael Bennet is a truly humble guy.
That I think is the big danger. I think he did a competent job as superintendent – no better but no worse than the previous 6 years. When he’s praised and his example is duplicated elsewhere, those other districts will be disappointed to see no improvement.
I think it’s very important to put Bennet’s initiatives under the “no impact” category and try initiatives different from his.
Romanoff has never said nor insinuated any such thing. Get over it!
I would vote awful for any DPS Super since 1920.
Segregated schools (de facto that was really de jure) until 1967.
Piss poor results since 1967.
How can any superintendent be rated better than awful?
The only person who will get a better rating from me is the one who drops a nuke on the entire DPS system.
Found this article:
Some of the highlights:
Jerry Wartgow became superintendent in 2001 after three years of disaster at DPS. THe BOE selected a superintendent in 1999 who was under administrative suspension or somesuch from his last district. The man was hailed as a savior and lasted less than a year. The BOE terminated his contract because he failed to seek the BOE’s approval before okaying a major federal program. BOE reportedly paid out a quarter of million dollars remaining on the contract. There was an temporary superintendent for a year and then Wartgow was hired. When the first CSAP scores were published (and I don’t think that year counted)DPS had 21 schools listed as “failures.” Wartgow turned that around. He centralized management and standardized curriculum, the schools turned around and he was recognized by then governor Owen as having run the district showing the most improvement.
Wartgow left a hero. I think he laid a firm foundation. He was writting his book…”Why School reform is failing and what we need to do about it.”: based on his experiences and successes, when Bennet showed up and suddenly was pr’d as the savior of a bad school system….. ProComp was instituted during Wartgow’s administration. As was the program called “Revitalization” after Denver voters approved a mil levy which gave $20 million ANNUALLY…. for innovative programs. That money went into effect in 2004.
Now what exactly did Bennet do? He closed schools and reshuffled kids around…you know, that is what business people do with failing businesses…consolidate. The hidden statistic which I don’t think anyone has been able to properly examine is what happened to the kids in the failing schools or more properly the schools which he closed……..did they improve in new situations? Or, where their poor scores minimized in a new demographic mix?
Manual, as you know, closed and the 500 or so kids there were tossed out. North, as was mentioned here, also had a disasterous redesign.
I think, David, in all seriousness, you should study first and at least be able to identify what changes Bennet made. As you probably know, statistics don’t always tell the full story.
From what you say and the CSAP scores it looks like he was doing a great job improving things. Why did he leave?
As to identifying the specific changes, I prefer to start by looking at measurable results. Because that is the way to determine the impact of efforts. If you get mired in specific programs, it can all get obfuscated 12 ways from Sunday and make it say anything you want.
And best example, those CSAP scores mirror the improvement starting when Wartgow came on-board.
But I don’t know any backstory around it. I don’t even live in Denver. He may have just been ready to retire. I’ve seen pictures of him with Bennet after that, so they at least seem to get along reasonably well.
I was trying to answer your thoughtful questions.
Briefly. (You luck out)
1)Wartgow left to “spent more time with his family..grandchildren, I think.” But, he had come out of retirement to take the DPS job. He went on to write his book, work at DU and now has a top position at CU. He is an expert in urban education; but, he is getting on in years..in his 70’s I think.
2) CSAP DID NOT START UNTIL WARTGOW’S TENTURE. The baseline CSAP showed 21 failing schools in DPS and he did turn them around. But there were no CSAP scores before 2000 or 2001..
PS. SH…I say that with love…
I suggested that you should be aware of the changes which Bennet initiated. You said:
Simply stated: “What results are you measuring?”
Looking a CSAP scores in the aggregate and over time for a District can obfuscate, also. For example, BVS is a district which is stable, demographically, and basically not been subject to sweeping changes in the system. So you can look at CSAP over time and maybe come to some conclusions about the effectiveness of programs.
DPS is totally different. The population is very mobile. Kids change schools all the time. The demographics are changing dramatically. There has been a steady increase in the proportion of white middle class parents in Denver, over the last ten years. This alone could cause a rise in CSAP scores as parent’s educational level and income correlate positively with academic achievement. Plus, there have been both sweeping changes and multiple changes in programs both system wide and at individual schools
North High School, for example, has had three or four “redesigns” in the last 10 years. Lake Middle School is in the midst of its second or third “redesign.” So the overall CSAPs for DPS don’t necessary mean anything. to say nothing of all the schools which have been closed and their populations redistributed.
There are much better ways to do this – no question. But until someone does a better look, I think this look has value and is better than nothing.
There is NO independent, ongoing evaluation of reform efforts. Not in DPS, not anywhere.
Very thoughtful points and opinions in this thread. I especially like:
I really could not have said it any better. Bennett is a nice enough guy and well educated and certainly experienced but I does anyone consider the guy a masterful educator or an educator of any significance?
Oh, and I shill for no one. I plan to vote for Bennett but really what other reasonable choice do I have? Romanoff has no chance of winning. And let’s not get started on the big pile of crazy on the GOP side of the ballot…
The growth scores have been created by CDE over the past couple of years. They measure how much kids test scores go up between two grade levels. You can see them by school and subject at http://www.schoolview.org. There are also summary statistics by district. DPS summary statistics for 2007, 08 and 09 show DPS was about average (https://cedar2.cde.state.co.us/documents/Growth2009/DistrictSummary/0880.pdf). The 2007 growth is for the school year ending in 2007. In some ways this is to be expected. The districts that do the best (and worst) are small…larger numbers of students make it hard not to be average. But you clearly cannot say DPS tanked during this time nor can you say it lit the world on fire.
There is mobile population. Kids change schools all the time. I am not a statistician, but I did ace my stat course in grad school. The first rule of which was if you need to do statistical analysis, hire a statistician. The grad course only set us up to be able to ask questions about data and methodology.
Therefore, to measure change, you have to identify the base line…..if there is a year of uncertainity based on new curriculums, new staff, etc….the kids scores may go down…if you use that as a baseline, then, when things settle down, you get an improvement in scores the next year or so, because things are more stable….I don’t know if that is what happened. But my original argument with David was that you had to look at specific changes which Bennet initiated and then chart the consequences school by school, and/or student by student (if possible) to maybe get an idea of how well he may have done.
I think, quite frankly, it really isn’t fair to try and judge a superintendent in a few years. Wartgow came in with a CSAP basline showing 21 failing schools and he instituted changes and then had only one failing school. If you take the CSAP has the only measure, then I think Wartgow’s situation was unique and he can claim credit for a turn around.
is a very poor measure. My niece was staying with us for a year and attended a DPS middle-school. She had the good fortune to start the month before CSAP when the school basically stopped teaching and started horsing for the CSAP tests. CSAP (and this applies equally to SB191) drives the wrong behavior. It is a misguided attempt to measure standard results across the state when nothing else about our many independent school districts is standard.
What a month of a school year is spent teaching to a test, I think we can the consign CSAP to the trash-bin of history.
I’d prefer to see graduation rates, college acceptance rates and rates for those accepted to college who actually complete their undergraduate degree in four or five years.