The French Ministry of Education publishes since 2008 figures about the added value of high schools. These figures measure the intrinsic quality of the teaching staff of each high school: the objective is to eliminate from the success rate at the baccalauréat what is due to the initial level of pupils when they arrived at the high school three years before. Thus the added value of a high school measures « the difference between the expected and actual results, taking into account the socio demographic and academic characteristics of pupils ».
Laudable initiative. With more than 65 billion euros in the 2015 budget, education if the first item on the French State spending list. Giving the taxpayer the possibility to choose the high school that will maximise the ROI of his/her taxes – and of course give the best chances to his/her children – is perfectly legitimate.
The available papers on the method used to calculate this added value are rather concise. Not easy to understand precisely how they are calculated and thus to directly evaluate their quality. But there is an indirect way to assess them. If these added value indicators live to their promise, they should be correlated only with the characteristics of the teaching staff, excluding all other factors (structure of the pupils’ population, administrative organisation…)
We had a closer look.
No way we could collect data on each high school and its pupils’ population: that would be much too long and costly. In the same way as other work presented in this blog, the idea is to use data freely available on the web. We have combined these added value indicators, available for each high school, with census data at the town level. The indicator our model is explaining is the average of the 5 measures published by the Education Ministry for each high school.
Merging the data at the high school level is a messy business: there are a little bit more than 2000 high schools, and 36 000 towns. The attraction zone of a high school goes beyond the limits of its town and a town can have several high schools. All figures used have been calculated at three levels; town, canton and sous-préfecture, in order to best capture the attraction zone of high schools.
First variable explaining the added value of a high school: is it public or private? Among high schools where results are above expectations, 41% are private. Only 25% of underachieving high schools are private. The Education Ministry thus tells us that the teaching staff of private schools is on average better than in public ones… Is that for sure, though? Private schools do worse than public ones on only one indicator: the access rate to the bac of pupils who were in the same high school three years before. This points to a selection of pupils based on their background. It would be good to be sure the other measures have been cleaned from that effect.
Three other variables then have a similar impact: the percentage of immigrants in the town, the unemployment rate of the canton and the percentage of children of teachers in the canton.
Less immigrants in the towns of high schools with the highest added value: Among the high schools with results above expectations, 28% are in the first quartile in terms of immigrants’ number in the town. There are only 22% of them in the last quartile. Both figures should be equal to 25%, if the immigrant rate had no influence on the achievement of a high school.
Less unemployment in cantons of high school with highest added value, exactly in the same proportions: 28% of over achieving high schools are in the first quartile in terms of unemployment rate, 22% in the last quartile.
More children of teachers in cantons of over achieving high schools: the corresponding figures are 27% and 23%.
Our analysis shows that the added value indicators don’t fully correspond to their specifications: the « added value » seems to still depend on intrinsic characteristics of pupils or their environment, and not only on the quality of education in each high school. Is it possible to eliminate thas bias ? Without a full documentation on the calculation method – up to our knowledge -, it is difficult to answer this question.
It is a bit disturbing that using such noisy data – we don’t use any data specific to each high school – would lead to such a clear cut conclusion. Hopefully the Education Minister can come with better figures. We will anyway go on working on the subject and come back on this. Anyway, this shows again once more the power of analyses that can be done with freely available data, and with the relevant statistical methods.
Antoine Moreau
30/10/2014
Français
English