Why Measuring Learning Outcomes Does Not Improve Accountability in Education – Or Outcomes

In the last few years, the clamour for measuring learning outcomes and using that as a means to ensure accountability has grown louder. In fact the current Five Year Plan insists that learning outcomes be measurable and be measured. Corporate houses funding various foundations and NGOs are big on learning assessment and look to it as a means of bringing about improvement. Many sensible people are voicing views to the effect that if a teacher is unable to generate learning outcomes, he should be shoved aside and replaced by someone better. And, of course, the feeling persists that we are not measuring the quality of learning enough.

This is unfortunate. Not because measuring outcomes is not important or somehow wrong but because the present formulations of the issue are simplistic to the extent that they prevent underlying issues to be addressed. Here is how.

First, it is not as if the quality of learning is not being measured, or has not been measured in the last 20 years. The first all-India survey of learning levels was conducted by the NCERT in 1995, and there have been many since. Several large-scale independent studies of students’ learning levels have been run, including ASER and surveys of Education Initiatives. Small-scale learning assessments have been conducted for innumerable research studies (e.g. of 1 lakh children in Tamil Nadu to assess the state’s Activity Based Learning Programme) or pilot projects (for instance, several states have piloted their textbooks and used learning achievement as a benchmark). And of course at least hundreds (if not thousands) of NGOs/NGO-run programmes (often in government schools) have incorporated assessment as an effectiveness measure.

There are thus any number of assessments available – and they\’ve been telling us for the last twenty years that our children are not learning. Only, this doesn’t seem to have resulted in improved learning, thus questioning the assumption behind the clamour for measurement.

This is a little like weighing a child to assess the level of nutrition – unfortunately, merely weighing the child will not lead to better nutrition… Something else is clearly required, and that doesn’t seem to be happening.

Second, insisting on having \’measurable\’ outcomes is hugely misleading – just because you can measure something doesn\’t make it more worthwhile (e.g. we do want students to be creative or considerate or civic though there are no easy measures for these). Several of the assessments mentioned suffer from this. Thus an Adivasi child who displays great resourcefulness, knowledge of the environment and concern for others would be called poorly educated since the ‘tests’ measure only basic literacy and numeracy.

Measuring outcomes would be useful only when we measure what matters most to us. Not whether a child can read something aloud but whether he can form an opinion on it and give the reasons behind them. Not whether a child can do calculations but whether she can apply it in real world contexts to solve problems or take a decision. Some of these may be hard to measure, but it would be useful to remember that it is not the purpose of education to be assessable, but the purpose of assessment to measure what is considered most worth learning.

Third, measuring outcomes does not account for contexts and tends to disadvantage (and label) those facing adverse conditions. Which then makes it even more difficult for them to improve. There are many teachers who work very hard in difficult conditions – but don\’t attain the kind of outcomes expected because the curriculum assumes children will be able to attend daily or speak the school language at home (and several other such notions), which don\’t apply to the children they work with (some 60-70% in India). We\’ll end up shoving these teachers out if we take the advice to replace them – instead of overhauling the system which has designed itself in such a way that marginalized children WILL fail.

Fourth, there is a danger that the present focus on outcomes is actually obfuscating – instead of increasing – accountability. India\’s challenges now arise from its success in rapidly expanding the school system to bring in so many children. The consequence is that we now have students (at all levels) who traditionally never attended schools – working children, migrant groups, girls from various communities, children with disabilities, socially excluded communities…. the list is endless. What this means is that while the nature of our students has changed, the curriculum, pedagogy and assessment remain as they used to be and so, the DESIGN ITSELF leaves these learners out.

At a second level, when it comes to implementation, there is a tendency in those responsible to ignore laxity on the assumed ground that it is only happening to those who do not matter. (Just as it is easier to ask a poor person to push a stalled car rather than a well-dressed one, similar prejudices operation in all facets of our society, including government officials.) Even now, therefore, it is mainly those from better-resourced families who continue to succeed, and we continue to have poor education for the poor. So the accountability really needs to be demanded at the level of the system (NCERT, MHRD, Departments of Education) and state / district / block officials.

As long as people keep pointing fingers at teachers as the main villains, the really responsible will continue to escape accountability. For instance, when the NCERT\’s own national survey shows low levels of learning, why does nothing happen to anyone at any level, including the NCERT itself (whose curriculum has been taken by many states now performing poorly)? How come officials at various levels continue exactly as they have been for decades with impunity when every measure brings out dismal levels of learning in their watch? Recently, when our group, IgnusERG assessed class 9 students in a district we found 68% of them to be at class 4-6 levels, 7% below class 3 level, and only 4% at the class 9 level where they were expected to be. When this finding is shared, everyone finds a way to blame some one else!

Finally, let me leave you with this – in the current form, knowledge of outcomes attained does not help bring about improvement. Most states will be implementing SLAS (State Learning Assessment Survey) in the coming months. But once a state finds out it is performing poorly, say, in mathematics, that will not inform it of the reasons why this is so. It could be the poor curriculum (e.g. overambitious expectations) or weak syllabus (less time allocated than required), or inappropriate pedagogy (no use of concrete materials at an early age) or bad textbooks (poorly sequenced or giving discrete rather than contextual examples) or demotivated teachers or insufficient teaching time (because the state continues using teachers for non-teaching tasks even after RTE and court orders to this effect) or home vs school language issues or at least 10 other problems that can be named, each of which can seriously lead to poor outcomes. So where will the improvement begin?

The point, as mentioned earlier, is: do ask for outcomes, but don\’t keep it simplistic, or we\’ll continue to get the poor outcomes we\’ve been documenting over the last 20 years.

Have You Been Un-Hindu Today?

Once in a while I recall that I am born a Hindu. This is usually around times when a whole lot of people are suddenly finding the need to defend Hinduism.

1. This is a little ironic. Why do you need to protect that which cannot be destroyed? Can the words or images of another person kill or harm your religion? To those who believe in God/s: even if all the people who believe in God should cease to exist will God/s cease to exist? Similarly, does Hinduism need the acceptance and support of all those being fought against in order to exist and flourish? It seems very reductionist and belittles Hinduism for anyone to say that the religion needs protection.

2. This business of religious sentiments being hurt is even more ridiculous. Why are Hindu religious sentiments hurt only by words and images but not by un-Hindu actions such as rape, murder and the racism being practiced against people from the NE in Delhi, or the displacement of Muslims in Muzaffarnagar or a thousand such atrocious acts? We are a religion that believes in the whole universe being a family, isn’t it? Why are we not religiously wounded by such major offences that hurt millions of the universal family but hugely traumatized by minor pinpricks such as a book that will be read by a few thousand people?

3. Being the transcendent religion that believes animals and trees and various forms, animate and inanimate, have the element of the Divine running through them and are therefore nothing but mere manifestations of the Unified One, how can we even distinguish between ‘ourselves’ and ‘others’? Surely the distinction is impossible and the very idea of ‘not tolerating’ someone or some view would be inadmissible – for even the so-called offender is nothing but another manifestation of the same ONE divine. So the idea of ‘getting upset’ so militantly at someone’s view is, in my view, very un-Hindu.

4. In an ecological worldview that goes well beyond the physical world, the notion is that every component have a just and fair place, the justness and fairness of which is determined by the degree to which it links with others and desists from eating into others’ space and resources. Which is the idea behind being ‘content’ – to occupy that which fulfills your need without competing with another’s, thus maintaining the ecosystem. Wanting more than this justifiable space and resource takes you into the realm of that which does not (because it should not) exist – maya. And we are taught not to want more than our remit for this reason. This is a key principle by which the universe maintains its balance, and disturbances take place when this balance is upset. Every time we seek to dominate or attribute to ourselves the right to determine others’ activities in their spheres (such as what they may think or write), we are guilty of going beyond that which is justly ours – and again, being very un-Hindu!

5. And finally, like all great religions, Hinduism too believes that real victory is one that is over oneself. No matter how much you ‘defeat’ your enemies, if you are unable to overcome yourself, that is, your own limitations and the un-divine aspects of yourself, you cannot be considered a victor. So if anyone is claiming victory at having ‘vanquished’ something offensive, do desist, for you have not won.

What does ‘Education For Freedom’ mean to You?

Usually, it seems to mean: to become free from want. In the sense of being able to stand on one’s own feet, by being able to earn a livelihood or having a job (much more the last, in our case). But what education seems to be doing, in our context at least, is to create wants.

Just because a person has crossed, say, secondary education, ‘traditional’ work no longer seems to be enough for him, whether he has been prepared for any other career or not. And of course if a person does get a job, the desire to be more and more like the ‘educated’ and upwardly mobile – leads to more and more and more wants…

At the other end of the spectrum of views on this, freedom from want is seen as getting rid of the wants! When education is more religious and ‘environmental’, it helps a person realize that his wants are really few and that he is at his most free when helping others, and reducing from the earth the burden of bearing him. A nation of ascetics is an interesting idea but probably not a very desirable one!

So that leaves us the vast space in between the two extreme views (of ‘want more’ and ‘want nothing) on ‘education for freedom’. Where do you find yourself on this? Is this the lens from which to look at ‘education for freedom’? Is this even a worthwhile question in our times? What do you think?

Immigration and Agriculture

There are some basic facts about agriculture. One ; you cannot do away with agriculture; you have to eat after all. Two; However much you mechanise agriculture, there are large portions of it that have to be done by hand and you need manual labour. Three; agricultural labour is hard hard work. You and I cannot do it. Four; everywhere in the world, the natives do not want to do agricultural labour. Five; You therefore have to “import” labour – poor people from places other than where the farms are and usually from another country. Six; the natives do not want these “immigrants” to come.

This is a real problem, which as stated, does not have a solution. Witness what’s happening in the UK.

The Brits don’t want to work in the fields. Therefore most of the agricultural labour comes from Eastern Europe (white Christians, mind you; the problem is compounded in the US because the labour is Hispanic). The Brits have voted to leave the EU and don’t want anybody coming into their country. End result – fruit is rotting in the fields as reported in the article linked.

This was not even about illegal immigration – these workers all came perfectly legally from Eastern Europe and since it was seasonal work, actually often went back to their countries after the picking was over. And yet the Brits have spat on their face and told them they are not welcome.

The same rural folk in the UK are the ones who voted for Brexit. London overwhelmingly voted for staying. And the main reason for voting exit ? Immigration. We don’t want foreigners; period.

The UK farming lobby wants to reinstate the seasonal workers scheme. The deal is , please come and do the dirty work we don’t want to do, stay in some ghetto so that we don’t see you, give us all the fruit and then bugger off to where you came from. Couching it in polite language does not detract from what it really is meant to be.

Agricultural labourers should simply organise themselves and show the middle finger to the UK. If you want us to work, treat us as decent human beings and give us the respect we deserve. Or else, you can do your own dirty work.

To paraphrase the Duke of Norfolk – you cannot have your fruit and eat it too !

Over-aged — and loving it! — Part 1

It\’s difficult, but imagine for a moment that you\’re an 11-year-old who wasn\’t able to attend school. When you were very young, you can remember, your parents moved from place to place, working on construction sites. A few years ago, they got work back in your village as a canal made agriculture more possible. And you yourself started off being an assistant cattle-herder. Now, though, you\’ve graduated to full cattle-herder, with knowledge of all the grazing areas, the watering places, the dangers to look out for (that unexpected ditch into which all the young cattle are always falling) and the idiosyncrasies of owners who don\’t always pay on time. As you saw children going to the nearby school carrying weird little bags or screaming insults at you, you wondered what they did holed up the whole day in that building. Even the cattle seemed to be more free than they.

Then one day, the newly appointed teacher organised a meeting with all community members and explained to them something called \’Right to Education\’. Basically, this meant that your parents decided you should go to school. No one asked you. Your father only said, \’Now work is more regular here, we can manage.\’ So off to school you were dispatched. Being alone with a hundred cattle in the nearby jungle (with the possibility of that nasty jackal) seemed so much less fearful than entering that stark building, all yellow and white with blue things written on it here and there.

What are the children in there going to say? Your mother made you have your bath and put on the other pair of clothes, so no one would say you smell — but the beloved odour of cows isn\’t going away from you and your clothes anytime soon. There are some green-painted metal play things on small play ground. The smell of food being cooked mingles with the smell of something else (it\’s paper and chalk and sweat, though you don\’t know it yet). Your heart is in your mouth as you step onto the ramp climbing up to the school. The teacher comes out and is looking at you — and you\’re doing your best not to run away. Away, back to the beloved forest, with the hundred cattle who know you so well.

The Poor Make an Educational Choice

Though it had been around for a long time, in 2003-04, a disturbing trend began to be dramatically visible in the government school system: a large number of districts began to report a decrease in the number of children enrolled. However, this decrease was not due to any slowing down in the growth rate of child population. Nor was it because accurate data was now available in place of the earlier inflated numbers. And since the number of children reported to be out of school was not increasing either, what accounted for the children missing from government schools? Yes, you guessed it – they were shifting to the ever-spreading network of the low-fee private schools.

The number of districts reporting such decreased enrolment stood at 180 or nearly one-third the number of districts in the country. Nor was this confined to the so-called ‘backward’ states – for Andhra Pradesh, Delhi, Gujarat, Himachal Pradesh, Karnataka, Maharashtra and Tamil Nadu also reported the phenomenon. In the year 2005-06, six new states reported districts with decreasing enrolment in government schools. And the situation hasn\’t really improved since.

The private schools that children migrate to come under the ‘unrecognized’ category, hence few government records are available on their numbers or growth. However, it is apparent that the increase in their numbers is astonishing. A World Bank study estimated that 28% of the rural population in the area studied had access to private schools in their own villages, and nearly half the private schools were established after 2000. Studies in Punjab showed that around 27% children studied in such schools and a similar picture obtained in Andhra Pradesh and Orissa.

This large-scale exodus has been occurring at a time when the government is spending an unprecedented amount of money and effort on education. Since 2000, tens of thousands of new schools have been opened in underserved areas and the infrastructure of existing schools boosted. Around 8.5 lakh teachers have been appointed and around 87% teachers in place provided 20 days of in-service training every year over the last few years.

Despite such efforts, anybody with any means whatever is choosing to walk across to a (usually) nearby school and pay for what they consider good education. This is in a context where education is available free in government schools, along with other incentives such as free textbooks and mid-day meals.

Like mobile phones, private education is no more the preserve of the elite. Surveys have found that 20% students in such schools are first-generation school-goers, with another 14% having parents with four (or less) years of education. Visits to such schools in the poorer regions of a state like UP put all doubts to rest. Without fail, it is the poor who are sending their children to schools that charge fees in the range of Rs. 30-100 a month. Schools manage this by paying teachers Rs. 1000-1200 per month – well below the minimum wage for unskilled labour. It is usually the educated unemployed who take this up as a means to gain experience while being on the lookout for other jobs. Therefore, teacher turnover is high, but there is a continuous stream of cheap labour available. The result is a commercially viable venture that provides subsistence level education.

In the meantime, who remains in the government system? For those hovering around the poverty line or below, there is no other recourse. Over 80% of SC and ST children in school are in government schools, which also have a higher proportion of girls and children with disabilities. In a telling comment, it is common for families with meagre resources to educate their sons in private schools and daughters in government schools. Indeed children are often enrolled in the government schools (for entitlements such as mid-day meals or uniforms) but actually attend the nearby private schools (for education)!

Unfortunately, the exodus of the more powerful and influential families has led to a greatly reduced sense of accountability in government schools. Those who are ‘left behind’ are usually the more disadvantaged groups, already disempowered due to economic and social reasons. Teachers, school heads and education officials tend to feel that it is almost ‘pointless’ to serve ‘these people’. In fact, a common refrain across the country is to complain of the ‘poor stuff we get to teach’ (and by \’poor stuff\’ they mean children!). There is an increasing tendency to blame the poor for not being able to support their wards at home or provide educational resource and the like. What is forgotten in all this is that education is not a favour being done to the poor – it is their right!

This is perhaps one of the reasons why the dramatic increase in inputs into the education system has not led to outcomes in terms of children’s learning levels, which continue to remain abysmal. Surveys by the NCERT and the NGO sector have repeatedly brought out how only half the children seem to learn half of what they should! During field visits to government schools, it is very common to come across children sitting unattended in class, with the teacher either absent or simply not teaching. Often, of course, the teacher has more than one class to handle and is therefore unable to teach. However, it is the sheer lack of concern for children that strikes any observer the most.

Many take the view that the expanding number of private schools is contributing to universalisation of elementary education in the country. While that is certainly true to an extent, a greater impact seems to be that in leading to reduced accountability, private schools are also contributing to a reduction in the government’s ability to universalise education in its own schools.

Listening Workshops – Or the Simplest Step to Educational Reform

Is \’bottom up\’ change really possible?
If you are an educational functionary, by now you must be fed up of hearing how planning and change have to be \’bottom up\’. By which is usually meant that those who are \’under\’ you must somehow begin to contribute, own and implement a range of actions. And you inwardly wonder if this is ever going to happen!

It was during a discussion on precisely such views that the idea of a listening workshop emerged. Colleagues in the Institute of Educational Development (IED) in BRAC University, Bangladesh felt that a \’listening workshop\’ might help them understand teachers and grassroots functionaries better.

Listening workshop – a straightforward structure
It was agreed that before forming any views, it is critical to simply listen to teachers and head teachers. Hence a straightforward meeting / interaction / workshop was designed around the following three questions that would be asked of teachers and head teachers:

What do you really do? Exactly what does your work involve?
What do you like doing?
What do you find difficult or dislike doing?

It was also agreed that IED colleagues initiating the discussion would only listen, and not prompt or provide leading questions or offer any comment from their side. In other words, they really had to listen rather than talk!

So why is all this worth writing about? Because around ten such listen workshops were actually conducted, and most turned out to have a very interesting pattern, followed by an unexpected twist.

What teachers felt
The listening workshops, it transpired, tended to proceed in the following stages.

Teachers found it really difficult to believe that anyone could come down from the capital only to listen to them! There had to be a \’hidden conspiracy\’ or an \’agenda\’ they were not aware of… It would take anywhere from 40-60 minutes to convince the participants that the intention really was to listen to them. (What do you think this tells us about the functionaries that teachers usually deal with?)
Once teachers believed the above, their initial reaction was that of giving vent to all their frustration and anger at \’you people who sit up there and form all kinds of views about us without ever visiting the field and observing the realities for yourself.\’
Finally, teachers would pour their hearts out on the three questions given above.

The teachers\’ replies have of course begun to inform the work of the institute in many ways. However, it was the completely unanticipated outcome below that left everyone (cautiously) elated.

The unexpected \’reform\’
In the case of a large number of teachers who participated, a few days after the listening workshop it was found that they were implementing many new pedagogical actions in their classrooms! In the entire discussion, at no point had they been asked to make any improvement in their classrooms. So it was not as if teachers did not know improved methods – a large number of in-service interactions had ensured that they had had exposure. It\’s just that they were not using them. But for some reason the listening workshops triggered a change process in the classrooms!

What do you think this tells us about teachers, about their motivations, and about the kind of relationships they experience? If you can bear the initial first hour, isn\’t holding a listening workshop the simplest way to initiate educational reform at the local level?

Are You An \’Education Survivor\’?

If you\’re reading this you obviously went through the education system. And maybe you are among those who are grateful that your school days were lovely. And that what you learnt is being put to use every day.

Or maybe not.

Conduct a group discussion with people (friends, colleagues, family members), around their school days. You will find a mix of smiles, frowns and giggles — and the frowns will usually be about their experiences inside the classroom. Almost everyone has a story of how they were wrongly punished or discriminated against or didn\’t receive their just dues for something or the other. Around half the people will recall the oppression they felt at different times — examinations, punishment being handed out, the subject/s they could make neither head nor tail of, the quiet acceptance by their families that they would be mediocre and their own realization that they would not be \’good enough\’ in a number of things.