Things we still don’t know – The Tutor Trust

I’ve been waiting for a while for the publication of the EEF report on the Tutor Trust’s secondary programme. For a couple of months I’ve been clicking on the appropriate link, only to be confronted with the phrase ‘due spring 2015’ even though we’re well into summer.

Why was I waiting for this? Because there is a distinct lack of studies that look at tuition for secondary schools in the UK. Much of the literature looks at programmes aimed at younger students, mostly literacy and numeracy focussed. This was an evaluation of secondary tuition in the English system, something I’m really interested in.

Now the evaluation of the Tutor Trust secondary programme is out and it tells us… not very much.

No control group = no answers

The report says that students on the programme performed slightly better at maths and slightly worse at English, but points out that due the difficulty in creating a proper control group any conclusions should be treated as having ‘very low security’.

The qualitative elements of the study showed that school staff valued the programme. However, there appeared to be a real variety of kinds of tutoring here, with school defining which students benefited and how large tuition groups were. The report even says ‘ In at least one school, tutors had been deployed in a wider range of roles, including to support marking and assessment’ rather than working directly with students.

So while school staff might be positive about the tutors, there appeared to be no defined programme as such.  From my point of view there is nothing in here to give us any guidance one way or another on deciding if tuition works and nothing on what specifically works (though this isn’t the first time I’ve seen something to suggest maths tuition is more effective than English tuition).

Questions

This leaves me wondering about my lack of scientific knowledge. I get that it is vital to publish results that don’t show what you’d prefer them to, but is it worthwhile to continue for three years with an evaluation that can’t answer the questions you want it to? Did the problems with finding an appropriate control group only occur after 3 years of work?

I don’t know the answers to those questions and don’t mean to imply criticism of those who conducted this evaluation. But I do know this doesn’t help answer the question ‘does tutoring work?’.

A level results day. Whoop!

The Access Project twitter feed was a thing of beauty yesterday. Here’s Toyo. 4 A*s and a place at Imperial to study maths. He worked with one of our tutors, Olly, for well over two years.

I spent the day in Oasis Academy Hadley, helping students through clearing and celebrating success. It was the third A level results day since I started with the Access Project and certainly the best so far. Our students achieved amazing things, with the help of our amazing tutors.

 

 

 

What Tuition Isn’t (and some other thoughts)

Over the last two weeks I’ve attended a couple of workshops hosted by NESTA, the Teach First Innovation Unit and Brightside. The workshops have bought together people from various organisations that provide some sort of one to one support, either tuition, mentoring or coaching.

The aim is  to discuss these three areas and see if there is enough shared common ground that we are able to give some general guidelines to best practice in one to one intervention.

As I promised in my first post, this blog is very much going to be me thinking aloud, so here are some of the ideas that I came away with from those discussions. I don’t promise that there is necessarily a through line between them all!

1. Tuition and Mentoring are defined by their goals, coaching by its method. 

There is a well defined method to coaching that is particularly favoured in business. This involves open questioning with the aim of helping the recipient find their own answer to the problem they are trying to solve. The coach is not there to give answers, the idea is that the recipient should be able to find those answers.

Coaching can applied to different goals, but it is popular in business as a personal development tool. It has a well defined methodology.

Tuition and Mentoring on the other hand do not have well defined methodologies. There are things you can say about good tuition in general, but what a good tutorial looks like might vary hugely depending on the academic subject being taught and the age or ability of the student. Similarly there appear to be several ways of mentoring, although I have to admit my ignorance about this in detail.

However, tuition and mentoring are easily definable in terms of their goals. Tuition is one to one support aimed at addressing an academic need. Mentoring is one to one support aimed at addressing a personal development need.

2. A spectrum of directedness?

I’m sure I could eventually come up with a better term than that, but some members of the first workshop talked about the three kinds of one to one intervention existing on a spectrum. Coaching would give the least direction to the recipient, tuition the most, with mentoring existing somewhere in the middle.

This is interesting, because its definitely wrong. Sometimes a tutor will have to be very directive (to solve trigonometry problems you need to follow these steps), sometimes totally undirective (here are some problems with triangles. See if you can solve them). Indeed the hallmark of a good tutor would be the ability to judge how much direction the recipient needs. In some tutorials the tutor might in fact be playing a role very similar to that of a coach.

I can’t be as certain about mentoring but I suspect something similar might be true.

It might be fair to say that tuition is likely to be more directed purely in terms of setting your end goal, but not in how you get.

3. Engagement isn’t fun and fun isn’t the aim. 

The nearest thing to an argument (actually a very friendly discussion) was on the desirability of fun. Some delegates talked about the need for recipients of one to one intervention to enjoy the experience. Some even used the word fun.

Unsurprisingly, I disagreed. I have no idea if this applies elsewhere, but notions of fun and enjoyability have been overemphasized in education. So much so that the word ‘engagement’ has been misinterpreted as ‘fun’ in some circles. Taken too far this can lead to educators mistaking themselves for entertainers which can definitely be a mistake.

I’m not claiming that recipients should hate the experience, far from it. What matters is not a sense of enjoyment but a sense of fulfilment. A sense that this intervention is useful to me, that it’s what I need and also (in tuition at least) that I’m having to work hard and push into knowledge that I’m currently not secure on.

4. How do you tell someone they’re disadvantaged? 

Many of these charitable organisations are working with recipients who are socially disadvantaged in some way, most of them children or young people. So how do you let them know why they’ve been selected to take part in your programme without making them feel stigmatised?

I don’t know the answer to that, but I do know that at the Access Project we have the luxury of working with older children. That means that, while I’d be very unlikely to use the word ‘disadvantaged’, I can start an assembly by saying ‘do you think it’s right that how rich someone’s parents are should be a deciding factor in where they go to university?’. Working with older students means you can have a conversation about fairness. You can discuss the advantages that money buys some and show young people that the Access Project is a means to replicate some of those advantages. You don’t need this because there’s anything wrong with you, you need it because others have an unfair advantage.

In summary

I know this a list of ideas with little structure. Sorry. Like I said, thinking aloud. It’s been really interesting to discuss one to one intervention with others and consider different approaches to coaching, mentoring and tuition. It’s also fascinating to see how many organisations there are working in this space, making the world better.

Does tutoring work if it boosts confidence?

My two main questions about 1 to 1 tuition are:

• Does it work?

• What works?

But what do I mean by ‘does it work’? My usual assumption is that this refers to academic achievement but frequently students and parents report that improved confidence is their primary aim. I’ve seen this with Access Project parents and students too: confidence in a subject is often cited as the most important reason for having a tutor.

It was also an important factor for parents interviewed in Reading University’s review of the Explore Learning programme (PDF) . Parents interviewed for this study talked about their children’s ‘greater self-belief and confidence’, with many considering this more important than ‘increasing performance and attainment goals’.

This also chimes with Ireson and Rushforth’s survey of parental motivations (PDF) which showed increased self confidence to be the second most important reason for employing a private tutor, with 69% of parents interviewed citing this as their reason for doing so.

Of course confidence is not entirely unconnected to achievement, but there’s no guarantee they’re the same thing. for example that Explore Learning report apparently showed lots of students who were already above average in English still receiving tuition for little gain compared to a control group.

If pupils and parents want increased confidence in a subject and tuition gives them this, surely that means tuition is working even if the eventual grade might be the same as it would always have been.

Hard or Soft?

Maybe what we’re talking about here is the difference between hard and soft outcomes. Test results? You can put a definite number on those. They’re hard and robust, sciencey and reliable. And so people like me can assume they’re more important than soft, wooly stuff like people’s feelings.

When I put it like that (slightly facetiously) it’s obviously wrong. Emotions matter and just because they’re more difficult to measure doesn’t mean they’re less important. Hard and soft is a false distinction. Better to say easy-to-measure and tough-to-measure.

Is it useful to worry about a student’s emotional education?

Yes of course it is. Paul Tough’s book ‘How Children Succeed’‘* for instance talks about grit and the idea that certain characteristics, confidence being one, are vital to educational outcomes.

It’s not a clear cut issue though. Some studies suggest that confidence is possibly inversely linked to achievement. Over confidence can be a problem, perhaps a symptom of dumbing down. As I’ve written elsewhere:

David Didau writes about a report by Tom Loveless for The Brown centre looking at PISA Data from 2012 that suggests:

“Countries that do well on student motivation do poorly on maths attainment and vice versa. Contrary to every intuition, student engagement and motivation may actually be retarding learning.”

I know motivation and confidence might not be exactly the same thing, you can be motivated to engage with something you’re not confident about, but they’re not a million miles away from each other.

So what’s a tutor to do?

A good tutor is a master of balancing the student’s emotional outlook with their educational needs. If all you did was practice material the student was confident with they might very well leave your tutorials feeling confident and secure. But if you never address the material that challenges them, never push them into areas they might not feel comfortable with (to begin with) no learning is going to take place.

For a private tutor that might bring up extra challenges. Are parents going to understand the need for necessary levels of struggle? Is the tutor going to have to worry about balancing the desire for a happy student who wants to attend tutorials and a challenged student who has to work hard on new material?

For those of us in the not-for-profit arena is there a similar problem? We are keen to measure academic improvement and feel like this is easier to do (although there are all sorts of problems with educational data). Are we making a mistake if we neglect the confidence issue? Do we need to measure that?

Does tutoring work if it boosts confidence?

If that’s what you want it to do, yes it probably does.

Does boosting confidence necessarily equal better performance?

No, not necessarily.

*The blurb on Paul Tough’s website describes the book as ‘profoundly hopeful’. I do love a bit of needless hyperbole.

 

One to One tutoring? No-one knows anything (Including me).

“Cutangle: While I’m still confused and uncertain, it’s on a much higher plane, d’you see, and at least I know I’m bewildered about the really fundamental and important facts of the universe.

Treatle: I hadn’t looked at it like that, but you’re absolutely right. He’s really pushed back the boundaries of ignorance.

They both savoured the strange warm glow of being much more ignorant than ordinary people, who were only ignorant of ordinary things.”

― Terry Pratchett, Equal Rites

I didn’t study science at university. This is something I mildly regret. Not enough to think I might have made different choices – prioritizing music was the right thing to do – but I do regret my lack of expertise in some areas.

For example, I’ve had to learn how to read and understand studies and get to grips with ideas like effect sizes and randomized controlled trials and the like. The basics of these aren’t difficult (I think I sort of understand) but I am constantly left with the feeling that I don’t quite know what I’m talking about.

That’s good isn’t it? What is science if not a highly developed sense of doubt.

Is healthy skepticism warranted?

I mention this because I’ve been reading up on one to one tuition, trying to find answers to two questions:

• Does one to one tuition work?

• What works?

The simple answer to the first question appears to be ‘yes, probably, in some circumstances’. The simple answer to the second question might be ‘We’re not quite sure, but possible these things…’.

I have frequently seen Bloom’s 1984 work (pdf) cited. In this he claimed that tutoring was very effective, with students who were tutoring greatly out-performing those in control groups. So well cited was this that I thought it was sound, but others have cautioned against drawing too strong a conclusion from this relatively limited study.

The Education Endowment Fund (EEF) think highly of one to one tuition, saying that it can have an impact equivalent to 5 months additional education. However, they are clear that it is particularly useful for ‘younger learners…and for subjects like reading and mathematics’. There’s simply less research for older students and for a wider range of subjects.

This is true of lots of studies. The literature seems to focus mostly on reading or maths tuition programmes, and while it can be good research and show positive results in no way does such research cover the panoply of tuition programmes in existence.

And that’s just when trying to figure out whether such educational interventions work.  Answering my second question – what works? – is harder still. One to one tuition is by its nature hard to observe. One of those studies conducted by the EEF for example showed that one to one tuition made an impact, but there was little difference between the tutors who followed a particular programme and those who did not.

Huge areas of tuition are simply unknown, some of them unknowable. Judith Ireson and others have described a ‘shadow’ education system quite apart from the established school system. Much of it will be good, much of it will be bad, much of it we don’t know much about.

So What do we know?

One to one tuition might work. It seems to work for numeracy and literacy programmes, particularly with younger students.

It’s also very popular, and lots of parents and students are happy with it.

Beyond that, it is seems sensible to be cautious and admit what we don’t know. I might be coming at these questions from a non-science background, but even I can see that we can’t be more certain than that.

Why start a blog on One to One tutoring?

Why have I decided to start this blog?

The main reason is entirely selfish. Having taken on a new role at The Access Project coordinating the development of our tutoring programme, I want to get my thoughts in order. I find writing helps me do that.

Hopefully by making my thoughts public I’ll also be able to get others to take part in the conversation and share their ideas.

I will then steal their ideas.

Beyond that, tutoring is massive, and largely hidden, unregulated and ‘invisible’. The private tuition industry is huge, with some estimates putting it at £6 million a year. Tuition charities and social enterprises are growing and being used by many parents and schools to supplement children’s education. As many as 1 in 4 pupils may have used private tutors.

One to One tuition is one of the oldest educational tools, it’s widespread, and so it’s worth thinking about.

What will you find on this blog?

Some of the blog is going to revolve around my work. I’ll be developing training and support for volunteer GCSE and A level tutors and will use this blog to share ideas and talk through what I’m working on.

It will also contain wider reflections that relate to tutoring, centered around two basic questions:

1. Does one to one tutoring work? What do we mean by ‘work’? What constitutes success?
2. What specific models and pedagogical techniques are most effective in one to one tutoring.

In short Does it work? and What works?

As of yet I don’t have a regular schedule in mind. I’m sure one will develop.

Will I talk much about private tutoring?

I’m not sure. Beyond my experience as a music tutor, I don’t know much about private tuition. It certainly seems to be potentially controversial, or rather the distribution of it and the fact that it is not available to those from more disadvantaged backgrounds can be controversial. It is also in many ways invisible with no real regulation or oversight.

My natural inclination is to be wary of private tuition that supplements school subjects, not because I think it is inherently wrong but because I am suspicious of the profit motive being involved in that sort of education.

But that’s a tentative maybe, not a fully formed opinion. I don’t really know.

So let’s get blogging then..

Those are my questions and may aims. Now to write some blog posts…