Last week I started a discussion thread on the LingQ open forum in which I expressed my surprise at how popular the CEFR (Common European Framework of Reference) is among language learners and tutors. In the past I’ve tended to avoid describing my own Chinese level according to the framework because I don’t think doing so carries any information.
In this blog I will explain my reasons for having these misgivings.
What is the CEFR?
The CEFR is a widely used framework which ranks language ability on a six point scale, starting with A1 (pure beginner) to C2 (higher advanced). A description is provided for what you are expected to be able to do at each level.
Over the years I’ve noticed many learners boast about their CEFR level, treating it like an accessory they’re proud to possess. They might say ‘I’m B1 in Mandarin’ or ‘I’m C2 in French’ and pin these statements to their Twitter bio. This is often based on what their teacher has told them or a proficiency test they’ve passed.
The framework’s authors, the Council of Europe, provide several different charts on their website. For the purposes of this blog I’m focusing on the ‘Global Scale’, which is described as “a holistic summarized table….to provide teachers and curriculum planners with orientation points.”
What’s Wrong With Language Proficiency Scales?
I should first clarify that I think any attempt to create a language proficiency scale and apply it to all learners will inevitably be fraught with problems. In this sense many of the criticisms of the European Framework apply equally to all such scales.
To map language proficiency onto a scale you must first assume that, to a large extent, language learning progresses in a conveniently linear order and that this order is similar for most learners.
Conventional wisdom tells us we first learn to handle ourselves in simple everyday settings, such as a restaurant, before progressing to more formal language, eventually reaching the ‘pinnacle’ of language ability when we can write a PHD thesis or deliver a presentation on economics.
Reflecting this mindset the CEFR Global Scale tells us that at A2 (higher elementary) you “can communicate in simple and routine tasks requiring a simple and direct exchange of information on…routine matters.” At the most advanced end of the scale, learners who progress to C2 “can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation.”
In fact the reality for many learners is too messy to be described in this way. For example, learners who meet C2 criteria and can deliver academic presentations in a foreign language often struggle to meet the A2 criterion above when ordering food at a Chinese restaurant or buying a train ticket.
Elsewhere, the framework claims that at B1 (lower intermediate) you “can deal with most situations likely to arise whilst travelling in an area where the language is spoken.” But as I discovered when I first moved to Taiwan, using Chinese when travelling here regularly throws up all manner of unexpected challenges, regardless of your level.
A learner who can discuss nuclear physics with near-native fluency may be completely stumped when they first hear the phrase 有載具嗎？in a Taiwanese convenience store.
The framework also states that C2 learners “can understand with ease virtually everything heard or read.” But this is obviously impossible. Even in our mother tongue there are large swathes of language that we don’t understand about topics we haven’t studied or that aren’t relevant or interesting to us. For example, if I switch on the radio and they’re discussing cricket or the different stitches in knitting I will struggle to fully understand because I don’t follow these topics.
A few days ago I tested a Chinese friend who scores close to full marks on English proficiency tests, regularly reads novels with ease and can converse at a near native level of fluency on topics of interest. I asked her to read an article about the world’s most popular sport, football. Her comprehension immediately plummeted from close to 100% when consuming content that is relevant to her studies, interests and daily life to precisely 0%. All of the terminology, phrasing and metaphors used to describe the movement of the players and ball were completely alien and incomprehensible to her.
Would the CEFR’s authors still consider it appropriate to label her level C2? If they would, on what basis do they decide which topics a C2 learner should be able to understand and which topics they can get away with not understanding? If they wouldn’t, who on earth does qualify as C2?
It also strikes me as odd that the authors of the framework do not appear to think that pronunciation is terribly important. Although I found a separate scale for phonological control buried deep in the website, this has not been incorporated into the Global Scale which makes no mention of pronunciation. Perhaps this explains why many Chinese courses don’t prioritise phonetics and why so many allegedly C2 Chinese speakers have lousy tones.
How Should We Describe Our Level Instead?
When I raise criticisms of CEFR the question usually comes back: “what would you replace it with?”
I recognise this may be an issue for educators or course leaders who are forced to find convenient ways of measuring their students’ progress to justify themselves to their higher ups. But I don’t see why it should be of concern to anyone else.
If we’re interested in conveying our language level to others, be it employers or friends, why not use plain language that reflects our personal circumstances and describes what we can do. If we don’t trust our own judgment or aren’t sure about our own level we can ask a professional tutor to describe it for us based on their observations.
Let me illustrate this with my own case.
Suppose I tell you “I’m C1 in Mandarin”. Regardless of my reasons for citing my level as C1, it would tell you almost nothing. Take any sample of people who describe themselves as “C1 Mandarin” or have passed a C1 Chinese proficiency exam and you will find huge variation in their functional abilities across different competencies.
On the other hand if I say something like:
“I can communicate effectively in most everyday situations that come up living in Taiwan and discuss topics of interest with native speakers without strain. I can usually comprehend conversational listening content on topics of interest intended for natives speakers but may struggle if the language is more formal, specialised or if the speaker has a strong, unfamiliar regional accent (which is often the case). I’m also often thrown off by unknown cultural references and slang. “
This may not tell you a huge amount and you might want to ask some follow up questions to get a clearer picture but it does at least carry some information which is specific to my case. You now actually know something about what I can do with my target language which you didn’t when I told you I was C1.
The Mismeasure of Man
Much like ‘intelligence’ and IQ, language proficiency is too complex, subjective and multi-dimensional a concept to be measured or described with a single value. Discussing his classic book The Mismeasure of Man, in which he exposed the abuse of standardised tests to rank pupils’ intelligence according to IQ scores, the late biologist Stephen J Gould drew the following analogy:
“There’s a whole history in subject after subject in trying to encompass complex and independent attributes with a single number…in soil science people for decades got hung up on on getting a single number to measure the quality of soil. There is no such thing as the quality of soil, there’s just different things that soils can do!”
Stephen J Gould, Late biologist
Sadly the language education sphere is no exception. Educators have long been obsessed with measuring and pinning our abilities to arbitrary test scores and frameworks.
The rest of us would be well advised to ignore them and focus instead on what we can do with our target language.
*Sign up to our Chinese Masterclasses teaching you the key techniques needed to master Chinese tones and sentence mining
*Join our Facebook community to connect with other learners and get access to our weekly group language exchange call
*Subscribe below for regular podcasts, blogs and updates from our community pinged straight to your inbox
I always struggle to assess myself, which is what most men people in the online learning community seem to do. I’ve read the detailed descriptors, but there is so much ambiguity present. How do you determine what is a common everyday situation? Something that is common for me might not be for a resident of another country. It seems like there’s so much wiggle room to arrive at any conclusion you want.