PLENARY SPEAKERS

Why is it important for you to attend the 4th ALTE international conference?

The ALTE conference will afford me an opportunity to meet language testers from all over Europe and from other parts the world, to learn of their interests and concerns. It will also provide an opportunity for me to share my ideas with them. I’ve never been to either an ALTE conference or Kraków, so my mind and eyes will be wide open.

In your opinion, what are the key issues that you would like to see discussed at the 4th ALTE conference, both relating to the conference theme and beyond?

A key issue in my view is how to justify all the different appropriations that are being made of the CEFR for language testing purposes. Most of what I've seen referred to as “scales validated against CEFR”, is nothing but smoke and mirrors, just looking at a test item, and then saying, “This is a C1 item”, for example. In my view, more rigour, in terms of professionally accepted practice in creating, adapting, or equating scales, needs to be brought to this enterprise. A concomitant issue is that of providing adequate professional training in language testing and measurement to the many school systems in Europe that are being required by various government bureaucrats to “link” their measures to the CEFR.

Another on-going, continuous concern is the great amount of misunderstanding that exists regarding issues of validity and fairness, and how best to link these in a justifiable way. While there has been much discussion over the years of both validity and fairness, the integration of these two is, in my view, not well understood. The result is that some people focus on validity and ignore fairness, while others do the reverse.

I hope that the conference will provide an opportunity to debate these issues, and that delegates from all sides – theory, practice and policy – will gain a better understanding as a result.

Developers of large-scale language proficiency tests have historically drawn on theoretical frameworks of language to define the construct to be measured. Over the past half century, a wide variety of such frameworks, ranging from the “skills and components” framework of Lado (1961) and Carroll (1961) to the more recent “communicative” and “task-based performance” approaches, have informed the development of large-scale language proficiency tests. If these tests existed in isolation, being used for different purposes and with different populations of test takers, this multiplicity of language frameworks might not present problems.

However, in an increasingly global “market” of language testing, there is increasing pressure, on both test developers and test users, to find ways of “linking” tests that may have been developed for very different purposes, for different populations of test takers, and which may be informed by very different views of the construct to be measured.

The primary purpose of “linking” two tests is to enable test users to interpret and use the results of the two tests in the same way. Or, from another perspective, the developer of one test wants to claim that her or his test can be used in the same way as another test. For me, using two tests “in the same way” requires that the two tests measure the same construct, that the decisions to be made are similar, and that the consequences of these decisions are similar. In my view, many current linking activities do not provide adequate justification for these basic requisites. Given the pervasiveness of such claims and practice, I think it is imperative for us, as a profession, to address some very fundamental issues about the nature and justification of “linking” different language tests. One central issue is that of the extent to which the language frameworks upon which two tests are based are comparable, because this provides the conceptual basis for any empirical studies into the comparability of score-based interpretations. An equally critical issue is the comparability of decisions and consequences, because these are what ultimately affect the lives of test takers and other stakeholders.

In this presentation I will begin with a brief overview of the different language frameworks that have informed large-scale language tests in the past half century, and will argue that these frameworks fall into two general types: “can do/performance” and “ability”. I will then use an assessment use argument (Bachman & Palmer 2010) to analyze the ways in which these two approaches to defining language differ in terms of the claims they make about interpretations, decisions, and consequences. I will then discuss the different uses for which tests based on these two ways of defining language might be most appropriate. Finally, I will return to the issue of the difficulty of “linking” tests based on these two different types of language frameworks.

Professor Lyle Bachman is Professor of Applied Linguistics at the University of California, Los Angeles (UCLA). He has received numerous professional awards and honours from universities and professional organisations. His current research interests include validity theory, assessing the academic achievement and English proficiency of English language learners in schools, assessing foreign language proficiency, and epistemological issues in Applied Linguistics research. He has published numerous articles and books in the fields of language testing, educational measurement and applied linguistics. His most recent publications include Statistical Analyses for Language Assessment (CUP, 2004) and Language Assessment in Practice: Developing Language Assessments and Justifying their Use in the Real World (with Adrian Palmer, OUP, 2010).

Questions relate to the (negative) effects of the tool on teaching and learning, the unjustified simplification and reduction of the actual complexity of language overlooking important language and non-linguistic components that need to be considered. Moreover, there have been debates as to its enormous political power in various countries beyond the European Union and the belief that it can serve as a tool for all language needs regardless of circumstances and contexts. Questions are also raised as to the political and economic power of one tool and its marketing and the danger in using one uniformed homogenous tool. Other questions are raised about the limited breath of the construct given the expansion of the meaning of language in the past decade and the fact that the CEFR is still focused on monolingual notions of language, and thus is detached from the multi/bilingual aspects of language, excluding L-1s, as well as to language negotiations of meanings beyond single languages.

The purpose of this paper is to raise multiple questions and issues that challenge the CEFR construct and its wide use and thus to engage in critical language testing and assessment of the tool. These questions and critique will lead to a number of proposals as to the developments and expansion of valid types of criteria for assessing language(s) and its related factors that will be responsible for effective performances of language and its associated factors.

Recently, she has been working on expanding the 'language' construct that needs to be assessed to include bi/multi/hybrid languages, digital literacies and languages in the ecology (linguistic landscape). Elana is the author and editor of several books, such as The Power of Tests (2001), Language Policy (2006), Encyclopedia of Language and Education, vol 7, Language assessment, (2008), and Linguistic Landscape, Expanding the Scenery, 2009.

Why is it important for you to attend the 4th ALTE international conference?

I have been involved in the work of ALTE for some time: I organised one of the earlier ALTE meetings, held in Kraków in 2004, and I am acting as a representative of the host venue, the Jagiellonian University (an ALTE member since 2000). I am therefore honoured to be invited as a Plenary Speaker to ALTE Kraków 2011, both because the conference will be hosted in my home town and at my old university, and because ALTE is a member of the Professional Network Forum established by my own organisation, the European Centre for Modern Languages. As a result, I have many good friends in ALTE and I look forward to meeting them again at the conference.

In your opinion, what are the key issues that you would like to see discussed at the 4th ALTE conference, both relating to the conference theme and beyond?

The key issues I would like to see discussed are best summarised in the Graz Declaration on Language Education 2010, adopted by the Professional Network Forum and signed by a representative of ALTE. During the conference, I plan to promote ECML's new medium term programme of activities (running from 2012 to 2015) which aims to address the issues raised by the Declaration, and to help achieve this, I hope to gain further international co-operation from delegates.

From a professional perspective, what can delegates gain from attending the conference, both those closely involved in the theory and practice of language assessment and those with a broader interest in the subject, such as policy makers?

ALTE international conferences offer a unique opportunity to bring representatives from research and practice together with policy makers and decision makers. I have taken part in all three ALTE conferences to date and benefited greatly from the experience, as a researcher, a practitioner, and as a policy maker.

These challenges call for the urgent development and implementation of new quality approaches to European education in general, and language education in particular (policy-making, teacher education, learning, teaching, and assessment).

A new concept of education is required to meet the needs of plurilingual people living in increasingly multilingual societies of contemporary Europe. It needs to be based on inclusive plurilingual and intercultural approaches constituting a pedagogy integrating the needs of all learners. This pedagogy represents a broad vision of educational provision. It reaches beyond the foreign language classroom and impacts on all areas of formal and informal learning at all levels and stages in life.

Inclusive plurilingual and intercultural pedagogy provides learners with opportunities to develop their language knowledge, skills and attitudes as their linguistic repertory. In practical terms, this will enable them to use several languages at different levels of competence by finding a recognised place for all their linguistic abilities. Building and living in an integrated society, contributing to cross-border interaction and international collaboration are the ultimate aims of quality education for all learners.

Recently, the Language Policy Division of the Council of Europe launched a Platform of resources and references for plurilingual and intercultural education, expanding the scope of consideration beyond the domain of foreign modern languages and including classical languages and languages of schooling – learning, teaching and assessment of languages taught as school subjects (majority language such as Swedish in Sweden, Polish in Poland, etc.) and language competences required for other school subjects – language across curriculum. With this new instrument the Council consequently draws the attention to the needs of the individual learner underlining that access to education and success at school heavily depend on language competences. A major challenge for today’s education systems is then to support learners in acquiring adequate plurilingual and intercultural competences which will enable them to develop as strong individuals and operate effectively and successfully as citizens. The Platform offers a new, open and dynamic framework of reference, with system of definitions, descriptions and descriptors, studies and good practices which member states are invited to consult and use in support of their policy to promote equal access to quality education.

Within its upcoming 4th medium-term programme of activities (2012-15) the European Centre for Modern Languages in Graz aims at facilitating European cooperation on the implementation of the new concept of plurilingual and intercultural education initiated by the Language Policy Division. In the crucial stage of testing, awareness raising, adapting and implementing this educational policy in practice, the ECML can act as a laboratory for targeted policy application and innovative practice. The cooperation the ECML initiated in 2010 with the major INGOs – stakeholders in the areas of evaluation and assessment in language education, among them with ALTE – will become an important dimension of this work.

Teacher trainer, author of textbooks, curricula and testing materials for Polish as a foreign language, he also holds the position of Visiting Professor and Lecturer at several universities in Germany (Bochum, Giessen, Göttingen, Mainz, Münster), Switzerland (Basel) and in the USA (Stanford). He translated the Common European Framework of Reference for Languages (CEFR) into Polish (2003) and seconded to the Council of Europe, Language Policy Division (Strasbourg, France, 2005–2006). Since October 2008 he has been the Executive Director at the European Centre for Modern Languages of the Council of Europe, based in Graz, Austria.

Why is it important for you to attend the 4th ALTE International Conference?

The ALTE Conference always brings together a very interesting group of people, and offers a great opportunity to meet and engage with delegates from many different backgrounds. The Conference is also a time when we can sit back and think more about what we are doing.

From a personal perspective, what are you looking forward to most about the event?

I was both a student and a teacher (of English) in Kraków in the 1970s and I'm looking forward to revisiting a city which has great architecture, great culture, and also a great tradition of debate and discussion.

In your opinion, what are the key issues that you would like to see discussed at the 4th ALTE Conference, both relating to the Conference theme and beyond?

I am particularly interested in formative assessment – having been both a teacher of English and, post-PhD, a developer of theory, I am well aware that much work needs to be done to bring together the objectives of language frameworks and the needs of classroom teaching. ALTE's work with 'can do' strategies was developed in order to translate language teaching goals into practical objectives, and I am hoping that Conference delegates can discuss 'learning how to learn' in more detail. This demands a multi-level approach, both intellectual and strategic, but is the key to successful teaching.

I am also very keen to gain more insights into the different contexts across Europe in which language learning takes place. This is of particular relevance to my work with SurveyLang, where we are investigating the relationship between context and learning and the effect this has both on personal outcomes and on national policy.

From a professional perspective, what can delegates gain from attending the Conference, both those closely involved in the theory and practice of language assessment and those with a broader interest in the subject, such as policy makers?

The Conference offers a great opportunity for delegates to share best practice, and it is extremely useful to bring different voices together. Communication between policy makers and the assessment profession has always been an issue, and a Conference such as ALTE 2011 goes some way to bridge this gap.

The issue of frameworks is an important area of discussion. The climate in which language testing is now discussed is much more political than at the last Conference. In particular, language frameworks are being seized upon as possible policy tools in areas such as migration, but the danger is that the outcomes defined by a framework are analysed with only a very cursory understanding of what assessment is all about or what it can do. This was usefully summarised by McNamara and Roever in their 2006 publication Language Testing: The Social Dimension. They perceive 'a tendency for governments to use assessment as part of a general climate of greater accountability in education and, more specifically, as a means for the achievement of particular political goals. The politicization of assessment in these ways is perhaps the most striking feature of current developments in language assessment' (p213). They find the CEFR to be 'dominating language education at every level in Europe, in the most comprehensive example of policy-driven assessment yet seen' (p212).

At Conference, we need to stress the importance of the testing process when applying a framework, and the need for framework users to understand the testing context so that the framework fulfils its role as a means of identifying useful points of comparison, rather than as a regulatory and prescriptive instrument.

Above all, the ALTE Conference allows all those with an interest in language testing to 'think sensibly', and to bring thoughtfulness and consideration back into the process. Given the nature of some of the subjects we will be debating, Conference gives delegates a real opportunity to explore these difficult issues in depth.

'Second language' speakers may face the challenge of simultaneously acquiring both schooling and the language of schooling. Such different experiences result in different profiles of language ability, but these can be described in terms of configurations, differing over time, of the same parameters: informal learning, formal language knowledge, social and academic contexts of use, cognitive stage. A framework encompassing these parameters should encompass all three kinds of learner. We need such an inclusive theoretical framework because these learners de facto co-exist and intersect within educational settings which direct their learning, and qualifications frameworks which compare and judge them, on their language, or other skills mediated by language. And beyond education, they share all the personal and professional opportunities that specific language skills afford.

I see the CEFR as an instance of a more general framework, parameterised and illustrated for the case of foreign languages. By adding parameters and illustrations we can extend it without challenging its validity for that original purpose. Doing so should enable language assessment to deal elegantly with many linguistically complex educational contexts, and by better analysis and description of different learner groups to contribute to improving educational outcomes for language learners of all kinds.

In developing this I shall discuss two parameters not treated explicitly in the CEFR:

I will use as illustrations several real educational contexts where an inclusive coherent framework for L1, L2 and FL would clarify and facilitate the design and evaluation of teaching programmes and assessment.

Since joining Cambridge ESOL in 1992, having completed a PhD in the area of IRT, he has contributed to the development and implementation of the item banking system which now underpins all of Cambridge ESOL's language assessments. His current interests focus on the intersection of assessment and learning, particularly in relation to the construction and use of language proficiency frameworks, such as the Common European Framework. He is Vice Director of SurveyLang, a consortium currently working to deliver the European Survey on Language Competences.

Why is it important for you to attend the 4th ALTE international conference?

I have been a regular Conference attendee since they began, and I always go home with many new insights and useful ideas. I have greatly increased my knowledge of the work my colleagues have been undertaking in other European languages, which has also helped me in my work.

From a personal perspective, what are you looking forward to most about the event?

I find the chance to consult with experts in different languages most useful and stimulating. I am also looking forward to the workshops and lectures where I can deepen my wider understanding of the field.

In your opinion, what are the key issues that you would like to see discussed at the 4th ALTE conference, both relating to the conference theme and beyond?

Implementing the Common European Framework of Reference for languages (CEFR), is still a challenge, and prompts some key questions. For example, how does the concept of 'levels' impact on language teaching with regard to the type of language students learn? And what level should they reach when, and how should this be tested? These questions form the background of my own presentation on SurveyLang, the survey on language learning across Europe.

From a professional perspective, what can delegates gain from attending the conference, both those closely involved in the theory and practice of language assessment and those with a broader interest in the subject, such as policy makers?

Specifically, methods of validating and of analysing data from pre- and post-testing are two areas where I will learn more by attending the workshops. But I also think that the ALTE Conference is a great opportunity to hear more about new testing formats, such as the computer-based testing of oral proficiency.

The European Commission described the purpose of the survey as to provide participating countries with comparable data on foreign language competence and knowledge about good practice in language learning.

SurveyLang, a group of eight expert organisations representing five European languages was awarded the contract to carry out this survey in February 2011. The survey will test the two most widely taught European languages (from English, French, German, Italian and Spanish) in each country from a representative sample of pupils in their final year of lower secondary education.

Five language partners worked together to create language test tasks on four levels of the Common European Framework of Reference – A1, A2, B1 and B2. The aim was to create items with a high degree of similarity and which were at the same time as true to their respective cultural and linguistic origins as possible. In the context of such close collaboration of five language partners it became possible to observe to what extent it is feasible to create test tasks and items of identical difficulty without applying a method of mere translation.

The A1 level

A1 is of particular interest for several reasons. It is the first level that can be attained and therefore has a special status within the level system. It has the largest number of participants. From the point of view of test development it is the most demanding one because the language that can be used in rubrics, contextualisation, texts and distracters is so restricted. On the content side it is revealing to see how different languages interpret the fulfilment of basic needs. The corpus of language is still relatively well defined. For all these reasons this is why the focus of attention in this paper will be on the A1 level.

The paper

The paper will give an overview of the whole project in its main stages. It will focus on the perception of the A1 level as an example for cross language work by discussing the following aspects:

The examples will be taken from the German and the other languages.

She currently holds the chair of the Standing Committee and has conducted seminars on benchmarking of spoken samples according to the levels of the CEFR. She has been involved in developing tests for migration and integration, and has been director of the project 'Deutsch-Test für Zuwanderer' (German Test for Immigrants) which is being implemented in Germany nationwide since July 2009. Currently, she is responsible for the German test items in the European Project on foreign language competences of school learners – Surveylang.

The test will be administered to more than 200,000 migrants, beginning in 2011. In addition, a credit system has also been set up for legally resident long-term migrants. Language certificates as well as attendance at language or other training courses are part of this system. In the coming years, around 4 million migrants will be affected by this new legislation.

The Ministry of the Interior is responsible for the drafting and implementation of the new regulations. The Ministry asked four Italian Institutions - Università per Stranieri di Perugia, Università per Stranieri di Siena, Università Roma III, Associazione Dante Alighieri - which are officially recognised by the Italian Government to deliver language certificates - to work together on a joint project in order to develop and produce the new A2 test for migrants.

Several issues need to be addressed. In particular:

The project “Italiano, lingua nostra”: aims, methods, results
“Italiano, lingua nostra” is a national project recently carried out by CVCL (Centre for Language Assessment and Certification) at the 'Università per Stranieri' in Perugia. This Project has been promoted by the Ministry of Interior and funded by a European fund for the integration of third world citizens.

The project was planned, coordinated and carried out over a period of four months by CVCL in collaboration with several partners: state schools involved in language teaching to migrants, and charity organisations. 147 language courses (at A1 and A2 level) were organised in 10 Italian regions (using the same syllabus and the same teaching materials), 2880 migrants were involved and 1837 certificates were awarded (at A1 and A2 level).

A dedicated network has been established in order to enable different stakeholders to communicate, and exchange experiences and information. Video interviews with teachers and students were conducted and anonymous questionnaires were also administered. CVCL is currently analysing the data in order to investigate the impact of the Project from both a social and education point of view.

Conclusions
The feedback we get from projects such as “Italiano, lingua nostra” can be positive and rewarding for the institutions and people involved, and useful data can be collected and published by researchers. However these projects are expensive, and can only involve a limited number of migrants. In other words, they are not part of a systematic approach to the language education of migrants and what is probably needed is a more coherent, common and systematic approach that should be agreed among experts at both European and national level.

She is currently Director of CVCL (Centro per la Valutazione e la Certificazione – Centre for Evaluation and Certification). Her research interests are in language testing and language teaching.

Professor Bolli worked on the development of the CELI (Certificati di Lingua Italiana) Examinations Suite and has been working with ALTE since 1990. She is also responsible for the SurveyLang project for Italian. She was coordinator of the Reference Level Descriptions (RLD) project in Italian. She is the author or co-author of several publications on the CVCL's Certification activities and projects involving CVCL's work with the Council of Europe and several ALTE members with regards to possible applications of the CEFR.