Veille et Analyses
http://www.inrp.fr/vst

 

Dossier d'actualité n° 39 – November 2008
Assessment: a central issue for learning

by Laure Endrizzi and Olivier Rey

The educational treatment of knowledge generally appears inseparable from the processes used to assess it; examinations and the various forms of evaluation play a central role in educational systems, whether as proof of training (certification assessment), to check whether knowledge and competencies have been acquired (summative assessment), to assess the level reached and the student's potential to continue in one way or another (forecasting assessment), to measure the level reached by an age group or a school population (diagnostic assessment), etc.

All these forms of assessment are important and sometimes omnipresent in the way education systems operate. They invite consideration of the problem of the role of marking and the impact these different kinds of assessment have on the piloting of syllabuses and the curriculum They are also deeply questioned by the development of the competency-based approaches, which call into question conventional assessment forms and tools.

In this issue we have generally preferred to talk about "assessment for learning", sometimes also called "formative assessment", rather than "assessment of learning". In other words, how does assessment come into play in the course of the very process of learning, to facilitate it, redirect or control it, rather than as a form of disciplinary measure applied on an a posteriori basis?

Within this framework, special attention will be given to student engagement in the assessment processes, including the various dimensions that self-assessment may take. The question of the portfolio in education, which concentrates many expectations surrounding these subjects, will be dealt with separately.

Warning to readers:

  • Most of the links correspond to the relevant files in our bibliographic database, which includes complete references and, where applicable, access to the articles quoted (some offer free access and some require payment, depending on the article and the electronic subscription taken out by your institution);
  • You can inform us of your reactions to this Newsletter, suggest relevant themes or ask for more specific details by leaving a comment beneath the corresponding post in our blog: "Écrans de veille en éducation".

From one assessment to another

What kind of assessment are we talking about?

Few processes seem as "natural" in education as that of learning assessment, because of the existence of a whole range of testing and examination systems using increasingly more refined techniques as technologies develop.

However, while education has always existed in one form or another in most human societies, points out P. Broadfoot, it is only more recently that assessment has become a central and sophisticated professional area, in the context of mass schooling

The development of formalised and systematic assessment has obviously something to do with the legitimization of social standing via diplomas, the promotion of academic merit which marked the construction of modern States and the use of assessment as a tool for institutional control at all levels (Broadfoot, 2007).

If there is one word which is polysemous, it is “évaluation” (in French). It covers many processes in the educational system, since it can be used to indicate supervised testing by a teacher in his classroom, a national examination like the baccalauréat, an international assessment of pupils attainments like PISA, an examination leading to a diploma, a measure aimed at assessing the quality of a school, and even a teacher-auditing procedure, etc.

This simple list immediately shows that we are really not speaking about identical processes, even though each time it is question of measuring or judging, and even though it is obvious that the continuum which may exist between the "individual" assessment of a pupil and the more "institutional" assessment that takes place when standardised tests are used in the United States (Berlinet & Amrein, 2002) or international assessments in education systems is perceived (Emin in Baillat, De Ketele, Paquay & Thélot, 2008 ; Cytermann & Demeuse, 2005).

This may also be understood as a distinction between assessment which is an integral part of the teaching and learning process, and assessment for communication, which aims at providing information to potential users and partners of education, whether these be students, teachers, institutions or systems (Broadfoot, 2007).

Within the framework of this issue, there was obviously no question of dealing with the whole field potentially covered by assessment. Instead, we have focused on the particular issue of assessment in and for learning, i.e. the assessment of pupils in the particular context of school-type teaching situations. Although even in this school situation, several significantly different types of assessment can be distinguished. Traditionally assessment has been distinguished according to the functions which it fulfils in the learning process: diagnostic, forecasting, summative, certifying, formative, etc.

All these different forms of assessment do not have the same effects on people, even beyond the school context; summative assessment, for example, is frequently used as a marker of the educational identity of the pupil, owing to the fact that it is often recorded in an official and public document, such as the report card (Allal, in van Zanten, 2008).

Assessment can also be categorised according to what it evaluates (knowledge, skills, competencies, etc.) or according to the assessment system used (written tests, observation grids, or portfolios). At a time when several countries are rolling out an agenda to develop computer-based assessment, or even migrating tests and/or examinations on to digital media (Scheuermann & Guimarães Pereira, 2008), the fields to be investigated from the assessment standpoint grow exponentially: technical questions emerge (what tools?), and methodological (how?), or ethical (how far?) and psycho-cognitive ones (what is the impact on results?). In this context, the articulation between learning, assessment and ICT is a still relatively unexplored area.

The summative type "tests" or "examinations" encountered throughout schooling that spring most spontaneously to mind, are not of the same kind as certifying assessments (an examination leading to a diploma, for example), prognostic assessments (entrance examinations or examinations to assist with guidance) nor diagnostic assessments (large-scale surveys on a particular class group) with no "direct" stakes for the person being assessed. As for formative assessment, this is often mixed in with the other forms of assessment, insofar as it is its use as part of the learning process which makes it stand out.

While certification has long been the only outcome of assessment, the educational democratisation which has been developing since the nineteen-sixties has put forward the preoccupation with assessment as a continuous checking process to guide the approach to teaching and learning (Scallon, 2007). This explains why thinking today does not just involve a question of measurement accuracy, nor the end product (the test result, for example), but also deals with pupils' progress, the area we want to concentrate on here.

According to Scallon, this can also be seen in the vocabulary used in research in English: here, the terms “measure” and even “testing” have practically disappeared, and that of “evaluation” is more or less kept for organisations or systems, giving way to “formative assessment” or “classroom assessment”.

The attempt to find faithful, valid and non-subjective processes, which judge all that is supposed to make up the attainments of an individual at a given time and in one-off fashion, is no longer the priority, according to G. Scallon, even if this was, for a time, the sphere of docimology.

Moreover, certain researchers, such as B. Rey, stress that whatever technical refinements are involved in the assessment, "the dimensions that are measured and taken into account are the effect of the choice of a subject” (Rey in Baillat, De Ketele, Paquay & Thélot, 2008, p. 60).

Assessing is always a judgement according to a value, and the issue is therefore not so much to make the assessment more accurate and fair, but rather “to communicate to the person being assessed what is expected of him and therefore encourage him to subscribe to the aims of the formation”.

While this "objective" measurement is nevertheless what is often still sought through marking practices, which are the major characteristic of existing approaches to assessment, most experts defend the idea that assessment is a "message" more than a "measurement" (see also the profuse on-line article by Jacques Nimier: “Cette évaluation impossible et pourtant nécessaire”).

Marking: the impassable horizon of assessment?

Marks indeed concentrate many expectations in the educational system and still appear as the main symbol of assessment, of necessity summative, insofar as formative assessment still appears, in the eyes of the majority, as an optional complement more than as another possible type of assessment.

Whether to use quantified assessment has been the subject of much controversy which has sometimes seemed to dominate debate on the implementation of the Renouveau pédagogique in Quebec: a significant number of parents had requested "traditional" marking to be maintained in order to see how their children were progressing at school, so marking was reintroduced in secondary schools as of late 2006. In Switzerland, the assessment project set up in the canton of Vaud in the late nineteen-nineties had to return to the possibility of quantified assessment in 2004, once again under pressure from families.

During the teaching reform in the French Community of Belgium, Eric Mangez noted the same extreme sensitivity to the question of marking, giving rise to very different means of implementation according to where the schools were located in the social and education hierarchy (Mangez, 2008). At the bottom of the hierarchy, colour codes of the traffic-light icon (red, amber, green) are used both for the assessment of behaviour with regard to school work (participation, willingness, seriousness, effort, etc.) and for the assessment of cognitive aspects of learning (knowledge acquired). At the top of the hierarchy on the other hand, pressure is strong (in particular from parents) for quantified assessment to be maintained, and a certain refusal, including from pupils, of formative assessments "that do not count", is to be observed (with the issue of formative assessment coming down to whether pupils are marked or not).

But certain researchers regret that evaluators in the education system who are trained in teaching by objectives work out grids which aim to limit the arbitrary character of their decisions and to make the marks allotted incontestable. In doing this, these grids break up the various constituent parts of competencies, at the risk of completely denaturing the integrative approach to them. Other marking practices can be identified but have a hard time getting established in classrooms, regrets (Dauvisis, in Hatano & Lemaître, 2007).

In France, the experimental school Clisthène (Bordeaux) has, in contrast, succeeded in combining the use of traditional marks for knowledge with assessments of competency levels (symbolised by traffic light colours), to determine whether some know-how has been acquired, is in the process of being acquired or has not been acquired (Cédelle, 2008). The originality of the experiment can undoubtedly also be seen in the fact that each family receives a particularly dense individual end-of-term report (4 pages), mixing general assessments, marked "out of 20" and detailed assessments competencies, themselves broken down (always using the colour code).

It can therefore be seen that the question of marking remains a sensitive one, in spite of the profusion of studies each trying to outdo the other in underlining also its precariousness and lack of objectivity, related to the inevitable subjectivity of the teacher's judgement. The teacher's judgement does not just mean how pupils effectively perform, as many studies in the field of social psychology show (Bressoux & Pansu, 2003). Other elements come into play. This is the case for certain characteristics like social origin, or the kind and degree of learning difficulties experienced by the pupil. No matter how full of "good" will the teacher might be, he does not assess the child of a clerk in the same way as that of an executive, nor girls in the same way as boys, nor a pupil doing his year over again in the same way as the "good" pupil. As the purely anonymous assessment is, at the end of the day, rare, all the aspects of a pupil's direct or indirect knowledge play a role in forming the teacher's judgement (and not necessarily in a negative way).

Pierre Merle, who carried out a meticulous survey into the "manufacture" of marks in secondary education (Merle, 2007) also underlines the teacher's subscription to the idea of stability of school competencies from one year to the next: the bad pupil is doomed to remain so, the brilliant one to go from success to success, the one whose big brother was "poor" will probably tread the same path, etc. According to the researcher, these "evaluative stereotypes" are to be found at work in all the experiments that aim to check corrections.

Such a judgement is also influenced by the context of the class: the "stronger" the latter is, for example, the more the teacher's judgement is globally severe. However, these observations are not accidents but recurring and regular phenomena that have little to do with national contexts (Lafortune & Allal, 2008).

In addition, marks are still the fruit of a "do-it-yourself" system, or of an arrangement which is inherent in the very way they are manufactured, P. Merle underlines. He shows how many assessment situations "are illuminated" by examination of previous schooling, school reports and the reputation of schools already attended.

Marks are also an essential tool for firmly establishing the educational authority of the teacher (or his popularity), as well as for punishing or rewarding behaviour in class, over and above the checking of a cognitive attainment, or even for certifying progress throughout the year which "validates" to some extent the teaching work carried out.

Lastly, there is obviously the arbitrary fixing of top and bottom marks, seldom related to any sort of performance logic, but often justified by personal preferences: psychological threshold, support for motivation, maintaining high requirements, etc. In all cases, refusing to give "extreme" marks is used to define the relationship and the educational authority. This phenomenon is close to what the mathematician André Antibi has, since 1983, been calling “la constante macabre” and which is now well-known as the constant distribution of good and bad marks on either side of the average, whatever the "real" performance of the pupils.

In spite of this, P. Merle proposes, rather than just giving up marking, a continuous effort to improve techniques so as to ensure greater educational justice because, he feels, anonymous tests such as those of the baccalauréat are fairer than continuous assessment, which is more sensitive to the bias induced by the teacher's educational judgement.

Assessment as a technique for steering the curriculum

While, from a naive viewpoint, assessment is content to record or to give official recognition to what has been learnt, much work has shown that in fact educational contents and syllabuses are largely influenced and even steered by the way they are assessed.

Everyone has seen how pupils, in our contemporary school systems, tend to concentrate their efforts according to the weighting given to contents in the examination (especially differential coefficients) and how difficult it is to get a course "with no assessment” taken into account. This is much-denounced quirk of the reification of assessment in education, which results in evaluating what is measurable rather than measuring what has value (Broadfoot, 2007, p. 155).

Beyond polarising pupils' interest, assessment globally guides teaching, including its objectives, its strategies and even the tasks used to develop various skills, which are derived from anticipating the assessment. From a survey into the teaching of English as a second language in India, for example, a researcher has shown that the nature of the assessment at the highest levels of the learning cycle is crucial in forming teachers' perceptions as to what it is important to teach and assess, and as to the skills and competencies which are to be developed and encouraged (Agrawal, 2004).

The effects induced can be full of consequences, when a system gives centre stage to pupils' results in standardised assessment tests, as has been the case in certain Anglo-Saxon countries since the end of the twentieth century. Far from providing the expected progress in terms of learning, this focusing on tests has, on the contrary, produced tension and especially a deformation of teaching reducing it to an intensive preparation for the tests, or “teaching to the test”, as the comparative study of the processes related to the tests in 18 American states shows (Berliner & Amrein, 2002). In her talk at the Congress of the American education researchers association in 2007, its chairman, Eva L. Baker, wondered about the need to set up a system of "qualifications" which takes better account of personal development and what is actually learnt, side by side with the performance objectives pursued by the present tests (Baker, 2007). E. Baker was then also director of the important National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Since the nineteen-eighties there has been, an increasingly widespread request for the education system to be publicly, regularly and at all levels, accountable for its performance; this has reinforced pressure on the production of standardised indicators, by means of large-scale examinations or tests, and led to the proportional polarisation of the education system on this summative high-stake testing.

From this point of view, two British authors propose an interpretation of the reforms of the English primary school from an assessment of several pieces of research undertaken jointly from 1989 to 1997 (Broadfoot & Pollard in Lauder, Brown, Dillabourgh & Halsey, 2006). They believe, via the imposition of new assessment practices, that the performance model has gradually invaded primary schools, putting an end to their previous traditions of liberal and progressive education In place of assessment based on personal development and diffuse criteria, the performance model has substituted assessment based on production and explicit and quantified criteria (levels, targets, standards, league tables, value added, etc.).

In the eyes of researchers, this model has encouraged an increasing amount of tension in schools, the climate of competition going hand in hand with an increase in violence and social polarisation against a background of less solidarity. These effects of the performance model may finally prove to be counter-productive in the long term, including economically, because they are contrary to the ideology of a lifelong knowledge society, insofar as the satisfaction of short term indicators discourages longer-term learning, particularly for those who fail in their initial schooling.

At any event, assessment systems play a central role in changing the rules which govern educational ideas and practices, while making it increasingly difficult to even express an alternative conception of desirable educational objectives, Pollard and Broadfoot conclude.

Assessment systems can also be found which oscillate between an assessment used as assistance for learning and personal development, and assessment directed towards performance indicators, such as the SIMCE (Sistema nacional de medición de resultados de aprendizaje del Ministerio de Educación) in Chile (Cariola & Meckes, 2008). The SIMCE, created under the military government, aimed at proposing an instrument for measuring the quality of schools, to improve school competition with a view to developing an educational market. During the first years of the democratic government, from 1990 to 1995, the system was used on the contrary to describe inequalities between schools in order to guide compensatory public policies and to help teachers to improve their results. During second half of the nineteen-nineties the system was used more as a tool for controlling educational policy; schools' results were published and there was an impact on teachers' premiums and allowances for schools. Since the beginning of the 21 st century, the SIMCE has been tending more markedly towards accountability in terms of obtaining results from standardised national education, leaving to one side the initial personal development and pupil integration indicators, in favour of results-based performance alone.

  

Formative assessment or assessment for learning

From feedback to interactive regulation

Formative assessment concentrates most of current thinking on assessments carried out during learning and for learning (by contrast with assessments of learning), as can be seen, for example, from the publication of the work of the CERI-OECD on the question (CERI, 2005).

The concept of formative assessment was introduced by Michael Scriven in 1967 and then popularised by Bloom in 1971, when he integrated it into his model of the "mastery learning" (Allal & Mottier-Lopez, in CERI, 2005). At the time, formative assessment and summative assessment differed primarily by their objectives or the uses to which they could be put: to summative assessment, which indicates whether certain pieces of knowledge have been acquired by the pupil, formative assessment adds feedback, which involves providing information about the degree of acquisition and mistakes made, in order to repeat, to look further into, or correct the learning as a result.

Certain researchers believe that this original theory is still valid and that it would be not very appropriate to artificially make formative assessment autonomous, or even create a false opposition between "summative" and "formative" assessment (Taras, 2005). Others prefer to underline the modifications of the concept over the last twenty years or so, because while formative assessment was initially integrated into a linear type "learning - assessment - corrective measure" process, it gradually formed part of a conception of the interactive regulation of learning, in which guidance of the teaching situation is continuously adjusted in keeping with feedback from pupils (Scallon, in Gregoire, 2008).

This idea of regulation was developed, according to Linda Allal, specifically to indicate the mechanisms which provide guidance, control and adjustment of cognitive, affective and social activities, together with the way these are articulated (Allal, in Allal & Mottier-Lopez, 2007). Cognitive psychology also took an interest in the process involved in understanding the mental operations behind successes or failures, identifying problematic "preconceptions", and therefore going further than conventional behaviourist-type analyses (Gregoire, 2008).

Starting out from socio-cultural theorising, researchers are also attempting to better distinguish the many practices associated with formative assessment, with the acknowledged aim of going beyond the surface effects of labelling and taking the approach aimed at making pupil the subject more than the object of education to the very end (Pryor & Crossouard, 2008).

An investigation by Eleanor Hargreaves (2005) had indeed shown that English teachers have different conceptions of learning, behind a common, avowed subscription to formative assessment. For some, this was in fact a type of assessment that was quite comparable with conventional measurement, while for others it was a process of co-investigation. As, in addition, the growing number of standardised tests does not encourage a very complex conception of formative assessment, Pryor and Crossouard attempted, on the basis of research tested with practitioners, to bring out possible practices that count as true formative assessment. This led them to collect a set of practices around two ideal, typical formative assessments. On the one hand, we have the convergent model, inspired largely by behaviourism (stimulus-response), in which the feedback primarily aims at giving the signal for the right answer to pupils likely to know how to use these indications thanks to their cultural background. On the other, we have the divergent model, where exploratory, informational or even provocative feedback aims to engage the pupil beyond simply correcting his mistakes, to help the teacher and the pupil to better identify what knowledge is at stake, the related misunderstandings and prerequisites, in order to build learning on a collaborative basis.

It is this second meaning which is at the heart of work by Black and William, from their famous review of literature (1998) to the experimental research which they have been performing for twenty years. KMOFAP (King' s-Medway-Oxfordshire Formative Assessment Project) is above all a major stage in the description of the impact of formative assessment on pupils performances and consecutively in the dissemination of practices. At the conclusion of this experiment, four refined types of practices were identified (Black & William in CERI, 2005):

  • improving the interactive dimension of oral feedback so that dialogue with the class is richer and more useful. This mainly involves lengthening the response time granted to pupils, a serious treatment of their answers so as to better identify gaps in their knowledge or wrong ideas, teacher questions and answers that are more centred on pupils needs, and improvement of their understanding;
  • stressing the comments, in the written feedback, to the extent, in certain cases, of dispensing with marking, so that the mark does not subordinate the comments, which is what usually happens;
  • using peer-assessment and self-assessment so that pupils can better understand the objectives to be attained behind the tasks to be carried out, and the means to attain these objectives, by appropriating the assessment criteria of success;
  • making use of summative assessment tests for formative purposes, asking pupils to match traffic lights to subjects which will be tested, in order to set up strategies for preparation or to draw up the exam questions themselves so as to acquire a better vision of the subject.

These initial results have contributed to the popularity of the concept in Anglo-Saxon countries, and in particular led to the adoption in 2004 of assessment for learning as one of the key themes for the Key Stage 3 British national strategy and to the launching of the help programme Assessment is for Learning (AifL) in Scotland, also in 2004. In England today, assessment for learning is an integral part of the Personalised Learning programme now being made compulsory.

Between self-assessment and peer-assessment: what synergies are available?

The influence of assessment on pupils' engagement in learning is no longer open to question. As of the late eighties, the work of Crooks (1988) showed how assessment “guides their judgement as to what is important to learn, affects their motivation and self-perceptions of competence, structures their approaches in timing of personal study, consolidates learning and affects the development of enduring learning strategies and skills”.

With the exponential growth in learning opportunities in the digital world of today, the need to control learning and assessment requires a "durable" approach: to be able to manage ones relationship with knowledge in a formal, educational context and in a more informal social environment, in other words to be able, on a lifelong basis, to evaluate ones strengths and weaknesses before deciding on the next stage.

To subscribe to this viewpoint is to recognise, along with P. Broadfoot (2007, p. 135-136) that “Self-assessment, therefore, is not really just an assessment practice; it is actually a learning activity. It is a way of encouraging students to reflect on what they have learned so far, to think about ways of improving their learning and to make plans which will enable them to progress as learners and to reach their goals. [...] As such it incorporates the skills of time-management, action-planning, negotiation, interpersonal skills, communication - with both teachers and fellow students - and self-discipline in addition to reflection, critical judgement and evaluation”. At any event, this interdependence between learning and assessment makes their joint planning all the more crucial, and pleads in favour of a greater involvement of pupils in the assessment process, in connection with the development of a formative approach (CERI, 2005). In the case studies presented in the work, this participative dimension may take several forms, use various techniques and intervene at various times in education. Certain regular practices can however be observed:

  • recourse to feedback learning: the teacher does not provide an answer, but suggests leads to encourage pupils to continue their learning;
  • drawing up a repertory of learning strategies related to handling complex concepts, with the assistance of conceptual cards, for example, to allow pupils to take stock of already acquired knowledge and prioritise that which still remains to be acquired;
  • developing self-assessment competencies, that sooner or later will require confrontation between the pupils assessments and those of the teacher;
  • developing the role of pupils in peer assessment: promoting constructive criticism using criteria-based grids for example, and tutorials between pupils using rubrics (marking tools associating a points scale with quality standards).

Beyond the simple descriptive level, the way in which these classroom practices influence both school experience and the results of learning remains somewhat unclear. How is it that pupils' involvement in assessment, and in particular the fact of encouraging self-assessment and peer assessment is beneficial, and in what conditions?

Black and William's research into self-assessment (in CERI, 2005) is rooted in the work of Sadler (1989), showing that understanding learning objectives, and consecutively evaluating the efforts required to attain them, are at the heart of learning. In this self-assessment approach the assessment criteria therefore need to be clear : by managing to build a comprehensive view of the task to be performed and its underlying objectives pupils can develop sufficient metacognitive competencies to better manage and control this same task (White & Frederiksen, 1998).

But the construction of this overall picture is not a foregone conclusion. Weeden and Winter have shown, within the framework of the LEARN project, that pupils' representations of assessment were characterised by a high degree of dependence with regard to teachers' practices and by a lack of perspective: while they generally understand what is expected of them during a particular task, they are not able to fit this particular task into the big picture. In other words “they need constant signposting to help them on their journey of learning” (Weeden & Winter, 1999).

According to Black and William, peer assessment may encourage the construction of this big picture by offering a stimulating framework for introducing self-assessment: it “contributes to energizing the learning framework, helps pupils develop their social aptitudes, and prepares them for self-assessment” (CERI, 2005, p. 69). Verbal interactions between pupils authorise criticisms different from those given by the teacher and formulated in a language that is naturally more usual, even though pupils play at taking on the role of the teacher. Moreover, it seems that the pupils more easily appropriate the assessment criteria when they examine a work which is not their own.

The social competencies essential for a peer assessment activity to run smoothly may, however, not naturally be called up wittingly. In other words, to be effective, peer assessment must be guided, or even learnt. One of the activities frequently mentioned is that of marking homework using traffic lights: each pupil draws on his own exercise book the traffic light that corresponds to the degree of confidence which he has in his performance. In class, the pupils who drew a green or amber light work together to assess or help each other, while the teacher concentrates his attention on those who are the least sure of their performance (red light).

But overall, the "evidence" in favour of the positive impact of peer assessment is not very abundant; the fact that these practices are or are not widespread (formally) is not attested either. When examining peer tutorials, A. Baudrit noted that the most efficient control used by pupils seem to be the result of a certain asymmetry between the tutor and the tutored (Baudrit, in Allal & Mottier-Lopez, 2007). In other words, there needs to be a big enough difference to justify a complementary relationship which ensures a true confrontation of learning approaches and avoids the blur and uncertainty in which too-homogeneous pairs are likely to be plunged. Conversely, too wide a variation is likely to make turn the relationship in a situation of expertise, with the tutored becoming too passive, a situation that is less favourable for shared learning. The ideal might be a tutorial carried out "without telling each other".

The recent literature review carried out by Sebba et al. (2008) for the EPPI-Centre of the Institute of Education in London confirms this lack of empirical research on peer assessment, from the simple fact that the 26 studies selected in the analysis are mainly prior to 2000. The benefits, indistinctly associated with self-assessment and peer assessment, are primarily perceptible in pupils' engagement in learning and developing self-esteem, with, to a lesser extent, a certain impact on school results. This type of approach is encouraged by a class culture that fosters dialogue between teachers and pupils and by a move from a dependent to an interdependent relationship : the teacher then adjusts his teaching according to the feedback from the pupils (and vice versa). Checking the pupil on the assessment process does not seem to clearly provide any benefit, although pupils' participation in developing the assessment tests is regarded as important.

For Nicol and MacFarlane-Dick (2006), the distinction between self-assessment and peer assessment is an artificial one. Both are ultimately part of self-regulation at the heart of learning, and the question to be asked is more about feedback (interne and external) in formative assessment. They believe that the widely-held transmission view of feedback, requires calling into question. Firstly, because the teacher is not solely responsible for a process in which the pupils are supposed to become aware of what they are learning and because at any event the workload for the teacher in this case is not acceptable; and secondly because the fact of transmitting feedback cannot in oneself be enough to produce any regulative action, if the psycho-social dimensions are not taken into account in one way or another. The teachers' activity in this common transmission view must therefore be lessened, in favour of the active and central role which the pupils have to play, concentrated on the following seven principles, according to which the teacher who develops good practices as regards feedback:

  • helps clarify what good performance is (goals, criteria, expected standards);
  • facilitates the development of self-assessment (reflection) in learning;
  • delivers high quality information to students about their learning;
  • encourages teacher and peer dialogue around learning;
  • encourages positive motivational beliefs and self-esteem;
  • provides opportunities to close the gap between current and desired performance;
  • provides information to teachers that can be used to help shape the teaching.

From self-regulation to learning to learn

The Scottish ministry of education's programme Assessement is for Learning (AifL) distinguishes three types of relationship between assessment and learning (assessment for learning, assessment as learning, assessment of learning), thereby identifying learning-to learn as one of the key components of the programme. Implicit in the basic principles established by the OECD, this focus on learning to learn is also one of the 10 principles of assessment for learning defined by the Assessment Reform Group (ARG) in 2002 : “assessment for learning should focus on how students learn”.

While the very foundations of formative assessment relate to this "bet" that pupils must be able to assess and revise their own work, and so identify the next stages of learning, what should really be understood by "learning to learn" is not clear. Getting pupils to take part in the diagnosis of their own style of learning, for example using Howard Gardner's theory of multiple intelligences may help to achieve this objective; using the traffic light technique to make it possible for each pupil to diagnose his strengths and weaknesses also plays a part in developing awareness of learning to learn (CERI, 2005). But the prospects offered by the promotion of mere techniques are limited. And self-regulation should not be taken for granted, as shown in the work of Zimmermann and Schunk, quoted in particular by Nicol & MacFarlane (2006).

How can this learning to learn be taught and measured? What challenges does the acquisition of such a competency at school raise, and what relationships can one establish between learning to learn and assessment for learning? The researchers involved in the project Learning how to learn - in classrooms, schools and networks (LHTL) attempted to answer all of these still largely unexplored questions, by setting themselves the goal of producing a teaching model for learning to learn that can be used directly by teachers and pupils.

In the special issue of the review Research Papers in Education devoted to the project (2006), the article by D. Pedder (Organizational conditions that foster successful classroom promotion of learning how to learn) shows that the introduction of this dimension of learning to learn in classes is guided unevenly by three objectives: making learning explicit, promoting pupil autonomy, and watching over the progress of school results. The help provided by the LHTL project team allows a progressive reduction in the importance of performances, in favour of a more obvious focussing on the promotion of autonomy, making teachers' arbitrations less hampered by school syllabuses.

The methodology cannot however be generalized, as P. Black et al. have shown in another article in the same issue (School pupils' beliefs about learning). Designing an instrument for measuring the acquisition of metacognitive competency did not in fact turn out to be possible: the bias inherent in the analysis of pupils discourse (mutually contradictory answers or ones that are considered to be too compliant with what is expected, etc.), the difficulties encountered in translating their practices in an abstract way or in analysing their own engagement in connection with their school environment, the complexity of the parameters to be taken into consideration to design a reliable generic measuring instrument, mean that is highly unlikely that it will ever be used in schools.

One of the main conclusions of this project, which directly echoes those of the contemporary project Sustainable thinking classrooms, is therefore the necessary contextualisation of metacognitive competency. In other words, developing a teaching approach for learning to learn would involve embedding in the disciplines: teaching (and measuring) learning to learn, is teaching (and measuring) learning to learn something.

  

Feedback from pupils to regulate teaching?

Via outside consultations?

There is a certain amount of mobilisation in Anglo-Saxon countries in favour of the pupil as an appraiser, who takes part not only in his own assessment, but also in that of the teachers and the services which the school provides. The digests of the minister of education on the topic pupil voice in England, the SoundOut programme in the United States, and the fortnightly review of Australian good practices Connect are just some examples of this movement, frequently publicised by the expressions “pupil voice” or “student voice”.

But this question of pupils' feedback did not arise with the recent democratic concerns related to children's rights, asserted in movements of the “student voice” type. With the democratisation of higher education in the 1980s and the rise of competition between educational institutions, questions about the adoption of a Quality approach, and, as a corollary, about the methods to be used to measure students' satisfaction are emerging (Harvey, 2001).

Today national surveys are carried out to find students' points of view on their entire university experience: the National survey of student engagement (NSSE) in the United States, the Course Experience Questionnaire (CEQ) in Australia and the experimental English Student Listening Programme (SLP) for example; other dsurveys, such as the National Student Survey (NSS) in the United Kingdom or even the Australian Student Course Experience Questionnaire (SCEQ), are more centred on teaching experience, in particular in the first university cycle.

But the fundamentals of such an approach are not at all easy. The way in which information is collected and its frequency (twice a year, yearly, biannual, etc) will necessarily impact how the data is processed later. And the way in which the data is made public also serves various stakes more or less explicitly. (Brennan & Williams, 2004  ; Gordon, 2005). Faced with these tensions between a Quality approach, free competition, the obligation to achieve results and revision of the curricula, can the transition from “self-assessment” (the pupil is self assessed) to "self-evaluation" (the school is self-assessed), contribute to changing teaching practices in the long term? In other words, can one really identify the influence of these consultations on class practices, and on pupils' attitudes and performances? And while it is not enough for the benefits to be tangible, for pupils to give their points of view and for teachers to take note of them, under what conditions does this approach prove effective?

One of the first doubts which it seems possible to clear up relates to the reliability and the relevance of students' feedback, as shown in the literature review of Prebble et al. devoted to assessment of the support services offered to students in the first cycle. Investigating the impact of students' feedback on the overall quality of their lessons, the authors arrive at the following conclusion: “student assessments [of teaching] are among the most reliable and accessible indicators of the effectiveness of teaching. When used appropriately they are likely to lead to significant improvements in the quality of the teaching and student learning” (Prebble, 2005, p. 41). So although the overall relevance of students' assessments is unquestionable, it is important that they should be only one indicator out of many and that the way in which the data collected are exploited and presented to the teachers is dealt with carefully, in order to avoid any misunderstanding. Although the amount of research today makes it possible to understand the advantages and disadvantages associated with the various methods of data collection (Richardson, 2005 ; Hoban & Hastings, 2006), scientific work concerning the output, i.e. the ways the results that should (or should not) stimulate regulative action are presented, are definitely less clear.

Similarly, while certain experiments give food for thought as to the best methods for closing the loop, the conditions for widespread development to all higher educational institutions, do not (yet) obtain (Brennan & Williams, 2004 ; Gordon, 2005). There too the question of appropriateness between means and ends is crucial, because the approach is necessarily not the same if the outlook is summative or if it is formative; in other words whether the goal is to draw up an inventory at a given time of the degree of pupil satisfaction in order to evaluate the overall quality of the teaching, or if it is precisely a question of promoting a regulative approach that then requires greater integration into the teaching (Gordon, 2005). And these two options do not necessarily have the same impact on teaching practices, whether in higher education or secondary education.

Everyday learning?

In England, the pioneering work of Jean Rudduck within the framework of the project Consulting pupils about teaching and learning (2001-2003) highlighted the reciprocal benefit of pupils' feedback, provided this is not just a purely demagogic move. The results give an account of an increased pupil engagement in the educative community, of greater confidence in them and of a better grasp of their learning identity; teachers for their part see their representations evolving and feel stimulated to look anew at the way they work. In the work published with J. Flutter (2004), the authors consider pupil participation as the cornerstone of assessment measures, opening up prospects for progression to both pupils and the school.

The experiment carried out by McIntyre et al. (2005) with approximately forty pupils entering secondary school shows coherent results on several points: for the pupils, the expression of a relative consensus on what would them help in their learning, focused in particular on the social dimension of learning (more interactivity, more group work and more authenticity), and suggestions greatly inspired by what they know already ; for teachers, a greater divergence in taking into account these suggestions, although considered as relevant, revealing comfortable and uncomfortable learnings for teachers, and a more-or-less long-term commitment to overhaul their practices.

The survey carried out in Northern Ireland on pupils of AfL classrooms within the framework of the project CPAL (Consulting Pupils one the Assessment off to their Learning) provides a further convergent illustration (Leitch & Odena, 2007). According to the authors, this pupil voice may be broken up chronologically into four factors: it must have the opportunity of expressing a point of view (space), this voicing must be facilitated, encouraged (voice) and benefit from the attention of a public (audience), before being suitably taken into account (influence). According to the conclusions of the survey, pupils recognise that opportunities of expressing themselves exist (space and voice), but remain more circumspect as to the other two factors, believing that their feedback is not sufficiently listened to and almost never acted upon. The teachers for their part find the feedback from the pupils useful, in particular for better understanding their own practices. But they consider the space-time organisation of the school, the requirements of the curriculum and the obligation to achieve results as standing in the way of effectively taking into account this "pupil voice". In addition, the need to control the class, the lack of hindsight in situation, the school culture and the type of subject taught are also dissuasive factors standing in the way of regulating teaching.

To some extent embedding pupil consultation into the teaching routine from the regulative point of view raises a certain number of questions that remain unsolved by the experiments reported. Hoban & Hastings (2006) have shown that out of the four data collection methods examined, the student interview is most likely to influence the teaching practices of the teacher. Such a mode of pupil consultation not only does not make it possible to listen to all the voices, but would seem to be not easy to develop on a widespread basis. Should it be considered that the regulation hypothesis is badly stated and that this pupil voice, in particular in view of the restraints related to the potential competition between programs and teaching, can benefit only from being taken into account in a non-formal way? We are compelled to conclude, along with Michael Fielding, in an special issue of the review Discourse: Studies in the Cultural Politics of Education (2007) devoted to the series of seminars entitled Engaging Critically with Pupil Voice (2004-2006), that the emerging nature of this scientific field hardly makes it possible to provide clear answers for the moment.

  

The assessment of competencies

What assessment for what competency?

In the usual vocabulary of assessment, the term competencies can be used to qualify knowledge or know-how as well as to indicate a complex and integrated approach which is contrary to the idea of breaking down into detailed and fragmented objectives.

From the national "diagnostic" assessments of in primary school and for entry into the first year of secondary school, Sophie Morlaix has identified "competencies" for academic success (mental arithmetic, for example), bringing out the groups of response items that best predict the pupil's success (Morlaix, 2007). While these groups do indeed make it possible to identify elementary learning competencies over and above the disciplinary knowledge, they have little to do with complex "competencies" targeted by other researchers or reforms. On this subject, it may be said that the French Common base of knowledge and skills mixes the two conceptions, and that assessment via the "livrets de compétences" remains problematic through lack of clarification of what one understands by "competencies", and of a clarification of the functions which one would like to see used in various assessment tools (Houchot & Robine, 2007).

Without going into this general issue of the competency-based approach at greater length - it was dealt with in a previous INRP document (Rey, 2008) - we will look at the relationships between competencies and assessment, considered in its most "complex" version. The assessment period is the most problematic aspect of the competency-based approach, since what is involved in this case is to evaluate, not just school knowledge in a school situation, but to mobilise resources, skills, competencies, etc. in an unusual situation that is sometimes close to "real" life.

For Scallon, one of the specialists in the assessment of competencies (see his teaching site), it is necessary to distinguish the knowledge level assessment situation, which has to do with memorising information, and the skills level situation, in which the individual must use his knowledge to work out an answer that he has not learn by heart beforehand (Scallon, 2007, p. 33-46), the knowledge situation of including the two preceding ones.

A knowledge situation is characterised by a direct request for information that the pupil must have memorised and must have in his cognitive repertory. In a skill situation, the pupil must ask himself which knowledge to use, but the subject of the question and the field into which it fits are known (an identified school situation), whereas in the competency situation, it is necessary to identify the type of fields and relevant knowledge and skills to be used to answer a question put in terms that are not necessarily scholastic.

The notion of distance (temporal and conceptual) between the learning situation and the task requested is therefore of importance in appraising the assessment of competencies. The question of the development of a integrated assessment process, rolled out over the entire duration of the learning time (and the whole of the field covered) is also considered by Tardif to be paramount. He stresses the fact that the assessment of competencies is contrary to the juxtaposition of a series of "tests" separately designed in isolation from each other. The assessment of competencies must therefore be spread out over time so that its development can be determined (Tardif, 2006).

As for the differentiation between competencies close to elementary knowledge and broader competencies, Belgian researchers who have worked in some depth on the question of the assessment of competencies (Rey, Carette, Defrance & Kahn, 2006) distinguish three levels of which only the last two really deserve, in their opinion, to be called competencies:

  • elementary competency: being able to carry out an operation in response to a signal (an automated procedure or skill);
  • a competency with framing: interpreting an unusual situation and choosing the appropriate elementary competence;
  • a complex competency: choosing and combining several competencies to deal with a new and complex situation.

The aim of the authors is first to evaluate complex competencies, in other words, pupils' ability to choose and combine several of the procedures which they have already "learnt", in order to solve a new problem in an appropriate way. What is required in these "assessment models" is intellectual autonomy rather than automatic responses.

For the assessment to retain its character of learning construction, it should not be only summative but must also be diagnostic. The test must also include a second phase, where the complex task is broken up into basic procedures to be chosen (partial tasks). Finally, a third phase to check mastery of the basic procedures (simple tasks) out of context may be necessary at the end of the process to check at what stage difficulties are encountered.

It is therefore possible to determine whether the pupil is having difficulties with solving the new and complex situation (identifying the problem, choosing and using the relevant procedures), choosing the right procedure or even mastering the basic procedures or elementary competencies.

From this approach and from practical investigations which were inspired from these, V. Carette has put forward the hypothesis that competency-based assessment tests do not evaluate the same type of attainment as more traditional types of assessment centred on the results of objectives-based teaching, which makes one wonder about the relevance of judging the effectiveness of the various teaching approaches without questioning the type of assessment used (Carette, 2008).

Assessing transverse competency: the example of learning to learn

The question of knowing how to develop individual competencies relating to lifelong learning, and therefore of how to measure progress related to these competencies, has been on the European agenda ever since the initial thinking that fuelled the Lisbon strategy in 2000. But to move from a system which evaluates knowledge to a system which (also) evaluates competencies implies a change of viewpoint which places those involved in education systems in difficulty. This tendency to want to articulate what is traditionally the province of the educational establishment (hard or cognitive outcomes) with explicit stakes related to the personal, social and moral development of the person (soft or affective outcomes), generates further questioning on the respective responsibilities of those involved and the means that must be implemented.

The creation of the Centre for Research on Education and Lifelong Learning (CRELL) in 2005 was a deliberate move by European politicians to make this competency-based approach operational within the education systems. The first work led in December 2006 to the adoption by the European Parliament and the Council of Europe of a recommendation which identifies the competency "learning to learn" as one of eight key competencies for education and lifelong learning. We should note that the OECD regards it as a transverse competency, making up the three categories of key competencies of its project DeSeCo (2002): “interact in heterogeneous groups, act autonomously, use tools interactively”. Similarly, the Common base of knowledge and skills in France (2006) does not include learning to learn in the list of seven key competencies which it proposes, in spite of an obvious echo with the seventh competency "autonomy and initiative". What interests us at present is not so much the debates on the position of learning to learn in the curriculum, as the proposals put forth by a group of CRELL experts aiming at working out a European test to measure this same competency (Hoskins & Fredriksson, 2008). They believe that the base provided by the questions relative to problem solving in PISA 2003 (abandoned in PISA 2006 and 2009 but possibly reintroduced in 2012) offers only an indirect approach to learning to learn. The ambition of the CRELL test relates to an interdisciplinary and authentic approach, including the emotional, cognitive and metacognitive dimensions of this competency.

To build this formal definition and to design the appropriate measuring instrument, the experts based their work on existing experiments: the Bristol university test Effective lifelong learning inventory (ELLI) for socio-cultural aspects, the Finnish project The L2 factor. Learning-to-Learn at School - A Key to Lifelong Learning at Helsinki university and the Amsterdam university cross-curricular skills test (CCST) for cognitive aspects, and finally work of the faculty of psychology of the autonomous university of Madrid on measuring metacognitive capacities.

At the conclusion of the pilot phase (2300 14-year old pupils in about fifty schools in Europe), the national reports underline the difficulty of considering the emotional and cognitive dimensions of learning separately, and note the cultural differences in the pupils' answers. This preliminary work is directed towards reinforcing the interdisciplinary approach and therefore towards on the one hand, a stronger articulation between a formal and normative definition of competency, and a multiplicity of contexts calling on this competency on the other.

Another path explored in a complementary report by the CRELL (Hoskins & Deakin Crick, 2008), is that of a connection between the indicators of learning to learn and those related to civic competencies, also among the eight key competencies for education and lifelong learning (2006). Such a connection also echoes work on the social impact of education (for example CERI, 2007) which defends the thesis of a correlation between the level of education and social wellbeing, and which results in particular in the emergence of programmes for "educating in." (health, citizenship, etc) in the curricula.

In the CRELL report, Hoskins and Deakin Crick cross-reference conceptual analyses and empirical data to highlight the similarities between these two competencies, and in particular their importance for acting in society: learning to learn to become an active learner and civic competencies to become an active citizen. Regarding both as complementary, they suggest exploring the existing initiatives and measures further, such as for example the work of CitizED, an English community of interests, dedicated to initial and in-service training for teachers on the questions of educating in citizenship.

For in spite of points of convergence with the English project LHTL referred to previously, the work of the CRELL favours a macro-assessment at European level, whose goals - diagnostic, summative or political? - remain dubious.The fact that the micro-levels (the class) and meso-levels (the school) at the heart of the formative approach are not called up here, reopens the question "who evaluates?" and in particular "who evaluates the competencies?". If, within the context of the LHTL project, learning to learn is perceived as part of learning and is therefore the responsibility of the teacher, the CRELL opts for a specific, instrumented measurement, one that is interdisciplinary and stands outside the act of learning: a vision which can be seen as continuing the initiatives of the Helsinki university Centre for educational assessment (ECA) and which repeats the kind of (inter)national tests, that are generally not very likely to take into account the individual behind the pupil nor the learning environment. The only notable exception is undoubtedly the international survey by the IEA on civic education and citizenship - International Civic and Citizenship Education Study (ICCS), inspired by a previous experiment carried out in 1999 (Study of civic education or CIVED) - the first edition of which will be managed in 2009 in forty countries.

This choice of investing in an outside assessment programme seems more to be an answer to political requirements of accountability rather than for instrumentation at the service of learning regulation, even if the markers placed by defining the instrument might be one stage in the development of the competency-based approach. This choice does therefore not rule out writing these competencies explicitly into teaching, in a way such as may be supported, for example, by the work of the Citizenship Education Research Strategy Group of the London EPPI-Centre: the two literature reviews directed in 2004 and 2005 by R. Deakin Crick showed the effectiveness of pupil-centred learning in a school environment which favours both the quality of dialogue and confidence, and teaching based firmly in real life, to develop learning to learn and civic competencies These summaries also highlighted the benefit of this integration in terms of school results, consolidating, at the same time, initiatives such as ELLI whose aim is to build an instrument for measuring individual competencies as regards lifelong learning.

Since 2002, the work of ELLI, coordinated by Ruth Deakin Crick, Patricia Broadfoot and Guy Claxton (Bristol university), has been concentrating on the key concept of "learning power, defined as follows: “a complex mix of dispositions, lived experiences, social relations, values, attitudes and beliefs that coalesce to shape the nature of an individual's engagement with any particular learning opportunity”.

The seven dimensions that make up this capacity to learn (changing & learning, meaning making, curiosity, creativity, learning relationships, strategic awareness and resilience), have themselves made it possible to coarsely differentiate two types of learners: “efficacious, engaged and energised learners” on one hand, and “passive, dependent and fragile learners” on the other hand (Deakin Crick, 2007).

The use of the measuring instrument developed within the framework of ELLI (today made available to schools) has clearly shown that the more the pupils advance in their schooling, the weaker they become in all seven identified dimensions, and in particular creativity; in other words, the more fragile and dependent they become for learning. Using the detailed learning profiles of their pupils, teachers have developed new ways of working in the class, from reorganising the way they teach to focusing on self-assessment practices. The benefits have proved to be quickly measurable: after only two terms, pupils showed themselves more able to withstand the educational game of summative assessment, and more aware of their own learning strategies. According to the authors, the key factors of this change lie both in the teacher's representations and the school climate.

  

The portfolio: a reflective tool at the service of assessment?

An approach rather than a tool

The question of knowing what a portfolio is and what it is used for remain confused for a majority of those involved in education, so clouded are representations by the great many contexts in which they are used (from the primary school to adult education), by the many functions that are assigned to them (formative assessment, validation of attainments, etc.) and by the various conceptual universes that they call upon (cognitive psychology, use of ITC, etc.). France is no stranger to this confusion, where the more or less prescriptive reference to related media bearing various names is to be found in official texts: "livret scolaire" for the primary school, "livret de l'apprenti" or "livret d'apprentissage" for vocational training, "portfolio" for modern language skills, "livret de compétences" for assessment of attainments, "dossier scolaire" for guidance, etc. (Houchot & Robine, 2007).

In the field of education and training, medical studies and teacher training have historically benefited from the longest tradition in the use of portfolios (Butler, 2006), while experiments in secondary education are more recent (see also the Educnet file on the Portfolio numérique and the Cent références pour le portfolio numérique by Robert Bibeau, both updated in 2008).

One of the definitions usually quoted is that of Paulson et al. (1991, quoted by Goupil, 1998) who considers the portfolio as "a significant collection of pupil's work illustrating his efforts, his progress and his achievements, in one or more fields”.

This definition has gradually evolved, in particular under the influence of digital media. That provided by Challis (2005), the author of work on the use of portfolios in medical training, is today commonly used at European level: “selective and structured collections of information; gathered for specific purposes and showing/illustrating one's accomplishments and growth, which are stored digitally and managed by appropriate software; developed using appropriate multimedia and usually within a web environment and retrieved from a website, or delivered by CD-ROM or by DVD”.

Questioning about tools is marked by concerns of durability (standards, interoperability, etc) and of teaching continuity, particular at the key times of transition during the pupil's schooling (Nailsmith and Al, 2006). But according to George Siemens (2004), standardisation is prejudicial to the pupils getting the feel of the system and has more to do with institutional or even political logic, than with a teaching approach. In other words, the use of a simple tool at the beginning (presentation or word-processing software, blog at a first level, or dynamic modular platform of the CMS type at a 2nd level) is essential to speed up the adoption of the approach and to allow the pupil to make his decisions and to keep control on the contents of his portfolio and access to this content.

In his literature review, Butler also insists on the necessity for the tool to be flexible and modular (Butler, 2006), and on the balance to be found between a functionally limited interface and a too-complicated and not very intuitive tool. The report published recently within the context of the European project More self esteem with my e-Portfolio (MOSEP), takes this hypothesis as a starting point and offers a detailed comparison between existing open source portfolio solutions, thereby making it possible to check how suitable they are for initial users (Attwell and Al, 2007). The study by the BECTA (Hartnell-Young, 2007) also highlights this tension between pupils' creativity and the sophisticated architecture of the tool used, and underlines the potential of external social tools outside the formal school environment. In the light of the case studies analysed, the authors stress the importance of the social dimension: the tool must allow for feedback from third parties, teachers and peers (comments, discussion areas, etc.) so that the approach gains in effectiveness. This discourse on the necessary flexibility of the tool is also leading to the emergence of a related concept: that of a personal learning environment, guaranteeing everyone the possibility of filing records of his activities and taking part in building his digital identity, as Stephen Downes, a lecturer invited to the Eportfolio 2008 conference in Montreal explains (Downes, 2008).

The approach associated with the use for teaching of a portfolio is often described in the literature in the form of a cycle, thus echoing the idea of the feedback loop. Helen Barrett (2004), for example, breaks up the process into five phases: collecting, selecting, reflecting, projecting and celebrating. Graham Attwell proposes an interpretation made up of seven functions, corresponding to as many different teaching processes: recognising, recording, reflecting, validating, presenting, planning and assessing (Attwell et al., 2007). In spite of these attempts at modelling it, the digital portfolio is not very clear, seems difficult to grasp and suffers from contradictory explanations. Is it a question of privileging a self-assessment approach and of stimulating pupils' thinking about what they are doing? Does it involve adopting a new tool to validate or certify attainments? Or does it mean using a standardised and interoperable tool on a more widespread basis? The three types of discourse - formative, summative (or even certifying) and technical (or even commercial or political) - seem to operate in fairly watertight compartments, hardly giving practitioners anything concrete to get hold of.

Single or multiple goals?

So what is this portfolio for? The question of mixed goals is at the heart of all the questions: personal development? learning? assessment ? presentation? Can these apparently distinct goals exist side by side on the same medium?

Based on the existing literature, Helen Barrett (2004) concludes that the use of a portfolio for high-stakes assessment of learning differs basically from formative uses within the framework of an assessment for learning. In the first case, the collection of proofs is organised in relation to standardised outside expectations and the objective is to measure what has been learnt within a preset time frame: the appraiser is therefore interested in the product. In the second, information collection, possibly negotiated with the teacher, gives priority to the choice of the pupil, and the objective is to enable him "to tell" about his experiences as a pupil in order to evaluate his coming needs: here the focus is on the process. While a portfolio may serve one or the other of these aims, the author defends the idea that the very concept of portfolio is potentially better suited to a formative viewpoint, and she draws a parallel between the use of a portfolio and storytelling.

Out of these many meanings and uses, Gerard Scallon (2007) also considers that the principal value of the portfolio lies in self-assessment. From this point of view, he distinguishes three types of uses that may correspond to as many markers along the way towards a progressive deepening of a self-assessment approach :

  • the learning file, which collects all the pupil's work together with his comments about himself, and is built up progressively;
  • the presentation file, in which the pupil chooses work according to certain criteria, for example to show progress made or to recount a particular episode of his learning;
  • the assessment file, which corresponds to a selection of the pupil's best work, to provide support for the assessment of a competency, at the end of a period of training.

While the narrative dimension mentioned par H. Barrett can be found in his analysis, Scallon goes further, by postulating an articulation between the formative and summative dimensions of learning. The assessment file in this way is used for institutional assessment, while making a clean break with the tradition of standardised tests. The assessment of the portfolio, by peers and/or teachers, can in addition test the capacity of the pupil to accept external judgement (and therefore to self-assesshimself).

On the other hand, these files must be clearly different from school reports. The Swiss experiment is significant on this point (Gilliéron Giroud, 2007). The learning file, introduced in French-speaking Switzerland in the nineteen-nineties, gradually took on a communication function, to the point of becoming in the Canton of Vaud the only medium transmitted to families that explained institutional decisions (move to a higher class, choice of subjects, etc). This increased number of goals and recipients placed a significant restraint on widespread development of the measure, generating much incomprehension amongst the various parties, and resulted in a return to the marking and averages that had previously been abandoned.

Val Klenowski, the author of one of the rare works entirely devoted to portfolios (2002), expresses reserves similar to those of H. Barrett, when it comes to using a portfolio for certifying teachers. She believes that what is essential in the definition of the portfolio is not so much the collection of work that it contains as “the centrality of student self-evaluation and reflection and the opportunity to portray the processes by which the work in the portfolio is achieved” (p. 3). Illustrating her point with examples taken from a very wide range of contexts (from primary and secondary education to initial and in-service training for teachers and doctors, from Australia to Scotland, via Hong Kong, etc.), the author highlights the correlations between portfolio and assessment for learning. She stresses the existence of a common culture centred on the metacognitive development of the pupil, made up of elements such as pupils' understanding of the assessment criteria, their engagement in the assessment process and the importance of an integrated approach to assessment, going as far as grading of the portfolio. According to her, the portfolio can therefore answer two types of aims: the pupil is obliged to prove what he has learnt, either from the formative point of view (focused on the process of learning), or from the summative point of view (focused on the control of certain competencies), or a combination of both. In other words, the reflexive portfolio can give an account of both the results of learning and the quality of the learning process itself.

Uneven practices

The experiments recorded in the abundant literature on portfolios make it possible to state some invariants in terms of predictive factors for success. First of all the fact that the pupil feels himself to be the owner of his portfolio, i.e. responsible for the collected documents, the way they are organised and presented, is regarded as crucial by a majority of observers. As a corollary, the degree of flexibility of the portfolio is a prerequisite for integrating the approach into teaching as previously mentioned. This flexibility can particularly be seen in the diversity of the information which can be collected to be used as evidence, and in the diversity of the learning strategies which the work with the portfolio makes it possible to emulate; for example, not being limited to just what the pupil says that he has done, by including video records if necessary, by offering several methods for organising and sorting the proofs collected, or even by promoting validation on request, when the pupil is ready (Attwell et al., 2007  ; Butler, 2006  ; Hartnell-Young, 2007  ; Schärer, 2007).

The Belgian experiment with pupils in the first years of secondary education (Bernard & Vlassis, 2007) highlights a greater motivation from pupils and teachers taking part when several disciplines are involved in the project. The fact that the uses to which the portfolios are put are unevenly widespread depending on the professional environments under consideration, leads one to wonder about the potential affinities between certain more experimental or more professional disciplines and the approach induced by the portfolio. The investigations carried out in 35 different discipline areas, within the framework of the ISLE project (Individualised support for learning through eportfolios), show this hypothesis to be invalid, providing evidence that effectiveness is not related to the context of the discipline (ISLE, 2007).

While the approach appears easier to implement at primary school because of the more flexible organisation (Houchot & Robine, 2007), Louise Bélair shows that the organisation of the portfolio by discipline remains dominant; at the most it fulfils a self-regulation function, but does not account for a level of competency and is therefore not used formally in an assessment context. But it is subscribed to by both the pupils and the teachers questioned, regarding the activities generated by the portfolio as a powerful source of motivation. Could it be that its "summative" vocation would intervene only at a later stage? (Bélair, 2007).

Nothing is more uncertain if one refers to the Swiss experiment, which ended up by excluding any formal assessment function of in favour of the metacognition function of the portfolio. Patricia Gilliéron Giroud gives an account of many difficulties encountered by teachers and points to the weaknesses relating to the introduction of portfolios into the classes of French-speaking Switzerland (Gilliéron Giroud, 2007): on top of the confusion generated by its many aims, there is the cumbersome procedure of the process involved in personalising the approach, questions about the validity of the proof collected by the pupil and the difficulty of analysing the records of learning in order to qualify progress.

At any event, the nature and scale of the work by teachers and peers in the process of constructing the portfolio, getting it used on a more or less long-term basis (when is a portfolio finished?) and the way in which it is made public (who are the recipients?) are central to the questions being asked (Butler, 2006). As for all the opened-ended assessment tools, another important issue is to provide “closing keys” likely to reduce the amount of subjectivity or even arbitrariness. In the assessment of competencies by complex situations via portfolios, learners' familiarity of the criteria on which they will be assessed is therefore essential (Gérard, in Baillat et al., 2008).

While many studies tend to show the impact of the portfolio on pupils' motivation and postulate its influence on school results, insufficient light has been shed on the questions of implementation. For in spite of the restraints, institutional regulations exist: widedspread use of the ELP (European Language Portfolio) in Swiss secondary schools by 2010, widespread use of the assessment portfolio for the diploma marking the end of secondary studies in British Columbia, etc. Consequently, how can one create conditions that are favourable to the deployment of a portfolio project? Injunctions to reform school culture, to reorganise syllabuses and timetables, to include the portfolio in learning on a daily basis, etc are not enough.

Certain contexts are obviously more favourable than others: the study by the BECTA (Hartnell-Young, 2007), for example, highlights the strong correlations between e-portfolios and ITC, showing that the digital portfolio is easier to promote in a school where ITC for education exists already. All observers stress, however, the importance of planning of a sufficiently long pilot phase, making it possible to study the actual feasibility of widespread use in classes, to define the centralised or decentralised structures responsible for assisting and providing support for the development of the measure (council, in-service training, etc.) with the teaching staff, and to determine a reasonable schedule and realistic indicators (see especially Schärer, 2007). In this introductory approach, the work carried out at European level may assist with thinking on the subject.

European initiatives: PEL and MOSEP

The European Language Portfolio (ELP), drawn up by the Council of Europe, undoubtedly incarnates one of the most extensive portfolio experiments on a European scale. The ELP was launched in 2001 during the European year of languages and is being developed in twenty-eight member states. By late 2008, it included nearly a hundred validated models, covering all sectors of teaching, from primary school to adult education (see also the case studies collected by David Little in 2003).

The interim report 2006 (Schärer, 2007) reported significantly positive results both as an educational tool to help the development of autonomy and as a presentation tool testifying to language skills. But experiments in progress have not yet reached a sufficient level of maturity to make it possible to really analyse the impact of the portfolio on the results of learning. The fact that the ELP develops the "successes" from a more or less long-term point of view with and in a framework that goes beyond the school horizon greatly contributes to pupils subscribing to it.

Another key characteristic of the ELP is that it explicitly takes into account all pupils' language competencies, "whether they are learnt or acquired within the framework of the official education system or outside of it". This recognition of what the pupil knows, independently of the school space-time, is also a basic component of another European project, the MOSEP (2006-2008) directed by the Salzburg Research Forschungsgesellschaft (Attwell et al., 2007). This project, entitled More self-esteem with my e-Portfolio proposes a toolbox and a set of tutorials guiding teachers and careers advisors towards implementing a portfolio with teenager dropouts at the end of compulsory schooling. The methodology used is rooted in the work of Bandura and Pajares on teenage psychology. The objective therefore is clearly to implement personalised assistance for the pupil, via the use of a portfolio, to increase his motivation, counting on his self efficacy and his ability to use self-regulated learning. The Scottish project ISLE - Individualised Support for Learning through e-Portfolios - supports the founding hypothesis of MOSEP, showing that the portfolio can be more effective when it is associated with critical incidents during the learners' development (ISLE, 2007).

  

Bibliography

  • (2007). « Beyond "Voice": New roles, relations, and contexts in researching with young people ». Discourse: Studies in the Cultural Politics of Education, vol. 28, n° 3, p. 301-420.
  • (2007). ISLE: Individualised support for learning through e-Portfolios. Joint Information Systems Committee (JISC).
  • (2006). « Learning How to Learn, in Classrooms, Schools and Networks ». Research Papers in Education, vol. 21, n° 2, p. 101-234.
  • Agrawal Manta (2004). « Curricular reform in schools: the importance of evaluation ». Journal of Curriculum Studies, vol. 36, n° 3, p. 361-379.
  • Allal Linda (2007). « Régulation des apprentissages : Orientation conceptuelle pour la recherche et la pratique en éducation ». In Allal Linda & Mottier Lopez Lucie (dir.). Régulation des apprentissages en situation scolaire et en formation. Bruxelles : De Boeck, p. 7-23.
  • Allal Linda (2008). « Évaluation des apprentissages ». In van Zanten Agnès (dir.). Dictionnaire de l'éducation. Paris : Presses universitaires de France, p. 311-314.
  • Allal Linda & Lafortune Louise (dir.) (2008). Jugement professionnel en évaluation : Pratiques enseignants au Québec et à Genève. Sainte-Foy : Presses de l'université du Québec.
  • Attwell Graham, Chrzaszcz Agnieszka & Pallister John et al. (2007). Grab your Future with an E-portfolio: Study on New Qualifications and Skills Needed by Teachers and Career Counsellors to Empower Young Students with the E-portfolio Concepts and Tools. Salzburg : Salzburg Research Forschungsgesellschaft.
  • Baker Eva L. (2007). « 2007 Presidential Address-The End(s) of Testing ». Educational Researcher, vol. 36, n° 6, p. 309-317.
  • Barrett Helen (2004). « Electronic Portfolios as Digital Stories of Deep Learning ». Using Technology to Support Alternative Assessment and Electronic Portfolios. En ligne (consulté le 13 novembre 2008) :
    < http://electronicportfolios.org/digistory/epstory.html >.
  • Baudrit Alain (2007). « Tutorat entre pairs : Les processus de régulation mis en ouvre par les élèves ». In Allal Linda & Mottier Lopez Lucie (dir.). Régulation des apprentissages en situation scolaire et en formation. Bruxelles : De Boeck.
  • Bélair Louise (2007). « Le dossier de l'élève (portfolio) dans l'approche par compétences au primaire au Québec : Quels apports ? quelles perspectives ? quelles limites ? ». In Actes du 19e colloque de l'ADMEE-Europe, Université du Luxembourg, 11-13 septembre 2006.
  • Berliner David C. & Amrein Audrey L. (2002). « High-Stakes Testing, Uncertainty, and Student Learning ». Education Policy Analysis Archives, vol. 10, n° 18.
  • Bernard Hervé & Vlassis Joëlle (2007). « Le portfolio au service de l'enseignement des compétences transversales ». In Actes du 19e colloque de l'ADMEE-Europe, Université du Luxembourg, 11-13 septembre 2006.
  • Black Paul & Wiliam Dylan (1998). « Assessment and Classroom Learning ». Assessment in Education, vol. 5, n° 1, p. 7-68.
  • Brennan John & Williams Ruth (2004). Collecting and using student feedback: A guide to good practice. Bristol : Higher Education Funding Council for England (HEFCE).
  • Bressoux Pascal & Pansu Pascal (2003). Quand les enseignants jugent leurs élèves. Paris : Presses universitaires de France.
  • Broadfoot Patricia & Pollard Andrew (2006). « The Changing Discourse of Assessment Policy: The Case of English Primary Education ». In Lauder Hugh, Brown Phillip, Dillabough Jo-Anne & Halsey A. H. (dir.). Education, Globalization & Social Change. New York : Oxford University Press.
  • Broadfoot Patricia (2007). An Introduction to Assessment. London : Continuum.
  • Butler Philippa (2006). A Review Of The Literature On Portfolios And Electronic Portfolios. Palmerston North, New Zealand : Massey University College of Education.
  • Carette Vincent (2008). « Les caractéristiques des enseignants efficaces en question ». Revue française de pédagogie, n° 162, p. 81-93.
  • Cariola M. Leonor & Meckes Lorena (2008). « L'évolution du rôle du système d'évaluation : Le cas du Chili ». Revue internationale d'éducation - Sèvres, n° 48, p. 37-51.
  • Cédelle Luc (2008). Un plaisir de collège. Paris : Le Seuil.
  • Centre for Educational Research and Innovation (CERI) (2005). Formative Assessment: Improving Learning in Secondary Classrooms. Paris : OECD.
  • Centre for Educational Research and Innovation (CERI) (2007). Understanding the Social Outcomes of Learning. Paris : OECD.
  • Challis Diana (2005). « Towards the mature ePortfolio: Some implications for higher education ». Canadian Journal of Learning and Technology, vol. 31, n° 3.
  • Cloutier Martine, Fortier Guy & Slade Samantha (2006). Le portfolio numérique : Un atout pour le citoyen apprenant. Québec : Société de formation à distance des commissions scolaires du Québec (SOFAD).
  • Crooks Terence J. (1988). « The Impact of Classroom Evaluation Practices on Students ». Review of Educational Research, vol. 58, n° 4, p. 438-481.
  • Cytermann Jean-Richard & Demeuse Marc (2005). La lecture des indicateurs internationaux en France : rapport et avis. Paris : La Documentation française.
  • Dauvisis Marie-Claire (2007). « L'évaluation des compétences au risque des barèmes et des notes scolaires ». In Hatano Maude & Lemaître Denis (dir.). Usages de la notion de compétence en éducation et formation. Paris : L'Harmattan, p. 75-92.
  • Deakin Crick Ruth (dir.) (2004). A systematic review of the impact of citizenship education on the provision of schooling. London : EPPI-Centre.
  • Deakin Crick Ruth (dir.) (2005). A systematic review of the impact of citizenship education on student learning and achievement. London : EPPI-Centre.
  • Deakin Crick Ruth (2007). « Learning how to learn: The dynamic assessment of learning power ». The Curriculum Journal, vol. 18, n° 2, p. 135-153.
  • Downes Stephen (2008). « My Digital Identity ». In ePortfolio & Identité Numérique 2008, Montréal, 5-7 mai 2008.
  • Emin Jean-Claude (2008). « Que fait-on des évaluations internationales dans le système éducatif français ? ». In Baillat Gilles, De Ketele Jean-Marie, Paquay Léopold & Thélot Claude (dir.). Évaluer pour former. Outils, dispositifs et acteurs. Bruxelles : De Boeck.
  • Flutter Julia & Rudduck Jean (2004). Consulting Pupils: What's in It for Schools? London : RoutledgeFalmer.
  • Gérard François-Marie (2008). « Les outils d'évaluation ouverts, ou la nécessité de clés de fermeture ». In Évaluer pour former. Outils, dispositifs et acteurs. Bruxelles : De Boeck, p. 99-110.
  • Gilliéron Giroud Patricia (2007). « Le dossier d'évaluation ou portfolio en Suisse romande : Visées institutionnelles, mises en ouvre et perspectives d'avenir ». In Actes du 19e colloque de l'ADMEE-Europe, Université du Luxembourg, 11-13 septembre 2006.
  • Gordon George (2005). Responding to Student Needs: Student Evaluation and Feedback Toolkit. Scotland : Quality Assurance Agency for Higher Education (QAA).
  • Goupil Georgette (1998). Portfolios et dossiers d'apprentissage. Montréal : Editions Chenelière éducation.
  • Hargreaves Eleanore (2005). « Assessment for learning? Thinking outside the (black) box ». Cambridge Journal of Education, vol. 35, n° 2, p. 213-224.
  • Hartnell-Young Elizabeth (2007). Impact Study of e-portfolios on learning. Coventry : Becta.
  • Harvey Lee (2001). Student Feedback: A report to the Higher Education Funding Council for England. Birmingham : University of Central England.
  • Hoban Garry & Hastings Geoff (2006). « Developing different forms of student feedback to promote teacher reflection: A 10-year collaboration ». Teaching and Teacher Education, vol. 22, n° 8, p. 1006-1019.
  • Hoskins Bryony & Fredriksson Ulf (2008). Learning to Learn: What is it and can it be measured?. Ispra : Centre for Research on Lifelong Learning (CRELL).
  • Hoskins Bryony & Deakin Crick Ruth (2008). Learning to Learn and Civic Competences: Different currencies or two sides of the same coin? Ispra : Centre for Research on Lifelong Learning (CRELL).
  • Houchot Alain & Robine Florence (2007). Les livrets de compétences : Nouveaux outils pour l'évaluation des acquis. Paris : Ministère de l'Éducation nationale, n° 2007- 048.
  • Klenowski Val (2002). Developing Portfolios for Learning and Assessment: Processes and Principles. London : Routledge.
  • Leitch Ruth & Odena Oscar (2007). « Consulting secondary school students on increasing participation in their own assessment in Northern Ireland ». In Annual European Conference on Educational Research, Ghent, septembre 2007.
  • Little David (2003). The European Language Portfolio in use: nine examples. Strasbourg : Council of Europe.
  • Mangez Éric (2008). Réformer les contenus d'enseignement. Paris : Presses universitaires de France.
  • McIntyre Donald, Pedder David & Rudduck Jean (2005). « Pupil voice: comfortable and uncomfortable learnings for teachers ». Research Papers in Education, vol. 20, n° 2, p. 149-168.
  • Merle Pierre (2007). Les notes. Secrets de fabrication. Paris : Presses universitaires de France.
  • Morlaix Sophie (2007). Identifier et évaluer les compétences dans le système éducatif : quels apports pour la recherche en education ? [Rapport d'Habilitation à diriger des recherches]. Dijon : Institut de Recherche sur l'Éducation.
  • Nicol David J. & Macfarlane-Dick Debra (2006). « Formative assessment and self-regulated learning: A model and seven principles of good feedback practice ». Studies in Higher Education, vol. 31, n° 2, p. 199-218.
  • Paquay Léopold, De Ketele Jean-Marie, Thélot Claude & Baillat Gilles (dir.) (2008). Évaluer pour former. Bruxelles : De Boeck.
  • Prebble Tom, Hargraves Helen & Leach Linda et al. (2005). Impact of Student Support Services and Academic Development Programmes on Student Outcomes in Undergraduate Tertiary Study: A Synthesis of the Research: Report to the Ministry of Education. New Zealand: Ministry of Education.
  • Pryor John & Crossouard Barbara (2008). « A socio-cultural theorisation of formative assessment ». Oxford Review of Education, vol. 34, n° 1, p. 1-20.
  • Rey Bernard, Carette Vincent, Defrance Anne & Kahn Sabine (2006). Les compétences à l'école. Apprentissage et évaluation. Bruxelles : De Boeck.
  • Rey Bernard (2008). « Quelques aspects éthiques de l'évaluation ». In Baillat Gilles, De Ketele Jean-Marie, Paquay Léopold & Thélot Claude (dir.). Évaluer pour former. Outils, dispositifs et acteurs. Bruxelles : De Boeck, p. 57-67.
  • Rey Olivier (2008). « From the transmission of knowledge to a competence-based approach ». Dossier d'actualité de la VST, n°34. En ligne (consulté le 13 novembre 2008) : <http://www.inrp.fr/vst/LettreVST/english/34-april-2008_en.php>.
  • Richardson John T. E. (2005). « Instruments for obtaining student feedback: A review of the literature ». Assessment & Evaluation in Higher Education, vol. 30, n° 4, p. 387-415.
  • Sadler D. Royce (1989). « Formative assessment and the design of instructional systems ». Instructional Science, vol. 18, n° 2, p. 119-144.
  • Scallon Gérard (2007). L'évaluation des apprentissages dans une approche par compétences. 2e éd. Bruxelles : De Boeck.
  • Scallon Gérard (2008). « Évaluation formative et psychologie cognitive : Mouvances et tendances ». In Grégoire Jacques (dir.). Évaluer les apprentissages. Les apports de la psychologie cognitive. Bruxelles : De Boeck.
  • Schärer Rolf (2007). European Language Portfolio: Interim Report 2006. Executive Summary. Strasbourg : Council of Europe.
  • Scheuermann & Guimarães Pereira (2008). Towards a Research Agenda on Computer-based Assessment: Challenges and needs for European Educational Measurement. Ispra: Centre for Research on Lifelong Learning (CRELL).
  • Sebba Judy, Deakin Crick Ruth & Yu Guoxing et al. (2008). Systematic review of research evidence of the impact on students in secondary schools of self and peer assessment. London : EPPI-Centre.
  • Siemens George (2004). « EPortfolios ». Elearnspace. En ligne (consulté le 19 novembre 2008) :
    <http://www.elearnspace.org/Articles/eportfolios.htm>.
  • Taras Maddalena (2005). « Assessment summative and formative: Some theoretical reflections ». British Journal of Educational Studies, vol. 55, n° 4, p. 466-478.
  • Tardif Maurice (2006). L'évaluation des compétences : Documenter le parcours de développement. Montréal : Chenelière Éducation.
  • Weeden Paul & Winter Jan (1999). Learners' Expectations of Assessment for Learning: Report for the Qualifications and Curriculum Authority. University of Bristol.
  • White Barbara Y. & Frederiksen John R. (1998). « Inquiry, Modeling, and Metacognition: Making Science Accessible to All Students ». Cognition and Instruction, vol. 16, n° 1, p. 3-118.
  • Wiliam Dylan (2007). « Changing Classroom Practice ». Educational Leadership, vol. 65, n° 4, p. 36-42.
  
Written by: Laure Endrizzi and Olivier Rey

© ENS Lyon
École normale supérieure de Lyon
Institut français de l'Éducation
Veille et Analyses

15, parvis René-Descartes - BP 7000 - 69342 Lyon cedex 07
Standard: +33 (0)4 72 76 61 00, Télécopie: +33 (0)4 72 76 61 06