|
|
Remarks about a Book Edited by Thomas Jahnke and Wolfram Meyerhöfer: Pisa & Co –
Kritik eines Programms edited by Thomas Jahnke und Wolfram Meyerhöfer, Verlag
Franzbecker/Hildesheim 2006, 349 Pages, Subskriptionspreis 9,90 Euro, ISBN
978-3-88120-428-6)
There is little recognition for books published in languages other than English
from the international community of researchers and practitioners, so a book
written in German focussing on the special situation in Germany of ‘Tests like
PISA’ is often overlooked. However, this book includes some useful general
theses about testing, testing industry, education and policy that are important
for all countries. In this short reflection note about the book I will
concentrate on these aspects.
The main thesis of the book is not new to mathematics education but it is
disturbing the political business around testing and standards very much. Tests
like PISA or TIMSS do more or less correctly measure the ability of students to
solve the tasks in these tests – but nothing else. All the other hypotheses
based on test results about abilities of students, quality of schools,
efficiency of national education systems etc. are neither valid nor correct from
a scientific point of view. I am sure that there are some empirical results and
some good arguments that can be used to debate these hypotheses but there is no
proof of any exact or constant relationship (in a statistical or other sense)
between test results and other abilities or qualities. If we look at the public
debate about test results and the clamour for more tests (ranking as a result is
very easy to understand and motivates to do something) we see that there is a
sharp contrast. On one hand we have the unproved ranking based only on a very
limited aspect of understanding mathematics and on the other hand we have much
political activity in the educational sector that are based on test results.
What do we as mathematic educators know about the relationship between test
results and other abilities or qualifications? According to another thesis
written in the book we have to accept that we do not have enough information
about the PISA test, the selections of tasks and the tasks themselves to decide
completely about their relevance to other qualities and qualifications. One
precondition to this is to know all about the basic facts, decisions and ideas
behind the test development. But the companies doing the test (they are called
“test industry” in the book) say that this information is part of their company
secrets in the same way that the technical details of a new machine for a car
producing company.
But even without having all information we can start thinking about the
relationship using what we know from our practice and the collected results of
research in mathematics education over a long period of time. Doing this it is
very easy to find many general characteristics that are clearly not tested by
PISA or similar tests. An important component of curricula in German speaking
countries is the inclusion of aims related to being a critical citizen, social
competence (including teamwork) or ethics/responsibility. Teamwork is a simple
example for something that is not tested by PISA because participants in the
test are not allowed to solve questions in a group. I think in terms of this
type of general educational aim, PISA and Co fall short as the test contain no
items that relevant to these. Doing research about these aims is much more
difficult. We need much more complex and expensive research settings to find out
correct and valid results in this area.
What about the relationship between PISA results and special mathematical
knowledge or abilities? Is it correct to say that a student who solved all the
tasks in a test about fractions or geometry has understood fractions or geometry?
Mathematics education has answered this question very often and very clear with:
NO. I do not want to add a long list of references here but key concepts from
the literature to think about include ‘taxonomy of understanding from imitating
to self organized application and evolution of theory’, ‘fundamental ideas’,
‘real world problems and modelling’, historical and systematic views of
mathematics.
So finally there is no relationship? Never say no! At least it seems clear that
if someone did not solve any task then he or she would have little understanding
of this part of mathematics. But what is the relationship? Is there any
percentage we can estimate? “Someone who solved 80% of PISA tasks has understood
at least 10% of this part of mathematics?” I think this is the wrong type of
question or research direction. If we want a way to compare different
educational systems (or single schools) then we would have to think very
carefully about good research designs to ensure that these comparisons would be
valid and meaningful.
When international testing started and Germany and Austria were hit for the
first time with bad results, my colleagues and I had to decide how to react.
Many were very happy that after 30 years of stagnation in schools and
universities, where cutbacks in government funding had been ongoing, there
suddenly was concern and movement. Mathematic educators become important as
advisors for politicians and a lot of official attention was focussed on the
teaching of mathematics. But in Germany politicians concentrated on more testing
and more ranking. Only a little additional money was given to improve the
quality of teaching. And now after a period of intensive testing, mathematics
teaching seems to be under increasing pressure to lead to better test results.
Is this a way to better teaching and learning? Some mathematics educators say
that PISA test tasks are better than typical schoolbook tasks. So from their
point of view it is an improvement to teach to this test. From my point of view
teaching to the test is a deformation of teaching mathematics. I argue that
mathematics education has developed better ideas for improved teaching that have
been effectively evaluated. So the main question should be: “How is it possible
to change the average teaching in this direction?” This would be a very huge
task!
According to results of research done by ALM members on adults and mathematics I
like to add one point. If those policy makers we have to talk with about better
mathematics teaching would have had personal experience with a better type of
mathematics education it would be much easier to convince them. But now we have
a different situation: The commercial interests of the test industry meets the
needs of politicians to get simple and clear measurement and results. A ranking
is very simple and gives the motivation to do something if other nations have a
better ranking. The motivation of the politicians is economically based:
Industry says they need high qualified workers and listen to the test industry,
whose results seem to be valid and objective. It is high time to destroy this
myth.
Juergen Maasz, Linz
P.S. I hope that this short
reflection note will provoke you to react. I think it would be very nice if we
have a discussion about it in this mailing list. At least I like to add a “Thank
you, Janet!” for the help with language problems.
|
