OECD PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)

Reference Number: PR0889, Press Release Issue Date: May 03, 2013
 
The Programme for International Student Assessment (PISA) is an international comparative survey of educational achievement of 15-year-olds.  It aims to evaluate education systems worldwide every three years by assessing 15-year-olds' competencies in the key subjects: reading, mathematics and science.  The purpose of this programme is to assess the knowledge and skills in reading, mathematical and scientific literacy of students as they near the end of their compulsory schooling.  PISA seeks to assess students’ competences to address real-life challenges involving reading, science and mathematics and measure how well these young adults have mastered this knowledge and skills gained at school, at home and elsewhere to function as successful members of society.  This is in contrast to assessments that seek to measure the extent to which students have mastered a specific curriculum.
 
PISA was launched by the OECD in 1997 and takes place every three years.  The first was carried out in 2000 and PISA 2009 was the 4th survey.
·         In PISA 2009, there were 64 participating countries, including all 33 OECD countries and almost all EU countries.  26 of these countries were neither EU nor OECD members.
·         Malta, together with nine additional partner participants, was unable to participate within the PISA 2009 project timeframe; however, participated in the PISA 2009+ project.  These ten PISA 2009+ participants administered the same assessments as their PISA 2009 counterparts, but in 2010. These countries or economies were required to meet the same technical and quality standards as their PISA 2009 counterparts.
·         The PISA surveys assess students in reading, mathematics and science.  In each survey one of these is the main subject. In PISA 2009 the main subject was reading.
·         As well as tests for students, the PISA survey includes questionnaires for participating students.  In PISA 2009 these included some general questions, most related to socioeconomic background; students’ attitudes to reading and aspects of the teaching and learning of reading. The questionnaires also included aspects of school and classroom climate and effectiveness of teaching strategies.
 
3453 Maltese students participated in the PISA study included 1839 females and 1614 males. The sample comprised almost the whole population of 15-year olds and guaranteed a maximum margin of error of approximately 1% using a 95% degree of confidence.  As part of the PISA survey, students complete an assessment on reading literacy, mathematical and scientific literacies as well as an extensive background questionnaire.  School principals complete a survey describing the context of education at their school, including the level of resources in the school and staff qualifications. The purpose is to identify influences of all school, home and societal factors on educational outcomes.
 
PISA assesses outcomes primarily in the areas of mathematical literacy, scientific literacy and reading literacy. The emphasis on ‘literacy’ refers to ‘the capacity of students to apply knowledge and skills in key subject areas and to analyse, reason and communicate effectively as they pose, solve and interpret problems in a variety of situations’.
 
Mathematical literacy is concerned with the ability of students to analyse, reason, and communicate ideas as they pose, formulate, solve, and interpret solutions to mathematical problems in a variety of situations.
 
The PISA mathematics assessment has been designed in relation to:
 
  • Mathematical content: This is defined mainly in terms of four overarching ideas (quantity, space and shape, change and relationships, and uncertainty) and only secondarily in relation to curricular strands (such as numbers, algebra and geometry).
  • Mathematical processes: These are defined by individual mathematical competencies.  These include the use of mathematical language, modelling and problem-solving skills. Such skills, however, are not separated out in different test items, since it is assumed that a range of competencies will be needed to perform any given mathematical task. Rather, questions are organised in terms of competency clusters defining the type of thinking skill needed.
  • Situations: These are defined in terms of the ones in which mathematics is used, based on their distance from the students. The framework identifies five situations: personal, educational, occupational, public and scientific.
 
Science literacy
This is defined as the ability to use scientific knowledge and processes not only to understand the natural world but to participate in decisions that affect it.  Science literacy, in PISA aims to assess pupils’ scientific understanding which is needed in adult life and is defined as the individual’s scientific knowledge and use of that knowledge to identify questions, to acquire new knowledge, to explain scientific phenomena, and to draw evidence based conclusions about science-related issues, understanding of the characteristic features of science as a form of human knowledge and enquiry, awareness of how science and technology shape our material, cultural environments and intellectual, and willingness to engage in science-related issues, and with the ideas of science, as a reflective citizen. The PISA assessment is designed to measure scientific knowledge as well as scientific competency and understanding of scientific contexts.
 
  • Scientific knowledge constitutes the links that aid understanding of related phenomena.  In PISA scientific tasks, pupils are not simply expected to recall facts but to be familiar with scientific concepts related to physics, chemistry, biological sciences, and earth and space science and able to apply them to the content of the test items. In fact, the term “scientific literacy” rather than “science” underlines the importance that the PISA science assessment places on the application of scientific knowledge in the context of life situations, compared with simple reproduction of traditional school science knowledge.
  • Scientific Competencies are centred on the student’s ability to acquire, interpret and act upon evidence. Three processes are identified in PISA – firstly identify, describe, explain and predict scientific phenomena; secondly understand scientific investigation and thirdly interpret scientific evidence and conclusions. A student’s ability to carry out scientific competencies involves both knowledge of science and an understanding of the characteristics of science as a way of acquiring knowledge.  Moreover, the definition also recognises that the disposition to carry out these competencies depends upon an individual’s attitudes towards science and a willingness to engage in science related issues.
  • Scientific Contexts concern the application of scientific knowledge and the use of scientific processes. The framework identifies three main areas: science in life and health, science in Earth and environment, and science in technology.
 
 
Reading literacy
This is defined in terms of students’ ability to understand, use and reflect on written text to achieve their purposes.  However, PISA takes this aspect of literacy further by the introduction of an active element – the capacity not just to understand a text but to reflect on it, drawing on one’s own thoughts and experiences.  In PISA, reading literacy is assessed in relation to:
  • Text format: Often students’ reading assessments have focused on continuous texts or prose organised in sentences and paragraphs.  From its inception, PISA has used in addition non-continuous texts that present information in other ways, such as in lists, forms, graphs, or diagrams.  It has also distinguished between a range of prose forms, such as narration, exposition and argumentation.  In PISA 2009, the framework encompasses both print and electronic texts, and the distinctions outlined above are applied to both.  These distinctions are based on the principle that individuals will encounter a range of written material in their civic and work-related adult life and that it is not sufficient to be able to read a limited number of types of text.
  • Reading processes (aspects): Students are not assessed on the most basic reading skills, as it is assumed that most 15-year-old students will have acquired these.  Rather, they are expected to demonstrate their proficiency in accessing and retrieving information, forming a broad general understanding of the text, interpreting it, reflecting on its contents and reflecting on its form and features.
  • Situations: These are defined by the use for which the text was constructed. For example, a novel, personal letter or biography is written for people’s personal use; official documents or announcements for public use; a manual or report for occupational use; and a textbook or worksheet for educational use.  Since some groups may perform better in one reading situation than in another, it is desirable to include a range of types of reading in the assessment items.
 
 
Results
 
PISA gives the scores achieved in terms of Six Proficiency Levels, 1 to 6, in each of the reading, mathematics and science literacies. 
 
Reading Literacy
Malta placed in the 45th position among 74 countries. Maltese pupils achieved a mean score of 442 for reading, which was statistically lower than both the EU average (485) and OECD average (493).  In Malta, girls outperformed boys in reading by an average of 72 points.
 
Reading Literacy
 
 
 
 
 
Level
(score)
Below 1
(< 262)
1
(262-407)
2
(407-480)
3
(480-553)
4
(553-626)
5
(626-698)
6
(> 698)
Malta
8.3 %
28.0 %
23.0 %
22.3 %
13.9 %
4.0 %
0.5 %
OECD average
1.1 %
17.7 %
24%
28.9 %
20.7 %
6.8 %
0.8 %
 
 
Table below shows Malta’s average score for all three reading aspect subscales - Access and Retrieve, Integrate and Interpret; and Reflect and Evaluate and all two text format reading subscales - Continuous and Non-Continuous texts.
Reading Sub-scale
 
 
 
 
 
 
Reading
Scale
Access
& retrieve
Integrate
&
interpret
Reflect
and evaluate
Continuous
Non-continuous
Malta
442
435
442
448
437
454
OECD average
493
495
493
494
494
493
EU average
485
486
486
484
485
484
 
Mathematics Literacy
Maltese pupils achieved a mean score of 463 for mathematics, which was statistically lower than both the EU average (490) and OECD average (496).  Malta placed in the 40th position amongst 74 countries.  In Malta, girls performed better than boys by an average of 15 points.
 
Mathematics Literacy
 
 
 
 
 
Level
(score)
Below 1
(< 358)
1
(358-420)
2
(420-482)
3
(482-545)
4
(545-607)
5
(607-669)
6
(> 669)
Malta
16.3 %
17.4 %
21.9 %
21.3 %
15.4 %
6.4 %
1.4 %
OECD average
8.0 %
14.0 %
22.0 %
24.3 %
18.9 %
9.6 %
3.1 %
 
 
Science Literacy
Maltese pupils achieved a mean score of 461 in Science, which is significantly lower than both the EU average (497) and OECD average (501).  Malta placed in the 41st position among 74 countries.  In Malta, girls performed significantly better than boys in science by an average of 35 points.
 
Science Literacy
 
 
 
 
 
Level
(score)
Below 1
(< 335)
1
(335-410)
2
(410-484)
3
(484-559)
4
(559-633)
5
(633-708)
6
(> 708)
Malta
14.5 %
18.0 %
23.3 %
22.7 %
15.5 %
5.2 %
0.8 %
OECD average
5.0 %
13.0 %
24.4 %
28.6 %
20.6 %
7.4 %
1.2 %
 
 
 
Social, economic and cultural status in Malta
·         The student questionnaire provides information on pupils’ demographic and socio-economic backgrounds including parents’ level of education, their qualifications and their main occupations.  Other questions provide information about family structure, family members and family possessions.  The questionnaire also provides information about the students’ study habits, and attitudes to reading and reading activities at school.
·         The proportion of Maltese mothers living with their children (97.6%) is marginally higher than EU (96.4%) and OECD (95.8%) averages.
·         The proportion of Maltese fathers living with their children (92.7%) is significantly higher than EU (86.2%) and OECD (86.0%) averages.
·         The proportion of two-parent family structures in Malta (92.1%) exceeds EU and OECD averages by more than 10%.
·         The proportion of Maltese mothers doing home duties (49.7%) is more than double EU and OECD averages.
·         The proportion of Maltese mothers doing a full time job (27.5%) is approximately half EU and OECD averages; however, the proportion of mothers doing a part-time job (18.0%) is comparable to EU and OECD averages.
·         The proportion of Maltese fathers doing a full time job (88.7%) exceeds EU and OECD averages by more than 7%; however, there is significantly lower proportion of fathers doing home duties.  Compared to other countries, Maltese fathers are more likely to be breadwinners and mothers to be house carers.
·         The majority of Maltese parents (around 40%) have highest educational level a ISCED 2 (Secondary level education).
·         The proportion of Maltese parents with highest educational level at ISCED 3A or above (55.3%) is significantly lower than both the EU average (78.6%) and OECD average (77.6%).
·         In Malta, the proportion of mothers doing a white collar high skilled job (38.6%) is significantly lower to EU and OECD averages; whereas the proportion of mothers doing a blue collar low skilled job (18.6%) is significantly higher.
·         The proportion of fathers in Malta doing a white collar job (59.0%) is significantly higher than the EU and OECD averages; whereas, the proportion of fathers doing a blue collar job (41.0%) is significantly lower.
·         Compared to EU and OECD averages, Maltese students are more likely to have a computer to use for school work, an internet link, educational software, a DVD player, poetry books and books to help with school work but less likely to have a room of their own and a quiet place to study.
·         Maltese students tend to have more books at home.  Over 83% of Maltese students own more than 25 books which is significantly higher to both the OECD average (72.1%) and EU average (72.2%).
·         Maltese families own more cars, computers, televisions and rooms with baths/showers than EU and OECD averages.
·         Around 88% of Maltese 15-year old students speak their native language at home; whereas, most of the remaining 12% speak English.
 
 
 
Reading attitudes in Malta
·         Over 66% of pupils in Malta spend some time reading for enjoyment, while about 34% only read if they have to. Responses to statements measuring attitudes to reading were marginally more positive to EU and OECD averages.
·         18.6% of Maltese students believe that reading is a waste of time; however, this is significantly lower to EU (26.0%) and OECD (24.2%) averages
·         Attitude to reading is positively correlated with reading scores. Both internationally and in Malta, there was a large difference in reading attainment scores between those who never read for enjoyment and those who do, even for a short time daily.
·         For pupils in Malta the most popular reading materials were magazines (53.8%) and newspapers (45.3%). Pupils read fiction (39.3%) more often than non-fiction books (24.1%). Compared to EU and OECD averages, higher proportions of Maltese students read fiction and non-fiction books and lower proportions read magazines, newspapers and comic books.
·         Pupils in Malta spent significantly more time chatting online and reading emails than the OECD and EU averages but were similar to their EU and OECD counterparts in the frequency of other online activities.
 
 
 
Learning time in Malta
·         Learning time at school does not vary much between countries. Most students have on average 4 to 5 lessons weekly in mathematics, science and test language (English in the case of Malta) and the duration of each lesson ranges from 45 to 60 minutes.
·         The proportion of Maltese pupils attending additional lessons outside school hours is significantly higher than EU and OECD averages.
·         Malta has the highest proportion (58.2%) of students in EU and OECD countries who take additional lessons in Mathematics, followed by Greece (53.0%), Israel (41.1%), Korea (38.7%), Portugal (31.1%) and Japan (29.6%).
·         Malta also has the highest proportion (43.9%) of students who take extra lessons in Science, followed Greece (42.9%), Mexico (21.8%), Poland (18.4%) and Korea (17.3%).
·         Malta has the fourth largest proportion (21.2%) of pupils taking additional lessons outside school time in the test language (English) following by Greece (32.0%), Korea (27.0%) and Mexico (22.4%) and the third largest proportion (28.7%) of pupils who take extra lessons in other school subjects following Poland (40.3%) and Israel (33.1%).
·         The proportions (9.4%) of Maltese students taking extra lessons to improve study skills are below the EU average (10.6%) and OECD average (11.3%).  Japan (46.9%) leads the list followed by Mexico (24.1%).
·         Interestingly, those pupils taking additional lessons outside school time in Science and English scored significantly lower in the PISA science and reading tests than their counterparts who did not take additional lessons outside school time. In Mathematics, the mean attainment scores were comparable between the two groups taking/not taking additional lessons outside school time.
 
 
School and classroom climate and use of libraries
·         School effectiveness perceived by Maltese students appears to be more positive overall. Compared to EU and OECD averages there is a higher proportion of Maltese students who believe that the school has helped them gain confidence to make decisions and taught them facts that may be useful in future jobs; and a lower proportion of Maltese students who think that school is a waste of time. However, a higher proportion of Maltese pupils believe that their school could do much more to prepare students for adult life.
·         Students’ perception of their teachers in Malta is more positive compared to EU and OECD averages. The proportions of Maltese students who agree that most of their teachers are interested in their well-being, listen to what they have to say and assist them when they need extra help exceed OECD and EU averages by more than 10%.  Higher proportions of Maltese students also agree that most of their teachers treat them fairly and they get along well with them.
·         About a third of Maltese students stated that disruptions occur quite often during lessons. Around a quarter of the students stated that they start working well after the lesson starts and around 18% said that disruptions in class hinder them to work well.  However, these proportions are comparable to EU and OECD.
·         The proportion of Maltese students who visit libraries for reading activities is significantly higher than EU and OECD averages. Compared to EU and OECD averages there is higher proportion of Maltese students who borrow books for school work but significantly lower proportions who visit libraries to work on homework, to read magazines, to read for fun, to learn things and to use the internet.