The Relationship Between Iraqi EFL Teachers’ Creativity and Identifying Flaws in Multiple-Choice Questions: A Case Study of Najaf

Introduction

Assessing students’ performance in their academic affairs seems very important in all fields. Testing students and teaching are so inter-related that it is virtually impossible to work in either field without constantly concerned with the other (Heaton & Harmer, 1975).. Designing a test should not be considered as an easy act because a test first is a method, it has specific instruments, techniques and procedures (Brown & Abeywickrama, 2010). Tests are designed in different methods but specially during last decades procedure of designing tests changed (Madsen, 1983) “ testing during last century and the early decades of this one was basically intuitive, or subjective and depend on the personal impression of teachers. … Subjective tests began to be replaced by objective tests because the latter could be scored consistently even by untrained people” (pp. 5-6).

The most popular type of objective tests is Multiple Choice Questions (MCQ) (Farhady, Ja'farpur, & Birjandi, 2007). MCQs are today very popular and use in different aspects of language learning and this popularity has some reasons; high reliability, convenience in scoring, efficiency, and economy (Ghafournia, 2013). As each MCQ seems consisted of a few lines, many teachers believe they are easy to construct but they are actually very difficult to design correctly (Brown & Abeywickrama, 2010). In recent decades, its advantages are regarded so highly and it seems that it is the only method and way of test. The frequent use of it is in testing of large number of candidates (Hughes, 2003). As mention, MCQs are one of the most difficult items to write, so it is reasonable that they become time consuming too.

Inappropriateness of MCQs seems to come from two main sources, first one is content, and the other is technique (Hughes, 2003). MCQs are invested in many fields, consequently the current paper intends to investigate differences between Iraqi EFL male and female teachers’ creativity. Besides that the relationship between teacher's creativity and their ability to identify flawls MC items and what types of flaws are more popular in Iraqi EFL teachers’ MCQs.

2. Purpose of the Study

Due to the importance of Multiple-Choice Question (MCQ) in testing students and teaching, this study aims to shed some light on the relationship between Iraqi EFL teacher's creativity and their ability to identify flaws MCQs.

To be more exact, the present research sets out to answer the following questions:

Is there any significant difference between Iraqi EFL male and female teachers’ creativity?

Is there no significant difference between teacher's creativity and their ability to identify flawless MC items?

What types of flaws are more popular in Iraqi EFL teachers’ MCQs?

2.1.Research Hypotheses

H01: There is no significant difference between creativity of male and female Iraqi EFL teachers.

H02: There is no significant difference between teacher's creativity and their ability to identify flawless MC items.

Theoretical Background

3.1. Creativity

3.1.1. Importance of Creativity

Some researchers (Kandi & Kandi, 2013) emphasized importance of creativity by stating, “The most complex and outstanding perspectives of human thought is creative thinking, about which different views exist. The human’s creativity is the most important equipment by which he can eradicate spiritual pressures of daily life and extraordinary events.” (P . 1)

According to the (Vasudevan, 2013) “creativity is vital, especially at the present when current jobs are transforming rapidly with the advance of information technology that complements globalization. Creative thinking can increase teachers’ capability to learn and teach in a way that ignites interest in the classroom” (P . 1). Researchers like (Reber, 1995)defined creativity as “an ability to generate solutions, ideas, conceptualizations, artistic forms, theories or products that are unique and novel” (P. 1) (cited in (Kilianska-Przybylo, 2011)). Another concept of creativity also mentioned by other researchers (Almeida et. al. 2008) “creativity as the skills required for generating ideas and products that are (a) rather novel and unconventional; (b) high in quality; and (c) suitable to the task at hand. ” (P. 3) (cited in (Pishghadam, et al., 2012)). Csiksentmihalyi (1996) emphasized this point that “creativity does not take place inside people’s head but from the interaction of a person’s thought and their socio-cultural context” (P. 3) (cited in (Pishghadam, et al., 2012)). Other investigation (Cumming, 2011) indicated that “creativity, learning and teaching are fundamental, interdisciplinary issues. Within education, creativity is usually associated with innovation, synthesis and making connections in the process of learning and teaching.” (p. 2) (cited in (Kilianska-Przybylo, 2011)). In another study (Carter, 2004) indicated this point about creativity that “this phenomenon cannot be de-contextualized or studied in a disciplinary vacuum or seen as an exclusively mental process and that creativity is a social, cultural and environmental phenomenon as well as a psychological process.” (P. 3) (cited in (Ghonsooly & Raeesi, 2012)). In studying creativity, two important points should be considered: (Kandi & Kandi, 2013)) :

“The first one is that creativity can be the creation of new forms from old products or ideas. In this case, the previous ideas and thoughts are mainly the basis of new creations. Regarding the second definition, creativity is monopoly and the result of personal efforts and not essentially a general situation.” ( P . 2)

Another researcher (Torrance, 1972) mentioned that creativity composite of four main factors:

“1- Fluidity: Ability to produce much idea in image frame with assumption of response and giving opinion about it.

2- Innovation: Ability to produce ideas that are different from usual and current ideas.

3- Flexibility: Ability to produce different and diversified ideas in image framework and presenting new solutions.

4- Extension: Ability to adding details or completion imagery ideas.” (P . 2) (cited in (Emamverdi & Gahramani, 2013))

Investigating factors that affect on creativity is not an easy job. It is believed that creativity is a multi-dimensional phenomenon that is under the effect of social and individual factors. Some researchers also believed that creativity is not a biological phenomenon and something that affect on it, is gender characteristics (Ghafourian, 2012).

3.2. Multiple Choice Questions

Textbooks include many MCQs and they are used on high-profile English language proficiency (Coombe, 2010). Any standard MCQs has two basic parts: a problem (the stem) and a list of suggested solutions (alternatives) (Hotiu, 2006). A stem is developed to provide the test takers with “a complete idea of the problem to be solved in selecting the right answer” (Haladyna, 1994, p. 35). Since stems supply the problem under question they are alternatively referred to as “problems” or “leads” as well (Marshel and Hales ,1971; Payne, 1992). While a list of suggested solutions alternatives, alternates, choices or options. The correct, best or most appropriate solution is referred to as the answer, the keyed answer, or keyed response, whereas the remaining options are called distracters or foils (Khodadady, 1997).

Unfortunately many teachers believe that MCQs are superficial exercises and do not need much understanding and thought (Clegg & Cashin, 1986) . Although they seem to be very simple, they are extremely difficult to design correctly (Brown & Abeywickrama, 2010). There are many reasons that show why these kinds of tests are widely used among teachers (Simkin & Kuechler, 2005). They result in more content valid interpretation for test score, reliability of these tests seems very high and they are easy to use and store (Haladyna & Downing, 1989). Other researchers (Clegg & Cashin, 1986) and (Hotiu, 2006) claimed that MCQs could be used for evaluating higher level of learning, e.g. contrasting and comparing. (Clegg & Cashin, 1986) also mentioned that MCQs are used in university classes widely and emphasized that MCQs can be used as diagnostic purpose for understanding strengths and weaknesses. Teachers can use them effectively for measuring students’ understanding of class materials (Simkin & Kuechler, 2005).

Grammatical inconsistency between the stem and the response, providing cues that help students to choose correct response, and specific words that show gender bias are among common types of violations that may occur in a test (Coombe et. al., 2010).

3.2.1. Good Multiple-Choice Questions Features

(Zimmaro, 2004) provided some guidelines to write good MCQ item by stating that teacher should make sure that each item emphasize on significant aspect of the content not trivial, questions should begin with easy items to more difficult one and each item should has one correct answer on which all participants would agree. Other researchers (Burton, Sudweeks, Merrill, & Wood, 1991) listed tips for constructing MCQs. Some of these tips are as below: (Burton, et al., 1991)

“1- Base each item on a specific problem stated clearly in the stem.

2- State the stem in positive form (in general).

3- Word the alternatives clearly and concisely.

4- Keep the alternatives mutually exclusive.” (p . 17-21)

Teachers should try not to use items as “none of the above”, they should keep homogeneity in their test and avoid using grammatical clue and negative phrases in their items (Haladyna & Downing, 1989). More distracters make better results and items that allow students to challenge are more favored (Sobolewski, 1996). In another study (Clegg & Cashin, 1986) specified other features that make a MCQ more better, e.g. writing the stem first, omit unnecessary repetition of items, making all options at similar length and avoid using specific determiners. (Brown & Abeywickrama, 2010) stated in their book three important features for designing good MCQs:

1. Design each item to measure a specific objective.

2. State both stem and options as simply and directly as possible.

3- Make certain that the intended answer is clearly the only correct one” ( P . 68-70)

Other investigators (Coombe, Folse, & Hubley, 2007) claimed that teachers should consider background knowledge of students, try not to use items that include implausible distracters and consider that all questions should be independent from other questions. In addition, teachers should try to start stem with a word not a blank and pay attention to this point that correct responses should randomly and equally distributed among the alternatives (Farhady, et al., 2007).

4. Methodology

4.1. Participants

For this study, among many EFL teachers in Najaf 100 EFL teachers who are teaching English in high school of Najaf are selected. Selection of percipients is based on Krejcie & Morgan (1970) Sample Size Table with 95% confidence and 0.05 degree of accuracy. EFL teachers are chosen but only 100 of them participate in this study. These samples are teaching English at public school of Najaf in 2016-2017 academic years in different areas. The participants include 50 male English teachers and 50 female English teachers. All teachers are at B.A level in one field of English Language and most of them are familiar with MCQs flaws that may happen in a test.

4.2. Instrument

As this study investigates the relationship between teachers' creativity and their skill in designing flawless multiple-choice questions, two instruments are used:

Abedi (1985) creativity questionnaire is given to the EFL teachers. The reliability and validity coefficient of test of creativity has been estimated to be %90 to %72 . Each person should read its items carefully and answer to items based on provided instruction. It includes 60 items and the ranges of scores are between zero to 120. It takes about 15 minutes to answer to the included items.

One designed MCQ tests that include 15 items is given to the teachers. Among these items, 12 of them contain at least one flaw. Researcher distributes this questionnaire to the EFL teachers and asks them to check the incorrect items and mention type of flaw within them.

Al Muhaidib (2010) listed some of popular flaws that may happen in teachers' designed tests. Researcher in this reserch chooses the most popular of them that is introduced by him. Here is the list based on (Muhaidib, 2010) investigation:

Negative stem

All of the above

Logical clue

Word repeat

Coombe, et al., (2007) also listed some common flaws in teachers' MCQ. Researcher in this investigation will select these flaws and invest thier popularization in teachers' items.

Unparallel options

Grammatical inconsistency

Impure items

Double answer key

Gender bias in language

In another investigation ( Farhady, et al., 2007) also mentioned some considerations that should be taken by teachers when they start writing MCQs. Two of those considerations are selected by researcher in order to investigate on them.

All distracters should be grammatically correct.

The stem should not start with a blank.

There are some reasons and flaws that make MCQs poor (Dowing, 2010):

a) The use of absolute terms (e.g. always, never)

b) The use of ‘all of the above’

c) Making the correct option the longest or most detailed

d) Using word repeats or logical clues in the stem as to the correct answer

e) And grammatical clues”

4.3. Procedure

For collecting data, researcher distributes the Creativity Questionnaire to EFL teachers. All participants are EFL teachers and researcher assures them that their gathered information keep confidential. Teachers' selection is done randomly among Najaf teachers who have B.A. degree. For this reason, researcher sends require questionnaire and MCQs test to EFL teachers' email and gives to some of them printed paper format. After gathering data, researcher will calculate the creativity of each person and records them.

Afterward, researcher analyzes teachers' answers to MCQs test that invests teachers' identifying flaws in MCQs . Researcher illustrates to the teachers how to answer to the questionnaires. For making the situation more real, he asks EFL teachers to answer the items individually and assigned a limited time for them to answer. It should be mentioned again that some data gathered through internet by sending email to other EFL teachers in other places, so researcher has less control on their answering situation, but he assigns and sends mentioned situation for them.

After gathering all required information, researcher records them in SPSS software (Version 21) for gaining statistical information. Then researcher correlates teachers' recognizing of MCQs flaws and their creativity index to see whether his hypothesis are confirmed or rejected.

5. Results

The first question of the study is whether there is any significant difference between Iraqi EFL male and female teachers’ creativity.

To test the first hypothesis, independent- samples t-test is employed. An independent- samples t-test is used statistically for the comparison of two mean scores of two different groups. In the present study, an independent sample T-test is applied to examine the difference between Iraqi male and female EFL teachers with regard to creativity.

At first, in this hypothesis, Equal variances assumed are investigated through Levene‘s test. This tests whether the variance (variation) of scores for two groups (males and females) is the same.

Table 1 :

Group Statistics and t-test for Equality of Means

The significance level of Levene‘s test i.e. the P-value is 0.712 which is greater than 0.05 and this means that the variances for two groups (males and females) are same.

The result shows that the means of creativity for males and females are 2.30 and 2.23, respectively. The significance level is 0.070 and is more than 0.05. Accordingly, the first null hypothesis with the 95% confidence is accepted.

The second question of the study is whether there is no significant difference between teacher's creativity and their ability to identify flawless MC items.

To test the second hypothesis, independent- samples t-test is employed.

H_0: There is no significant difference between teacher's creativity and their ability to identify flawless MC items.

H_1: There is significant difference between teacher's creativity and their ability to identify flawless MC items.

As table two shows, descriptive statistics are reported to summarize the characteristics of the data including minimum, maximum, mean, standard deviation values. As can be seen, the lowest and highest ability to identify flaws are respectively 0.07 and 0.8, and the mean and standard deviation are respectively 0.23 and 0.13.

Table 2 : Ability to Identify Flaws

To implement statistical methods and calculate the appropriate test statistic and logical deduction about research hypotheses, the most important step before any action is to select the appropriate statistical method for the research. For this purpose, knowledge of the distribution of data is a priority. In this study, Kolmogorov-Smirnov test is used to test the assumption of the normal variables of the research. The statistical constraints of Kolmogorov-Smirnov's normalization test are as follows.

H0 :data are normally distributed.

H1 :data are not normally distributed.

Null hypothesis of Kolmogorov-Smirnov test is the normality of data. If the obtained P-Value is less than 0.05 then the null hypothesis is rejected. The results of the Kolmogorov Smirnov test are shown in the table below. Considering that the significance level of the test (Sig.) in all variables is more than 0.05, the claim of the normality of the data is accepted and all variables have a normal distribution. Therefore, the parametric methods can be used to study the hypotheses.

Table 3:

One-Sample Kolmogorov-Smirnov Test

Table 4: Group Statistics and t-test for Equality of Means

As mentioning in previous investigations that teachers’ gender has no significant role in their creativity (Ozkal, 2014), no significant role of teachers’ creativity found between genders, In this study t-test comparison is conducted and researcher reaches to this conclusion that there is not any significant relationship between males’ and females’ creativity.

The present findings also refer that there is no significant difference between teacher's creativity and their ability to identify flawless MC items.

Amabile (1983) asserted that creativity should not be looked at as a personality trait or general ability but an attitude that results from the interaction of personal characteristics, cognitive ability, and environmental factors. Thus, it could be taught through the application of some simple techniques and strategies (Pishghadam, Ghorbani Nejad, & Shayesteh, 2012, p 207). The findings of this study also demonstrated that the role of teacher's creativity has no association with their ability to identify flawls MC items. This findings conclude that a teacher with low creativity can make high performance in designing a MCQ.

We can use test-strategies in many test formats but the primary use of it is in multiple-choice questions (Ghafournia, 2013) and (Haladyna, 2004). So, knowing this points leads teachers attention to the importance of strategies and flaws that might happen in their tests. Furthermore, knowing types of flaws and guiding teachers to avoid using them in their tests can enhance classroom tests validity and reliability, leading to better students’ performance and facilitating teaching environment.

As previous investigation and the result of this study revealed that teacher’ skills and training have significant role in their success, so test administrator can consider this point that for test administration suitable teacher with related major study should be selected. This investigation showed that teachers with BA degree in Teaching English as Foreign Language are more suitable for this aim.

Some types of flaws are more popular between teachers. Among these flaws Unparalleled options which is the most common one in the designing of multi-choice questions among Iraqi teachers which has 18.3%. Then the two flaws; Grammatically incorrect items and Unreasonable options 13.3% each. These valuable data can help Teacher Training College administrator to know teachers week points and ask their teachers to emphasis this points to their students.

References