By Chen, Chao-Hsiu
ABSTRACT. Research findings indicate that teachers’ beliefs play an important role in their deciding how they will integrate technology into the classroom. The author used qualitative research methods to explore the relations between teachers’ pedagogical beliefs and technology integration. Participants were 12 Taiwanese high school teachers, and findings indicated inconsistency between the teachers’ expressed beliefs and their practices. The author categorized the reasons for the inconsistency into 3 interrelated aspects: (a) the influence of external factors, (b) teachers’ limited or improper theoretical understanding, and (c) teachers’ other conflicting beliefs. The author presents suggestions for school settings, professional development, and future research. Keywords: constructivist instruction, teacher belief, technology integration
Researchers have suggested that a crucial factor for successful technology integration into the classroom is the teacher (e.g., Bitner & Bitner, 2002; Loveless, DeVoogd, & Bohlin, 2001; Romano, 2003; Zhao & Cziko, 2001), because what directly determines the instruction that takes place behind the classroom door is the teacher rather than external educational agenda or requirements (Hodas, 1993; Tyack & Cuban, 1995). Researchers have found that teacher beliefs play a critical role in transforming teachers’ technology integration into more constructivist practices (e.g., Bitner & Bitner; Dexter, Anderson, & Becker, 1999; Ertmer, 1999, 2005; Niederhauser & Stoddart, 2001; Ravitz, Becker, & Wong, 2000; Sandholtz, Ringstaff, & Dwyer, 1991, 1997; Windschitl & Sahl, 2002). Therefore, the relations between teachers’ beliefs and teachers’ practices should help to shed light on how teachers make technology- integration decisions.
Judson (2006) and Levin and Wadmany (2006) specifically investigated whether teachers’ technology use in classrooms would reflect the teachers’ pedagogical beliefs, and regarding this technology use the researchers’ findings confirmed the existence of an inconsistency between (a) teachers’ expressed beliefs about such use and (b) teachers’ practices. Judson stated that novice teachers’ perception of instruction situations may differ from expert teachers’ corresponding perception and that, consequently, novice teachers’ technology-use practices may be inconsistent with their positive attitudes toward constructivist concepts. However, such an explanation merits further investigation. In their 3-year study, Levin and Wadmany found that teachers’ technology-use practices and pedagogical beliefs had changed reciprocally, and that teachers seldom held pure educational beliefs. However, Levin and Wadmany did not provide other reasons for the inconsistency between teachers’ beliefs and practices in technology use. Therefore, the purposes of the present study were (a) to investigate whether teachers’ pedagogical beliefs align with their practices regarding technology integration and (b) to explore the reasons for any inconsistency between teacher beliefs and teacher practices.
Since 1999, the Ministry of Education of Taiwan (i.e., the education department of Taiwan’s central government) has continually reviewed and revised educational policies to establish technology- rich environments in schools and encourage teachers to incorporate technology into instruction. In several studies, researchers have investigated perceived factors or barriers affecting teachers’ technology use in Taiwan (e.g., Chen, 2004; Hsu, 2003; Wang, 2004), but none have focused on how teacher beliefs and other factors interact with one another to influence technology integration. To implement national plans for technology integration, policymakers must know how teachers’ beliefs influence teachers’ practices regarding technology integration.
In the present study, 12 Taiwanese high school teachers revealed an inconsistency between their expressed beliefs and their practices. All participants reported high agreement levels on constructivist concepts promoted by educational policies, but the participants’ technology use was mainly for supporting content coverage. Further analyses revealed that external factors, teachers’ limited or inappropriate understanding of the promoted concepts, and teachers’ other conflicting beliefs could cause the inconsistency. The findings suggest that although teachers’ beliefs may affect their interpretation of proposed policies and hence their practices, it is necessary for researchers to consider teachers’ beliefs and various contextual factors all together when undertaking an educational innovation. Furthermore, it is beneficial to provide teachers with feasible examples of how to implement promoted ideas and resolve conflicts among various beliefs, organizational supports and constraints, and related practices.
Theoretical Framework
Characteristics of Teacher Beliefs
In a classroom, the teacher perceives and defines a teaching situation, makes judgments and decisions, and then takes related actions. Therefore, to improve teacher preparation and teaching practice, educators should pay more attention to teacher beliefs because these beliefs profoundly influence decision-making processes and teaching practices (e.g., Borko, Cone, Russo, & Shavelson, 1979; Borko & Putnam, 1996; Kagan, 1992; Nespor, 1987; Pajares, 1992; Richardson, 1994; Woods, 1996). On the basis of their beliefs about teaching, educational goals, and student learning, teachers choose specific strategies and materials from their repertories to tackle particular situations. Their beliefs help them to determine what problems to focus on and how to solve the problems (Borko et al.; Borko & Putnam; Nespor).
Although the study of teacher beliefs has great potential for providing educational communities with unprecedented insights, it faces the difficulty of being short on clear and commonly accepted definitions and conceptualizations of beliefs and belief structures (Kane, Sandretto, & Heath, 2002; Pajares, 1992). Scholars in the field use different terms to identify a similar concept (Kagan, 1992; Pajares; Richardson, 1994), and the list of terms seems to grow endlessly: attitudes, values, judgments, axioms, opinions, ideology, perceptions, conceptions, conceptual systems, preconceptions, dispositions, implicit theories, explicit theories, personal theories, internal mental processes, action strategies, rules of practice, practical principles, perspectives, repertories of understanding, and social strategy (Pajares, p. 309).
The difficulty in defining teacher beliefs centers on the difficulty and ambiguity of differentiating beliefs from knowledge (Calderhead, 1996; Pajares, 1992; Richardson, 1994). For example, Rokeach (1968) argued that all beliefs include a cognitive component representing knowledge, an affective component arousing emotions, and a behavioral component guiding actions. Therefore, knowledge is a component of belief. However, Roehler, Duffy, Herrmann, Conley, and Johnson (1988) stated that knowledge structures are the main force driving teacher behavior in a classroom. The research group proposed that knowledge is both fluid among different contexts and open to new experiences but that belief is both surrounded by emotion and too static to change. Therefore, Roehler et al. prioritized knowledge over beliefs. Kagan (1990) decided to use belief and knowledge interchangeably because evidence showed that a teacher’s knowledge is expressed in highly subjective terms.
Although knowledge and beliefs are “inextricably intertwined” (Pajares, 1992, p. 325), Nespor (1987) suggested that beliefs are distinguished from knowledge because the propositions or the concepts of belief systems do not require consensus between the belief holder and the outsider and because beliefs are usually disputable. Also, researchers can view belief systems as loosely bounded, as having no clear logical rules that connect these beliefs to events or situations in real life. Individuals can form the connections from personal, episodic, and emotional experiences. Green (1971) described the structure of belief systems, including the relation among beliefs, the strengths of beliefs, and the ways in which relevant beliefs cluster.
Belief systems are organized by individuals according to primary beliefs and to beliefs derived from other beliefs. A primary belief is too self-evident to explain, whereas a derivative belief is related to other beliefs, so it can be explained in reference to another belief that acts as evidence. In line with Rokeach’s (1968) description, Green (1971) argued that some beliefs are more central to belief systems (i.e., core beliefs) and more resistant to change than are other beliefs because the former beliefs are held with “passionate conviction” (p. 53). In contrast, peripheral beliefs are held with less psychological strength and further from the belief systems’ core. With examination and discussion, peripheral beliefs are easier to change. Consistent beliefs are held in the same cluster, but people may hold conflicting beliefs without noticing the conflict as long as they not only hold the conflicting beliefs in separate clusters but also refrain from comparing the beliefs with one another. Pajares (1992) stated that people have beliefs about everything and that to conceptualize a belief system is to recognize that the belief system contains various beliefs connecting to one another. Clusters of beliefs focusing on a construct form an attitude (Rokeach, 1968). Teachers’ attitudes about education- including attitudes about schooling, teaching, learning, and students-are usually represented as teacher beliefs (Pajares). However, the construct of educational beliefs may still be too general for research purposes, so Ertmer (2005) specified her investigation of teachers’ educational beliefs about teaching and learning by following the recommendation of Pajares. Researchers refer to these beliefs as pedagogical beliefs. In the present article, the specified teachers’ pedagogical beliefs focus on teachers’ educational beliefs about teaching and learning, but teacher beliefs collectively signify teachers’ various beliefs, including pedagogical beliefs and beliefs about how technology can facilitate the process of putting pedagogical beliefs into practices. On the basis of the aforementioned research on teachers’ beliefs, I speculated that teachers’ pedagogical beliefs can strongly influence their decisions on technology integration. However, these beliefs may compete with their other beliefs or external factors and consequently be transformed.
Teacher Beliefs and Technology Integration
Ertmer (2005) stated that most teachers-regardless of whether they are veterans or novices-have limited understanding and experience about how technology should integrate into various educational aspects to facilitate teaching and learning. When trying to integrate technology into their instruction, teachers refer to their existing beliefs and prior experiences. Teachers’ existing beliefs can influence the development of beliefs about both technology integration and related practices. Hence, administrators should take teacher beliefs into account at different stages of technology integration.
Zhao and Cziko (2001) highlighted the important role that teacher beliefs play in technology integration. For teachers to use technology, they must believe that (a) technology can help them to achieve higher level goals more effectively, (b) technology use will not disturb higher level goals, and (c) the teachers will have adequate ability and sufficient resources to use technology. Zhao and Cziko further explained that teachers may be unwilling to adopt technology if the promoted use is inconsistent with their existing beliefs or practices. Teachers’ beliefs serve as a filter through which they determine the priorities of different factors. Certain factors can be regarded as closer to the core beliefs. Therefore, what types of applications and to what degree technology will be integrated into a classroom depend on each teacher’s beliefs (Zhao & Frank, 2003).
Although many researchers have treated teacher beliefs as a crucial factor in technology integration, research findings (e.g., Judson, 2006; Levin & Wadmany, 2006) have shown that teachers’ technology use in classrooms is perhaps not necessarily aligned with their reported beliefs and that teachers could hold conflicting educational beliefs about how to integrate technology into instruction. Ertmer (2005) suggested that contextual factors might cause inconsistency between expressed technology-related pedagogical beliefs and implemented technology-related practices. Contextual factors that may influence teachers’ technology integration include policy, school culture, and availability of appropriate equipment, training, and integration examples (e.g., Bitner & Bitner, 2002; Bullock, 2004; Cuban, Kirkpatrick, & Peck, 2001; Norris, Sullivan, Poirot, & Soloway, 2003).
Contextual factors in schools and classrooms can significantly affect the process by which teachers’ beliefs and knowledge change (Richardson, 1996). As described by Doyle (1977), complex classroom life involves various activities and processes with different purposes. Many events occur simultaneously or even haphazardly, and these events usually demand a teacher’s immediate attention. To manage this complexity, teachers may develop different coping strategies, which may be inconsistent with the teachers’ own beliefs (Davis, Konopak, & Readence, 1993). Tabachnick and Zeichner (2003) suggested that consistency of teacher beliefs and of teacher practices is a consequence of an ongoing negotiation process by which a teacher resolves conflict between organizational supports and organizational constraints. Hence, teachers do not base each decision solely on their pedagogical beliefs. And hence, the relationships among various beliefs and contextual factors should be the focus of research on teacher beliefs and practices.
In sum, teacher beliefs are often related to teacher practices, and some beliefs are more resistant to change. Teachers may hold conflicting beliefs as long as the incompatible beliefs are not explicitly compared. To incorporate technology into their instruction in innovative ways, teachers may have to reconsider their pedagogical beliefs. However, requiring teachers to change their pedagogical beliefs can be a daunting task because it may involve challenging fundamental beliefs. Moreover, different contextual factors may combine to affect teachers’ beliefs and technology use, and researchers need to consider the relations among those factors.
Proposed Technology Integration in Taiwan
Understanding that education quality may improve through the promotion of technology integration, the Ministry of Education of Taiwan proposed a plan to make the Information Education Infrastructure a component of the ministry’s Educational Reform Action Program implemented from 1999 on. Later, in 2001, the Ministry of Education announced the “Blueprint of Information Education in Elementary and Junior High Schools” (Ministry of Education, 2001a). In this report, the Ministry of Education stated that teachers could be the driving force for guiding all citizens to becoming proficient in technology literacy and learning skills. Eventually, all citizens would become computer literate and lifelong learners. The goals of the blueprint explicitly emphasized information literacy, critical and creative thinking, effective learning strategies, active learning, collaborative learning, and lifelong learning (Ministry of Education, 2001b). In addition, a central reform policy entitled “Nine-Year Consistent Curriculum for Primary and Secondary Schools” emphasized students’ abilities of active inquiry, problem solving, communication, and cooperation (Ministry of Education, 2000).
At present, proposed instructional theories, teaching strategies, and technology integration reflect the concepts of constructivism (Jonassen & Land, 2000; Reigeluth, 1999). According to constructivism, learners actively construct their understanding on the basis of their prior experiences and existing knowledge structures (Ginsburg & Opper, 1988; Greeno, Collins, & Resnick, 1996). By interacting with environments, tools, and other people, learners gradually apprehend a shared knowledge, language, and culture (Greeno et al.; Lave & Wenger, 1991; Wells, 2000). In constructivist education, the teacher designs learning activities to engage students in active problem solving and genuine inquiry. Educators design the learning tasks to be authentic and challenging so that they motivate students. Teachers encourage multiple viewpoints, and students can discuss and debate their opinions.
Research has revealed that learning with technology can foster student understanding by engaging students in higher-order thinking, self-regulated learning, and collaborative or cooperative learning (Jonassen, Howland, Moore, & Marra, 2003; Lowyck & Elen, 2004). Hence, reform policies encourage teachers in Taiwan to align technology integration with constructivist concepts, and the policies require teachers to adopt so-called constructivist mathematics teaching. Under the context of educational reform, the present study investigated the relationships between teachers’ pedagogical beliefs and practices regarding technology integration.
Method
In the present study, I investigated how Taiwanese high school teachers’ pedagogical beliefs affect the teachers’ technology integration. Researchers indicate that teachers seldom examine or articulate their beliefs. In addition, teachers may hold some beliefs unconsciously. Sometimes they either do not know how to describe the beliefs or feel reluctant to express the beliefs if they are unpopular (Fullan, 2001; Kagan, 1990; Richardson, 1994). Therefore, researchers (Kagan, 1990; Kane et al., 2002; Pajares, 1992) have stated that it is inadequate to investigate teacher beliefs on the basis solely of teachers’ talk and that the same inadequacy afflicts studies that, by focusing exclusively on teachers’ actions, neglect teachers’ talk. Following this suggestion, I used qualitative methods to collect data from multiple data sources. In particular, I relied on interviews, documents, and over 2 months of classroom observation. The collected documents included syllabi, lesson plans, handouts, PowerPoint slides, and products created by students.
Participants
Purposive sampling can increase the range of data and maximize the possibilities of uncovering multiple realities (Lincoln & Guba, 1985). Therefore, to obtain abundant information related to the focus of the study, I conducted it in a public high school in Taipei, where-I anticipated-the teachers probably undertook projects of technology integration more often than did teachers in other parts of Taiwan. The studied school had a reputation for technology integration, and the academic achievement of the high school students had been above average in comparison with the academic achievement attributable to students at other Taipei-area high schools. The technology coordinator of the school was familiar with the technology use of his colleagues, and he helped me to recruit 14 teachers of different grades and subjects so that I could acquire a more comprehensive perspective from them. After talking with all the teachers and observing several classes, I chose 12 teachers who were most comfortable with being observed and interviewed as my participants. Table 1 lists the demographic information of these 12 teachers. The participants’ teaching experience ranged from 2 to 28 years, and most teachers had integrated technology into instruction for 2-4 years. The studied school has been a so-called whole school, consisting of students from 7th grade to 12th grade. However, there were 3 times as many classes and students in the senior high section as in the junior high section, and 11 of the 12 participants were teaching in the senior high section. Most Taiwanese high school teachers do not have their own classrooms but share offices with colleagues. At the school that I studied, most teachers were assigned a seat in one of their department’s offices. The students usually stayed in home classrooms but went to special classrooms or labs for classes taught by the teachers who were in charge of those classrooms or labs. Most teachers went to their home classroom to teach according to schedule and stayed in their office during free periods. The teachers who needed to use a special classroom or a lab reserved it in advance. Students sat in rows in homerooms, computer labs, and most special classrooms. This kind of seating arrangement was not convenient for collaborative work.
Procedure
Because of the participants’ schedules, I observed 9 of the 12 participants for more than 2 months. During the period of classroom observation, I kept field notes and reflection journals, collected documents, and talked with participants and their students informally. After finishing classroom observation, I formally interviewed all participants once or twice. All interview sessions were semistructured, and the prepared questions focused on the following aspects: (a) teachers’ beliefs about teaching and learning, (b) teachers’ beliefs about the potential of technology, (c) teachers’ ideal instruction and technology integration, (d) pros and cons of technology integration, and (e) factors preventing teachers from implementing their ideal instruction and technology integration. Each formal interview session was about 1 hr, and all interview sessions were audiotaped for further transcription and analyses. Also, to better understand the participants’ pedagogical beliefs, I asked them to fill out a form inquiring into their agreement levels on 11 constructivist statements before interviews. I had developed the 11 statements on the basis of literature describing constructivist instruction (e.g., Greeno et al., 1996; Jonassen & Land, 2000; Jonassen et al., 2003; Lowyck & Elen, 2004; Reigeluth, 1999; Wells, 2000). On a Likert scale ranging from 0 (totally disagree) to 9 (totally agree), each participant reported his or her agreement on each constructivist statement. During the data collection, I refined my data-collection processes by constantly making comparisons and by generating concept-related questions (Strauss & Corbin, 1998).
I based the data-coding processes on the strategy specified by Merriam (2001). First, I printed out all the raw data. Then, I chose a part of my field notes and read it. While reading the notes line by line, I wrote down-both in the notes and on a separate memo-my reflections and the names of the categories that I created. I added, modified, or deleted the names of categories on the list during this process. I repeated this coding process several times until the temporary coding of these notes was satisfactory. Then, I moved to another part of the field notes, transcripts, or documents. I repeated the process but also compared the previous data set with the current list, adding new category names to the latter accordingly. I repeated the process over and over until I finished coding all my data. Then, I opened all the related documents on my laptop and typed category names onto the documents. As I typed in the names, I continued to add, modify, and delete categories. I repeated these processes and noticed certain patterns and regularities emerging. Having identified sufficiently relevant categories, I integrated and refined them to highlight the data related to the research questions. The relationships of main categories represent concepts emerging from the data. Last, I identified the relations of all categories and conceptualized the findings.
Results
During the processes of data coding and analyses, I easily identified inconsistencies between participants’ expressed beliefs in survey data and practices manifested in other sources of data. The participants’ responses to the survey questions (listed in Table 2) provided me with general information regarding the participants’ pedagogical beliefs. Except for two statements whose mean scores were about 7, the mean scores of the statements were greater than 8. This result showed that the participants identified most strongly with constructivist instruction. However, the classroom observation, collected documents, and interviews indicated that most participants did not integrate technology into instruction in ways to facilitate students’ problem solving, collaborative or cooperative learning, and self-regulated learning.
All participants confirmed that they used various technologies for personal purposes, for instruction planning, and for administrative work, but very few of them viewed technology as a means to achieve instructional goals other than covering curricular content, preparing students for examinations, and highlighting important concepts. By analyzing and interpreting the collected data, I identified three categories of factors, and these factors explained the inconsistency between the participants’ reported pedagogical beliefs and the participants’ technology-integration practices. The three categories were the influence of external factors, teachers’ limited or improper understanding of constructivist instruction, and teachers’ other beliefs conflicting with the expressed pedagogical beliefs. Most important was my finding that these factors did not independently influence teachers’ instruction and technology use. It was the interplay among different factors that caused the inconsistency.
Influence of External Factors
When the form asked the participants to explain why their instruction or their technology use did not always reflect their agreement with constructivist concepts, all participants identified various external factors such as lack of access to computers and software, insufficient time to plan instruction, and inadequate technical and administrative support-factors that other researchers have documented (e.g., Bitner & Bitner, 2002; Bullock, 2004; Cuban et al., 2001; Dexter et al., 1999). Participants reported that the average class size was about 40 students, that the students differed from one another regarding ability level, and that the large class size prevented the participants, as teachers, from paying sufficient attention to individual students’ variable learning.
Every semester, the participants conducted themselves -as do most high school teachers in Taiwan-by adhering to the following routines: choosing a textbook from a variety of textbooks fulfilling the curriculum standards announced by the Ministry of Education, deciding a schedule for covering the textbook content, and taking turns to compose papers for three major tests. The three major tests usually accounted for 70% of the final grade, so the teachers had little flexibility to use either supplementary materials or different assessment methods.
Most participants had no intention of reserving a computer lab that would enable students to conduct activities with technology, because there were only five computer labs for the school’s approximately 170 teachers and because the participants were uncomfortable with teaching in an unfamiliar environment. Thus, one participant noted, “a teacher in a computer lab is like an octopus handling many things all the time.” She added, “I also wish computers wouldn’t freeze because I can’t present the content fast enough. . . . Whenever I was facing a problem with technology, I needed prompt technical support. Without support, I could only idle there, wasting class time.”
Actually, all of the reported barriers to implementing teachers’ ideal instruction and technology use seemed to relate to the high- stakes examination that the Taiwanese educational system has used for more than 50 years. Schools use a highly selective examination called the College Joint Examination to filter students, and students’ scores in this examination will determine the colleges to which the students can obtain admission (Pan & Yu, 1999). The stakes of the examination are so high that most high school teachers, especially those who teach academic subjects, consider the data on students’ exam performance of vital importance. Taiwanese society treats the data as if they represent the competency of students, teachers themselves, and schools. Although the government of Taiwan proposed the concept of integrating technology into education to foster student understanding, a primary goal of the school where I conducted this research was to prepare students for the examinations. The inflexible school organization and assessment system discouraged the teachers from using technology to conduct creative but time-consuming activities.
For instance, during the observation period, most observed participants and students were busy transmitting and receiving textbook content, and only one participant used technology to support students’ project work and collaborative learning. This participant divided students into groups to compose gatepost couplets, which are a kind of poetry ascribed to ancient Chinese intellectuals, and students were amazed at their ability to create such works. This participant used educational technology to present learning tasks, to support the creation of student work, and to share the final products. Another participant required students to discuss controversial topics by using an online discussion board, and a purpose of designing such online projects was to “connect to students’ daily life” because “they had experience and they had their own perspectives on it.” She viewed herself as a guide who should encourage students to express their perspectives, debate with each other, and finally reach mature consensuses, and the network provided a convenient means for open discussion.
Yet, three participants whose willingness to incorporate technology into extended learning colored their talks, syllabi, and student work unanimously declared that there was no easy way to allocate class time for those timeconsuming activities. For those participants, content coverage was the main classroom theme. They conducted such activities only once or twice per year, and students sometimes had to undertake those activities during summer vacation or winter break.
Some teachers and students might experience inconvenience in accessing necessary equipment because of the joint exam. For example, the 12th-grade classrooms were not equipped with computers and projectors because the 12th-grade students, facing the approaching joint exam, had no time for such distractions. Participants confirmed that, in a school meeting, the 12th-grade classroom teachers had decided not to install the equipment in their classrooms for fear that some students could not resist the temptation to play games. Three participants who taught the 12th grade considered this decision a great inconvenience to their technology use in instruction. Furthermore, when parents expressed their concerns about some teachers’ technology use and instructional strategies, the administrators did not always support the teachers. A participant described his unpleasant experience of fixing a computer during the previous semester: “I had to fix the computer and the projector twice during three or four periods of the same class, and, instantly, parents of that class called the principal.” During interview sessions, 9 out of the 12 participants mentioned that pressure from parents was a substantial obstacle to ideal instruction and technology integration. Various types of pressure might compel or even force teachers to resume lecture-based instruction and repetitive practice.
Limited or Improper Theoretical Understanding
Although external barriers could prevent teachers from implementing the promoted constructivist instruction and technology integration, in some cases teachers’ lack of theoretical understanding could explain the inconsistency between teachers’ expressed beliefs and the teachers’ practices. As Fullan (2001) suggested, teachers may value and precisely state the concepts of a promoted change but fail to understand how to put these concepts into practice. For example, almost all participants reported high agreement with constructivist concepts, which the Minister of Education in Taiwan has been promoting. However, without specific guidelines and guidance, teachers implemented the policies on the basis of their own interpretation and understanding.
Most participants considered constructivist concepts ideal rather than practicable. A participant viewed her technology integration as “a compromised improvement given the situations of real life” because the students were used to traditional teaching and declined to change their learning habits. Also, she was unsure whether her technology use could promote students’ self-evaluation, although she recognized the importance of this ability.
During the interview sessions, no participant could confidently explain how to apply those concepts to instruction. Even 2 participants whose strategies of technology integration were more aligned with constructivist concepts were uncertain about how to design technology-based learning activities that would facilitate students’ active knowledge construction. The 2 participants were uncomfortable in experimenting with technology integration without knowing how to connect the constructivist concepts to the teaching practices. One of the 2 participants said, “Everybody has talked about technology integration for many years, but there has been limited influence” and “Most applications we saw were traditional approaches with new tools.”
The 2 participants emphasized that teachers who are learning about effective technology integration should have examples of it. According to interview records and training materials obtained from a participant who was a regular instructor of many professional- development programs, the school provided teachers with intensive professional-development opportunities that nevertheless focused on computer-operating skills. The program content was irrelevant to ideas about the creative integration of educational technology into instruction and to ideas about fundamental changes in instruction.
Meanwhile, a participant persisted in talking about the importance of using technology both to pay attention to individual students and to encourage self-regulated learning, but she could not identify strategies to achieve these goals. She pleasantly declared that her students were spending most of the class time engaging in a student-centered activity, which was to practice test-taking repetitively. In this type of student-centered activity, she would list multiple-choice questions and answers on slides to speed up the process of reviewing questions and correcting answers. This participant prioritized the advantage of technology use in content coverage, and she believed that, with PowerPoint slides, she could keep students focused and meanwhile cover much more content.
Another participant stressed that her instructional goal was to promote students’ application of multiple viewpoints and of critical thinking, but my observation of this participant’s classroom did not reflect such emphases. Actually, she promoted students’ application of multiple viewpoints and of critical thinking by telling the students about these valuable qualities. The teacher was the primary performer in this class, trying to persuade the students to accept the ideas. Hence, although the educational policies encouraged teachers to integrate technology into instructional strategies and to align this integration with constructivist concepts, teachers did not necessarily have appropriate understanding about the theoretical concepts or about how to incorporate the promoted concepts into their daily teaching.
Other Conflicting Beliefs
As I mentioned in the theoretical-framework section, individuals may hold conflicting beliefs without being aware of the conflicts, and some beliefs are more central to belief systems, so that individuals may resist the belief change. Sometimes, participants’ other conflicting beliefs had a greater effect on instruction and technology use than did the participants’ expressed pedagogical beliefs. Moreover, external barriers in teachers’ daily teaching might reinforce those conflicting beliefs.
For example, all participants reported that they were under pressure to cover all content, and most participants were unwilling or hesitant to allow students to spend valuable class time in exploring content on their own with technology. Although the pressure of content coverage might come from external factors such as school organization, a commonly accepted belief is that teachers need to cover more content both to guide student learning and to fulfill teacher obligations.
This belief influenced the participants’ decisions in allocating class time. A participant who was a physics teacher stated that “If I skip some content, the students will think I’ve neglected my duty.” He was not comfortable with skipping some content and spending more time in conducting activities with technology because he felt, “If the students fail to answer the relevant questions while taking exams, their parents will call administrators to complain.” A participant who was an English-language teacher claimed that her instructional goal was to help her students score high in the high-stakes tests, and she thought that the students needed to “learn a lot of stuff in a short period of time” because “learning a language requires memorization of a lot of stuff.” According to her and another participant who also taught English, most English- language teachers agreed on adopting many materials because “the questions on the exam could be related to all kinds of areas, and different areas have different keywords.”
Also, teachers might believe that students would not actively learn the subject matter if the teachers do not test students frequently. A participant said, “Students need to prepare for many subjects, and teachers of other subjects may push the students to study their subjects by giving more tests.” Therefore, the participant concluded, “It seems teachers giving fewer tests can’t improve student performance. I have no choice but to have the students take tests all the time.” Although recognizing the importance of students’ independent problem solving and self- regulated learning, most teachers confirmed, “the students are used to being forced to study by teachers.”
Consequently, student learning became passive, and teachers tended not to believe that the students could be selfregulated learners. A math teacher claimed, “For students to achieve higher standards for math scores, old methods work the best.” The old methods were the lectured-based, teacher-centered instruction, which required students to engage in much test-taking practice.
While recognizing the advantages of using technology to motivate students and to present instructional content effectively, eight participants were concerned that students might undertake off-task activities or make no learning progress when unexpected incidents such as computer breakdowns preoccupy teachers and students alike. To most participants, avoiding uncertainty and maintaining order and control in the classroom were important, and PowerPoint slides could organize teaching materials to help reduce teacher uncertainty and anxiety. The slides could alleviate information overload of the teachers and enabled them to attend to student behavior, so all participants’ technology integration seldom went beyond PowerPoint presentations. Discussion
This study identified inconsistency between teachers’ expressed pedagogical beliefs and their practices regarding technology use. All participants reported high levels of agreement on constructivist concepts, but the participants’ instruction remained teacher centered and lecture based, and their technology use was to support such instruction. Although teaching is usually regarded as an intentional activity, not all teaching activities are based on teachers’ intentions or beliefs because the environment surrounding the teachers has a strong influence on teachers’ decision making (Lowyck, 2003). Teachers’ decisions about instructional strategies are based on different information and concerns, including (a) information about students, (b) teachers’ beliefs or instructional purposes, (c) the characteristics of learning tasks or curriculum, and (d) the constraints and support of the instructional situations (Ball & Cohen, 1996; Borko et al., 1979; Calderhead, 1987).
In Taiwan, the assessment systems implemented in most schools emphasize competition over collaboration among students and even teachers. Accordingly, most teachers are reluctant to base their technology integration on constructivist strategies. Even when willing to explore new instructional approaches and the potential of technology to improve teaching and learning, teachers can squeeze only limited time from their busy schedules to undertake experiments and exploration. Under the pressure of preparing students for high- stakes tests, teachers may be torn between their ideal instruction and covering the content that may be related to the items on the tests (Pellegrino, 2004). Teachers may compromise their ideal instruction to meet the needs and the expectations of students, parents, and administrators.
Another issue causing the inconsistency between teachers’ pedagogical beliefs and practices is their limited or inappropriate understanding of the expressed beliefs. This finding highlights the inadequacy of investigating teacher belief without considering the relationship between a teacher’s knowledge and a teacher’s undertaking an innovation. The decision of Kagan (1990) to use teacher belief and teacher knowledge interchangeably is of debatable soundness because teachers may lack a satisfactory understanding of their own strongly held concepts. Some terms may lack clear definitions and clear relations to other terms so that teachers who express fidelity to the corresponding concepts can rigorously connect them neither to one another, nor to theoretical foundations, nor to concrete instructional strategies.
Teachers may mistake a learner-directed practice for constructivist instruction, and teachers may view nonscaffolding learning as student-centered learning (Land & Hannafin, 2000). Also, teachers’ existing beliefs may function as a filter and influence them to consciously or unconsciously modify the proposed instructional and assessment methods to fit their existing beliefs (Borko, Davinroy, Bliem, & Cumbo, 2000; Borko, Mayfield, Marion, Flexer, & Cumbo, 1997).
The present study’s findings suggest that, immersed in traditional educational systems, some teachers might be more likely to ignore or reject the proposed ideas and practices about how to implement technology integration. To handle complex classroom dynamics, teachers may develop various routines or coping strategies that are contradictory to their beliefs, thereby creating inconsistency between teachers’ expressed beliefs and observed practices (Davis et al., 1993; Fang, 1996). In this regard, some participants of the present study recognized the negative effect of heavy lecturing and repetitive test-taking practices but considered traditional instructional approaches as best for achieving the primary goal of the school, and the participants’ instruction and technology integration supported this goal.
Asking teachers to change their routines is not easy because without a familiar routine to support their practices, teachers may have less time and energy to monitor students’ behavior and learning progress (Borko & Shavelson, 1990; Shavelson & Stern, 1981). Hence, some participants worried that unanticipated technical problems or the complexity of handling both teaching and computer operations would interrupt instruction so that they-the participants-would end up wasting time and exacerbating classroom-management problems.
The participants attached great importance to being able to control classroom processes. To most participants, more student autonomy and fewer tests would lead to lower student performance because the students would devote less time and energy to learning the subject. Believing that surrendering authority could hinder student learning, most participants assumed substantial control of their classrooms and attended to details of student learning. Using presentation software to present instructional content was therefore the most common application of technology because it enabled the participants to maintain control and improve classroom management.
Last but not least, teachers’ beliefs and contextual factors may affect each other. For example, a school structure that grades pencil-and-paper tests to motivate students will reinforce some teachers’ traditional beliefs that value knowledge transmission (Blumenfeld, Fishman, Krajcik, & Marx, 2000). Most participants had developed the belief that scoring high on the joint examination was an important goal of high school students in Taiwan. On the basis of this belief, the participants used appropriate resources, instructional strategies, and technology applications to achieve this goal. Also, the school settings, evaluation methods, and attitudes of other stakeholders such as students, parents, and administrators all confirmed and strengthened the belief.
Conclusion
Although educational reform in Taiwan encourages teachers to incorporate technology and constructivist concepts into instruction, in the present study I found that teachers who reported high levels of agreement with constructivist concepts did not integrate technology in ways that aligned with the participants’ reports. Regarding technology use, three categories of factors contributed to the inconsistency between teachers’ pedagogical beliefs and teachers’ pedagogical practices: (a) the influence of external factors, (b) teachers’ limited or incorrect understanding of constructivist instruction, and (c) teachers’ other beliefs conflicting with the teachers’ expressed pedagogical beliefs.
The findings suggest that the beliefs that teachers hold may not resonate in their practices and that to achieve successful innovation, educators should consider multiple aspects simultaneously. It is inadequate to focus solely on one aspect of a reform (Blumenfeld et al., 2000). Simply imposing reform-based ideas on schools and teachers will not result in substantial change in instruction. Educational reform may encourage teachers to integrate technology to engage students in activities of problem solving, critical thinking, and collaborative learning, but a culture emphasizing competition and a high-stakes assessment system can strongly discourage teachers from undertaking such innovative initiatives. Teachers may feel the need to demonstrate their accountability to students, parents, and administrators and may often feel disoriented when they encounter conflicts between their own beliefs and other stakeholders’ expectations. Teachers need more flexibility in using assessment methods other than pencil-and-paper tests. Support from other stakeholders-such as parents, colleagues, and administrators-is also critical for transforming teacher practice.
Teachers may have incomplete or incorrect understanding of proposed ideas and may hold conflicting beliefs without noticing the inconsistency. Professional development programs should go beyond teacher training that emphasizes basic skills, such as how to operate a computer, over skills of greater refinement. Moreover, the design of professional development programs should identify teachers’ beliefs about effective teaching, strategies for improved teaching and learning, and curriculum design appropriate for pedagogical purposes (King, 2002; Schwab & Foa, 2001; Windschitl & Sahl, 2002). Feasible examples of effective technology integration should be readily available to teachers, who could thereby solidify their technology use to improve teaching and learning in the classroom (Bitner & Bitner, 2002).
Regarding future research, despite the difficulty of measuring teacher beliefs, more studies should focus on how to develop methods or instruments that can help in the rigorous identification and evaluation of teacher beliefs. When asking participants to report their beliefs, researchers need to ensure that the participants do not misunderstand the queried statement. Also, as Davis et al. (1993) suggested, rather than provide teachers with more educational theories, researchers should help teachers to cope with the difficulties and the complexities of classroom life. More studies should document examples of how teachers accomplish meaningful and effective technology integration without trendy and powerful equipment because most teachers may benefit more from such studies (Ertmer, Gopalakrishnan, & Ross, 2001). Such studies can result in concrete and attainable suggestions that meet the needs of most teachers. REFERENCES
Ball, D. L., & Cohen, D. K. (1996). Reform by the book: What is- or might be-the role of curriculum materials in teacher learning and instructional reform? Educational Researcher, 25(9), 6-8, 14.
Bitner, N., & Bitner, J. (2002). Integrating technology into the classroom: Eight keys to success. Journal of Technology and Teacher Education, 10, 95-100.
Blumenfeld, P., Fishman, B. J., Krajcik, J., & Marx, R. W. (2000). Creating usable innovations in systemic reform: Scaling up technology-embedded project-based science in urban schools. Educational Psychologist, 35, 149-164.
Borko, H., Cone, R., Russo, N. A., & Shavelson, R. J. (1979). Teachers’ decision making. In P. L. Peterson & H. J. Walberg (Eds.), Research on teaching: Concepts, findings, and implications (pp. 136- 160). Berkeley, CA: McCutchan.
Borko, H., Davinroy, K. H., Bliem, C. L., & Cumbo, K. B. (2000). Exploring and supporting teacher change: Two third-grade teachers’ experiences in a mathematics and literacy staff development project. Elementary School Journal, 100, 273-306.
Borko, H., Mayfield, V., Marion, S., Flexer, R., & Cumbo, K. (1997). Teachers’ developing ideas and practices about mathematics performance performance assessment: Success, stumbling blocks, and implications for professional development. Teaching & Teacher Education, 13, 259-278.
Borko, H., & Putnam R. T. (1996). Leaning to teach. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 673-708). New York: Macmillan.
Borko, H., & Shavelson, R. J. (1990). Teacher decision making. In B. F. Jones & L. Idol (Eds.), Dimensions of thinking and cognitive instruction (pp. 311-346). Hillsdale, NJ: Erlbaum.
Bullock, D. (2004). Moving from theory to practice: An examination of the factors that preservice teachers encounter as they attempt to gain experience teaching with technology during field placement experiences. Journal of Technology and Teacher Education, 12, 211-237.
Calderhead, J. (1987). Introduction. In J. Calderhead (Ed.), Exploring teachers’ thinking (pp. 1-19). London: Cassell Educational.
Calderhead, J. (1996). Teachers: Beliefs and knowledge. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 709-725). New York: Macmillan.
Chen, X. (2004). A study of stages of concern, layers of adoption, encouraging and obstructive factors about integrating information technology into instruction in junior high school language domain teachers in Kaohsiung City. Unpublished master’s thesis, National Sun Yat-sen University, Kaohsiung, Taiwan.
Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technology in high school classrooms: Explaining an apparent paradox. American Educational Research Journal, 38, 813- 834.
Davis, M. M., Konopak, B. C., & Readence, J. E. (1993). An investigation of two chapter 1 teachers’ beliefs about reading and instructional practices. Reading Research and Instruction, 33, 105- 133.
Dexter, S. L., Anderson, R. E., & Becker, H. J. (1999). Teachers’ views of computers as catalysts for changes in their teaching practice. Journal of Research on Computing in Education, 31, 221- 239.
Doyle, W. (1977). Learning the classroom environment: An ecological analysis. Journal of Teacher Education, 28(6), 51-55.
Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: Strategies for technology integration. Educational Technology Research and Development, 47(4), 47-61.
Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25-39.
Ertmer, P. A., Gopalakrishnan, S., & Ross, E. M., (2001). Technologyusing teachers: Comparing perceptions of exemplary technology use to best practice. Journal of Research on Technology in Education, 33(5). Retrieved April 18, 2005, from http:// www.iste.org/jrte/33/5/ertmer.htm
Fang, Z. (1996). A review of research on teacher beliefs and practices. Educational Research, 38(1), 47-65.
Fullan, M. (2001). The new meaning of educational change (3rd ed.). New York: Teachers College Press.
Ginsburg, H. P., & Opper, S. (1988). Piaget’s theory of intellectual development (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Green, T. F. (1971). The activities of teaching. New York: McGraw- Hill.
Greeno, J. G., Collins, A. M., & Resnick, L. B. (1996). Cognition and learning. In D. Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 15-46). New York: Macmillan.
Hodas, S. (1993). Technology refusal and the organizational culture of schools. Educational Policy Analysis Archives, 1(10). Retrieved April 18, 2005, from http://epaa.asu.edu/epaa/v1n10.html
Hsu, W. (2003). Integrating information technology into elementary science class: A case study of teacher knowledge transformation into action. Unpublished master’s thesis, National Sun Yat-sen University, Kaohsiung, Taiwan.
Jonassen, D. H., Howland, J., Moore, J., & Marra, R. M. (2003). Learning to solve problems with technology: A constructivist perspective (2nd ed.). Upper Saddle River, NJ: Pearson.
Jonassen, D. H., & Land, S. M. (2000). Preface. In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. iii-ix). Mahwah, NJ: Erlbaum.
Judson, E. (2006). How teachers integrate technology and their beliefs about learning: Is there a connection? Journal of Technology and Teacher Education, 14, 581-597.
Kagan, D. M. (1990). Ways of evaluating teacher cognition: Inferences concerning the Goldilocks Principle. Review of Educational Research, 60, 419-469.
Kagan, D. M. (1992). Implication of research on teacher belief. Educational Psychologist, 27, 65-90.
Kane, R., Sandretto, S., & Heath, C. (2002). Telling half the story: A critical review of research on the teaching beliefs and practices of university academics. Review of Educational Research, 72, 177-228.
King, K. P. (2002). Keeping pace with technology: Educational technology that transforms: Vol. 1. The challenge and promise for K- 12 educators. Cresskill, NJ: Hampton Press.
Land, S. M., & Hannifin, M. J. (2000). Student centered learning environments. In D. H. Jonassen & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. 1-23). Mahwah, NJ: Erlbaum.
Lave, J., & Wenger, E. (1991). Legitimate peripheral participation. Cambridge, MA: Cambridge University Press.
Levin, T., & Wadmany, R. (2006). Teacher’s beliefs and practices in technology-based classrooms: A developmental view. Journal of Research on Technology in Education, 39, 157-181.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.
Loveless, A., DeVoogd, G. L., & Bohlin, R. M. (2001). Something old, something new . . . : Is pedagogy affected by ICT? In A. Loveless & V. Ellis (Eds.), ICT, pedagogy and the curriculum (pp. 63- 83). London: Routledge Falmer.
Lowyck, J. (2003). Teacher thinking and teacher routines: A bifurcation? In M. Kompf & P. M. Denicolo (Eds.), Teacher thinking twenty years on: Revisiting persisting problems and advances in education (pp. 101-110). Lisse, Netherlands: Swets & Zeitlinger.
Lowyck, J., & Elen, J. (2004). Linking ICT, knowledge domains, and learning support for the design of learning environments. In N. M. Seel & S. Dijkstra (Eds.), Curriculum, plans, and processes in instructional design (pp. 239-256). Mahwah, NJ: Erlbaum.
Merriam, S. B. (2001). Qualitative research and case study applications in education. San Francisco: Jossey-Bass.
Ministry of Education. (2000). The report of the expanding needs for the information education infrastructure plan. Retrieved July 26, 2005, from http://www.edu.tw/EDU_WEB/EDU_MGT/MOECC/EDU7892001/ information/itstatus/report/expanding.htm
Ministry of Education. (2001a). Blueprint of Information Education in Elementary and Junior High Schools. Retrieved July 26, 2005, from http://masterplan.educities.edu.tw/conference/ index1.shtml
Ministry of Education. (2001b). The meeting of reviewing and revising the educational reform. Retrieved July 26, 2005, from http:/ /www.edu.tw/EDU_WEB/EDU_MGT/E0001/EDUION001/menu03/sub02/ content_020101/03020101_04_09.htm
Nespor, J. (1987). The role of beliefs in the practice of teaching. Journal of Curriculum Studies, 19, 317-328.
Niederhauser, D. S. & Stoddart, T. (2001). Teachers’ instructional perspectives and use of educational software. Teaching and Teacher Education, 17, 15-31.
Norris, C., Sullivan, T., Poirot, J., & Soloway, E. (2003). No access, no use, no impact: Snapshot surveys of educational technology in K-12. Journal of Research on Technology in Education, 36, 15-27.
Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62, 307-332.
Pan, H., & Yu, C. (1999). Educational reforms with their impacts on school effectiveness and school improvement in Taiwan, R.O.C. School Effectiveness and School Improvement, 10(1), 72-85.
Pellegrino, J. W. (2004). Complex learning environments: Connecting learning theory, instructional design, and technology. In N. M. Seel & S. Dijkstra (Eds.), Curriculum, plans, and processes in instructional design (pp. 25-48). Mahwah, NJ: Erlbaum.
Ravitz, J. L., Becker, H. J., & Wong, Y. (2000). Teaching, Learning, and Computing: Constructivist-compatible beliefs and practices among U.S. teachers (Report No. 4). Retrieved April 2006, from http://www.crito.uci.edu/tlc/findings/computeruse/
Reigeluth, C. M. (1999). What is instructional-design theory and how is it changing? In C. M. Reigeluth (Ed.), Instructional-design theories and models: Vol. II. A new paradigm of instructional theory (Vol. 2, pp. 5-29). Mahwah, NJ: Erlbaum.
Richardson, V. (1994). The consideration of teachers’ beliefs. In V. Richardson (Ed.), Teacher change and the staff development process: A case in reading instruction (pp. 90-108). New York: Teachers College Press. Richardson, V. (1996). The role of attitudes and beliefs in learning to teach. In J. Sikula (Ed.), Handbook of research on teacher education (2nd ed., pp. 102-119). New York: Macmillan.
Roehler, L., Duffy, G., Herrmann, B. A., Conley, M., & Johnson, J. (1988). Knowledge structures as evidence on the “personal”: Bridging the gap from thought to practice. Journal of Curriculum Studies, 20, 159-165.
Rokeach, M. (1968). Beliefs, attitudes, and values. San Francisco: Jossey-Bass.
Romano, M. T. (2003). Empowering teachers with technology. Lanham, MD: Scarecrow.
Sandholtz, J. H., Ringstaff, C., & Dwyer, D. C. (1991). Changes in teachers’ beliefs and practices in technology-rich classrooms. Educational Leadership, 48(8), 45-52.
Sandholtz, J. H., Ringstaff, C., & Dwyer, D. C. (1997). Teaching with technology: Creating student-centered classrooms. New York: Teachers College Press.
Schwab, R. L., & Foa, L. J. (2001). Integrating technologies throughout our schools. Phi Delta Kappan, 82, 620-624.
Shavelson, R., & Stern, P. (1981). Research on teachers’ pedagogical thoughts, judgments, decisions, and behavior. Review of Educational Research, 51, 455-498.
Strauss, A. L., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage.
Tabachnick, B. R., & Zeichner, K. M. (2003). Teachers’ beliefs and classroom behaviours: Some teacher responses to inconsistency. In M. Kompf & P. M. Denicolo (Eds.), Teacher thinking twenty years on: Revisiting persisting problems and advances in education (pp. 165-175). Lisse, Netherlands: Swets & Zeitlinger.
Tyack, D., & Cuban, L. (1995). Tinkering toward Utopia. Cambridge, MA: Harvard University Press.
Wang, C. (2004). A study on information technology infused instructional innovation behavior and the influencing factors for Taipei municipal vocational high school teachers. Unpublished master’s thesis, National Taipei University of Technology, Taipei, Taiwan.
Wells, G. (2000). Dialogic inquiry in education: Building on the legacy of Vygotsky. In C. D. Lee & P. Smagorinsky (Eds.), Vygotskian perspectives on literacy research: Constructing meaning through collaborative inquiry (pp. 19-30). Cambridge, England: Cambridge University Press.
Windschitl, M., & Sahl, K. (2002). Tracing teachers’ use of technology in a laptop computer school: The interplay of teacher beliefs, social dynamics, and institutional culture. American Educational Research Journal, 39, 165-205.
Woods, D. (1996). Teacher cognition in language teaching. Cambridge, England: Cambridge University Press.
Zhao, Y., & Cziko, G. A. (2001). Teacher adoption of technology: A perceptual control theory perspective. Journal of Technology and Teacher Education, 9, 5-30.
Zhao, Y., & Frank, K. A. (2003). Factors affecting technology uses in schools: An ecological perspective. American Educational Research Journal, 40, 807-840.
CHAO-HSIU CHEN
National Chiao Tung University, Taiwan
Address correspondence to Chao-Hsiu Chen, Institute of Education, National Chiao Tung University, 1001 Ta Hsueh Road, Hsinchu 30010, Taiwan, Republic of China. (E-mail: [email protected])
Copyright (c) 2008 Heldref Publications
Copyright Heldref Publications Sep/Oct 2008
(c) 2008 Journal of Educational Research, The. Provided by ProQuest LLC. All rights Reserved.
Comments