Technology has brought important changes with its use and the transformation of not only academic environments but also human relationships and working experiences.
Furthermore, it has strengthened the process of globalization with today’s need for immediate responses and the correct selection of information from a discouraging vast amount. Knowledge of appropriate data use has been acquired in the current learning process.
Access to Information and Communication Technologies, (ICT) has also had an impact on different segments of the population (Katzmann, 2010). In some cases students who do not have access to either Internet or digital devices in education environments cannot experience the outside world within the classroom walls (Groff, 2013). ICT has been able to present knowledge in different forms, making it more vivid, attractive and mind imprinting for the students.
Additionally, the use of ICT has been designed and implemented with the purpose of improving existing study plans and having a positive impact on students, and it has proven, if used appropriately, to provide adequate support for the achievement of educational objective (Carneiro, Toscano & Díaz, 2009).
As a result, the pedagogical focus has changed as it now places the student at the center of the learning process (Pedró, 2011), which in some cases has created uncertainty and discomfort in the teacher. The pupils may now know more about technology than the expert on the subject contents. Thus an immeasurable strain has been placed on the educator (Ebner, 2017). The professor has been forced to adapt to the new technology being used, feeling uncertain of what he or she must do with it.
Furthermore, ICT has increased the support of education by providing a great variety of pedagogical resources and approaching situations to the classroom environment that were not possible before. Therefore, the need for continuous evaluation of ICT integration practices and their impact on student’s performance is necessary so as to make the correct decision about its use and convenience in time assignment.
Pedró (2011) states that ICT is nowadays used to communicate with people, as well as get, produce and share information through networking in different types of devices, applications and digital contents. Moreover, Selwyn (2007) states that use of ICT in education has revolutionized the learning process by introducing new resources into the classroom environment. That is why education technology plays a major role in five different areas: school systems, evaluation systems, students, teachers and school centers.
Moreover, Koper (2004) declared that ICT has also changed the learning environment, making it possible for people in distant places to connect to a virtual class through Internet. This has enabled them to keep up with their studies without time or space constraints, by adding flexibility, efficiency and becoming inclusive in the learning process, as Pedró also indicated in 2011. In turn, society members can be better prepared to cope with the new competitive and demanding working world.
The digital integration has been undertaken not without profound changes in the academic arena (Henessy, Ruthven & Brindley, 2007). ICT’s have become unprecedently indispensable tools. Their proper use is considered one of the required skills for the 21st century. Thus, it must be evaluated to determine its adequacy. Besides, Roycroft-Malone, Seers, Titchen, Harvey, Kitson and McCormack (2014) state that in any transformation, past experiences should be taken into consideration, as well as evidence of the knowledge of good practices, the exchange of research findings, and the use of technology to share that information.
Katzman (2010), on the other hand, points out that social transformation, due to the use of technology, has generated digital literacy with working and personal benefits, strengthening social, cultural and economic activities. Currently schools are proud of using ICT, and they advertise it as their main contribution to education. Unfortunately they are making use of it only for a short period of time, having less access to information and therefore weaker knowledge. Hicks (2010) states that having such access does not guarantee the better performance of the students. At the same time, technology integration should be adequately and continuously evaluated.
Furthermore, the availability of digital devices with Internet access at home may encourage the students to navigate more frequently and therefore increase self-learning in an autonomous manner, with time flexibility (Shroff & Vogel, 2009). This may awaken some curiosity in the students and trigger more in depth research on a topic of their choice.
In addition, technology has generated fairer methods to evaluate students and design activities. Formative and summative assessments now take the evolution of personal knowledge into consideration and also provide constant and personalized feedback (Goodison, 2003), with a significant benefit for the students.
Moreover, the new challenges have to be accepted by both the pupils and the teachers. The latter, in turn, have to prepare better for the changes ahead. So far, they have been skeptical about technology integration within their methodology. Hence, educators have to consider that: “Technology integration is not about technology itself, it is about the contents and effective instructional practices which are transforming the educational environment” (Earle, 2005).
According to Carneiro, (2009) ICT is the main lever in the education transformation process. Mooij (2007) indicates that mankind is on the verge of a technological transformation that could cause uncertainty as the learning institutions adopt a new teaching-learning process. Integrating ICT and making it challenging for the actors involved has created three paradigms: education as a close service industry, schools that teach schools to learn, and from “associationism to constructivism”, as people become the builders of their knowledge and do not limit themselves to register what the world wants to show or give to them.
One of the primary objectives of integrating technology into the learning process should not be mainly to acquire a lot of computers but to know the correct time and methodology to use them in the classroom. That involves creating strategies, developing pedagogical resources for the appropriate use of ICT in the classroom. In the future there will be more technological changes and what is and isn’t being done should be thoroughly evaluated. Hodson (2010) mentions that the changes generated due to ICT integration in education could provide short and long term results. Undoubtedly, educational systems have a great deal to learn, and constant cooperation from different educational actors will be necessary so that the past mistakes are not recurrent.
Furthermore, to understand the importance of ICT integration evaluation in learning English as a second language, a survey about the practices and additional support provided to the students as a result of the use of the Tell Me More software in a private Mexican University was carried out, with its impact on students’ academic performance, with the intention of knowing whether its use, frequent or sporadic, is a convenient supporting tool in their English as a second language learning process.
The general objective guiding this research was:
Evaluate the characteristics of the Tell Me More software used at a private Mexican University, with the intention of knowing if the English language acquisition process truly benefits from it and if it has a positive impact on the students´ academic performance.
The specific objectives were:
To learn the guidelines and recommendations of the English department for the Tell Me More software use.
To find out the use teachers give to the Tell Me More software.
To learn more about the use students give to the Tell Me More software.
To corroborate the deficiencies mentioned by either the coordinator, the teachers or students of the Tell Me More software.
To verify the frequency and difficulties in the use of the Tell Me More software and a convenient access to the Internet.
To evaluate the Tell Me More software impact on academic performance quantitatively and comparatively with results in tests and year-end scores from different groups.
All of the above become the interest of this investigation because Bonilla (2003) states that new technologies were not conceived for education; they do not participate or appear naturally in the education systems; they are not sought after by the teachers; they do not adapt easily to pedagogical use, and probably in the future they will only be partially developed to satisfy the demands of the education sector (p. 120).
Therefore, three questions arise:
Does the Tell Me More software used at the language laboratory at a private Mexican university provide support for English as a second language students, from the coordinator, teachers and undergraduates’ perspectives?
What impact does the integration of the Tell Me More software have on academic performance in students of English as a second language?
What characteristics should an educational software have to provide support for students of English as a second language at the university level?
Bearing in mind the objectives mentioned above, two axes were followed for this investigation. On the one hand, the use of the mixed method was considered pertinent due to its peculiar searching features, which gave direction to the data collection and analysis; on the other, the combination of quantitative and qualitative approach throughout the stages of the research process, following Valenzuela and Flores (2012). Therefore, once the inquiries were made about the use of the Tell Me More software by the coordinator, the teachers, and the students, a qualitative approach was followed, while a quantitative method was used in order to analyze students’ scores at the end of school year in their English language courses.
At the same time, a number of indicators proposed by UNESCO in their Guide to measuring information and communication technologies in Education (2009), were used to collect and categorized data about the use of the Tell Me More software at the university’s language laboratory with the purpose of making some recommendations on whether to stop using the software, or to use a more suitable version of it. These indicators have standardized definitions of key concepts that provide multiple angles for the assessment of ICT penetration in education systems from a comparative perspective (UNESCO, 2009). In Table 1 we show the UNESCO indicators taken into consideration, due to their relevance for topic researched, they appear with the area of interest according to the number.
It is of great importance to know the real advantages of ICT use in English as a foreign language considering that: “these indicators enable to gauge the performance ratio and the maturation of the systems as an essential instrument to favor the decision-making, regarding educational policies based on solid data and specific knowledge (Severín, 2010).
The research instruments, indicators and the type of methodology used to answer the research’s three main questions are shown in Table 2.
Each instrument was designed differently considering the characteristics each of the participants possessed and their dissimilar circumstances due to their position in learning English as a second language; the objective was to obtain answers to the questions posed, without leaving different participants’ perspectives unattended. T
he semi-structured interviews were conducted with all participants. They provided a good source for data collection and comparison from all actors involved, following the UNESCO indicators, with respect to the use of the software. These interviews were elaborated taking into account the previously mentioned objectives. The interviews were intended to collect relevant information about the use of Tell Me More through the University’s language coordinator, professors, and students so that a more detailed explanation and perspective of each of those participants could be obtained (Barragán, Salmán, Ayllón, Sanjines, Langer, Cordova and Rojas, 2003).
A 22 question interview was conducted with the coordinator to identify the guidelines followed by the language department with regards to the number of language professors, students, classes, courses, computer laboratories, software licenses, as well as the characteristics of Tell Me More. As for the professors’ perspectives towards Tell Me More, an interview was performed, which involved answering 14 questions about the attendance to the language laboratory, the percentage of the total class time invested in using the software, and what its benefits and disadvantages were. As for the students, their interview consisted of 11 questions which included the same categories as above.
Regarding the features that an educational software should have, a Pedagogical Usability Test was answered which included 10 questions. This test consists of observing website or software users while they perform a task, in order to evaluate the website’s design (Liu et al., 2015).
This test was created by Nielsen (2000) and tailored by Nokelainen (2006) to suit the educational needs. He divided the test into 10 pedagogical usability criteria: learner control, learner activity, cooperative-collaborative learning, goal orientation, applicability, added value, motivation, valuation of previous knowledge, flexibility, and feedback. The usability test was used in this research taking into account the specific benefits it could provide. It included the criteria in order to deliver important feedback regarding the undergraduates’ usage of Tell Me More software. It is important to mention that there hasn’t been that much research using this kind of pedagogical usability testing in the area of software in English as a second language (Liu, Traphagan, Huh, Koh, Choi & McGregor, 2008; Razaei, 2010).
In this research there were 162 participants from eight different level English courses with different career studies and ages, selected according to their class schedules; eight teachers with between one to ten years of experience also participated as they were in charge of the previously mentioned groups, and one coordinator from the English as a second language department, who has been in charge since 2010.
This investigation was executed following the four stages proposed by Barraza (2010): planning, implementing, evaluating and dissemination-socialization.
Planning. An area of research is identified then a design for the educational intervention is generated, as well as the design for a possible solution. It is necessary to plan ahead and program the strategies and the activities with their resources.
Implementing. The planned activities are performed. Interviews and the pedagogical test are answered by a specific number of students, professors and the language coordinator. Data is collected.
Evaluating. This stage consists of analyzing the characteristics that bear the inclusion of this software in the students’ learning of English as a foreign language process, with undergraduates from different majors and their academic performance, in a particular and complex real situation, to evaluate the benefits that this software could offer (Yin, 2003).
Dissemination-socialization. Once the results and the recommendations for giving a solution to the problem are identified, it is necessary to share the findings with the university.
In order to know if Tell Me More software works as a support tool in the English learning process at this university, its current situation and use, the strengths and weaknesses of the mentioned software were analyzed by selecting and using different ICT in education indicators from UNESCO (2009) model, which are depicted in Figure 1.
In the first place, ED9 bis indicator, the proportion of grades using ICT-assisted instruction, allowed to establish the percentage of groups using Tell Me More as an additional tool in the language department. This indicator showed that only 18%, which is 162 students of the total amount of 900 students in this institution, are able to use this software.
Additionally, ED10 indicator -average number of hours per week of ICT use- showed that this software is used in 25% of the regular class time, by 162 students who use the software, which is equivalent to one hour and a half per week of class at the most. This information establishes a possible correlation between the students’ performance and the English learning process.
Furthermore, the ED11 indicator - average number of hours per week of ICT use in practical exercises - illustrated that the Tell Me More software is used only 5% of the time for practicing, even when this tool was aimed to offer enough opportunities for students to do so more frequently, as is depicted in Figure 2.
At this point, there was a significant difference among the answers given by the coordinator, the teachers and the students. The coordinator answered that Tell Me More software is used for practical exercises about 25% of the classes; the teachers said that it is only used 20% of the time, while the students reported using it only in 5% of the classes (Figure 2).
On the aspect of the impact of Tell Me More on the students’ performance, an improvement in the four main skills of the language (listening, speaking, reading and writing) was shown. According to the coordinator’s numbers, this software shows an improvement of 20%, while it was 15% for the teachers and only 10% of the undergraduates as depicted in Figure 3 for the UNESCO indicator ED49.
In addition, reading was the skill with the lowest improvement, being 3% of the students who confirmed it, while teachers considered a 9% and the coordinator expressed a 12% difference by using Tell Me More, as shown in Figure 4, again reporting significant differences. Reading is the least practiced skill in the use of Tell Me More software, as shown in Figure 4, also taking into account UNESCO indicator ED49.
Some limitations and weaknesses of the Tell Me More software were reported in the interviews. 90% of the students agreed that this software was ineffective for practicing speaking. At the same time, teachers said that it was only 7% who considered it effective and the coordinator reported 10% effectiveness. These results are shown in Figure 5, considering UNESCO indicator ED49.
As for ED22 indicator -proportion of schools with computer-assisted instruction- showed that 900 out of the 5000 students at this institution are learning English, representing 18% of the total population of the campus. Even when this software is installed on all the computers at the laboratory, only 14% of those students use it on a regular basis as shown in Figure 6.
Concerning the measurement of students’ improvement in the acquisition of English language by using Tell Me More, it was important to evaluate the relationship between ICT use and students performance at the university. Such relation showed that there was no significant improvement in the student’s performance.
As mentioned by Smeets (2005), ICT works as an effective tool in autonomous language learning. Nevertheless, the results of this study showed a low impact of the Tell Me More software in English classes.
The ED24 bis indicator -proportion of educational institutions owning license for or subscribing to virtual experiment laboratories- evaluated the number of Tell Me More licenses at this language department. The coordinator answered that all computers (25) had the software installed. However, the number of available computers is not enough for the total number of students, leading to a lack of opportunities for students to practice the language by using a computer.
Regarding the ED49 indicator that measures the promotion rate of learners in grades, receiving ICT-assisted instruction, the interviews showed that only 20% of the total number of students promoted to the next level reported using the Tell Me More software, as shown in Figure 7. Therefore no impact of its use could be detected.
As for the ED51 indicator -ICT-assisted instruction performance ratio- it was evaluated and showed that only 20% of the students using this software improved their grades by only 2%. This is not a significant number to justify the investment of acquiring other software of this kind.
Furthermore, taking into account the information retrieved from the interviews at the language department, no relevant English language learning improvement was detected due to using this software. This result may be directly related to both students and teachers’ perceptions of this software, because they considered the Tell Me More software outdated, as the activities seemed monotonous and unattractive for them.
Area (2008) stated that some digital resources are ineffective and inadequate for some students’ characteristics and features. The ICT overuse can lead to the lack of motivation in students because the variety and quality of activities that are provided have a direct impact on students. Considering this information, this study showed a low impact of the Tell Me More software in students’ performance, as they see this tool only as a requirement for them to approve the levels.
The Pedagogical Usability Test was applied to some students in the Language Department in order to learn what characteristics an educational software should have so it can provide adequate support to students of English as a second language at the university level. This test evaluated the characteristics of the Tell Me More software. According to the results given by the test, 50% of the students reported Tell Me More software neither to be effective, fast nor easy to use; 10% answered it was effective, and 40% said it was sometimes effective, as shown in Figure 8.
Moreover, 48% of the students found activities in the Tell Me More software unattractive. This is in comparison to 20% of them who considered it as a good tool for practicing English, as shown in Figure 9.
Additionally, more than 80% of the interviewed students considered that the activities and lessons offered by Tell Me More have a clear objective, despite the fact that 100% of the students answered that this software does not provide opportunities for them to interact with other users. This result is of vital importance since interaction among students allows them to support each other. With regards to this information, Squires and Preece (1999) describe an educational software as a learning facilitator that provides students with enough opportunities to interact with others. However, that was not the case in this investigation.
The Pedagogical Usability Test also showed that 50% of the students found exercises somehow natural and realistic, whereas 38% found them useful for real communication: 12% of the students considered this software to provide an added value for their learning process, while 68% considered it was irrelevant. At this point, it is important to highlight that students are facing an ICT centered learning process, so they need to have the knowledge, abilities, techniques, and skills that are necessary for them to use ICT. This is the reason why students should find these tools appealing and attractive. (Rojas, Mendoza & Gómez-Zermeño, 2013).
Regarding students’ motivation, 25% of them were motivated; and 20% were not, as is shown in Figure 10. Nevertheless, 80% of the students reported Tell Me More to be flexible in connection and schedules as one of the main software strengths. In addition, 80% of the students considered that some previous knowledge of the language is necessary to practice most of the exercises and activities presented by this software. Finally, 75% of the students considered the feedback provided by Tell Me More as relevant for their English language learning process. In, Figure 10, results regarding the Pedagogical Usability Test are depicted.
Discussion and Conclusion
The main objective of this research was to evaluate the characteristics of the Tell Me More software used at a private Mexican University, to know if it produced a real benefit in the acquisition of the English language impacting the students’ academic performance. In regards to these findings, it became evident that once the software becomes outdated reduces its positive impact on the students’ learning. In this case it was noticeable that only 14% of the students used it. This could be due to the fact that they no longer found their needs and practice satisfied; therefore they preferred to avoid its use as stated by Higgins, Xiao and Katsipataki (2012).
Regarding to the first question made to the coordinator, the teachers and the students about the use of the Tell Me More software and the support it provides, the answers given in the semi-structured interviews showed a discrepancy among pedagogical practices. These differences were significant and in some cases, it increased from 10 to 15%, which is an important figure. It became evident that the teachers and the students, in general, did not like using the software as it had proven not to have a significant positive impact on the students’ academic scores.
Concerning the second question about the impact of the Tell Me More software integration for students of English as a second language, 25 out of 25 computers had this software installed. Nevertheless, the amount of licenses was not enough for the number of students in the University, 900 learners of English as a second language, as they were not able to work simultaneously. Moreover, the interviews showed that the time set aside for the students to practice with software was insignificant, leading to a low improvement in their language acquisition process, due to the lack of use of the software.
ED49 and ED50 indicators pointed out that only 20% of students using this software showed a slight progress in their grades and that a similar percentage of students had a better performance in the English language. The information retrieved from the interviews revealed there was not any significant improvement in the students’ performance in connection with the use of the Tell Me More software. This result can be directly associated with both undergraduates and teachers’ perceptions of the program, since they considered it old and the activities monotonous and unattractive.
As for the third question, the characteristics that an educational software should be conducive to provide support to students of English as a second language at university level, more than half of the students did not think that using Tell Me More software use gave them an added value to the learning process, as they considered it an irrelevant tool. Consequently, they did not feel motivated to use the software.
Furthermore, some relevant comments could be identified when selecting a pedagogical software as an underpinning to studying English. Setting goals, flexibility, feedback and the necessity to use previous language knowledge were the four characteristics that students liked the most about Tell Me More. Therefore, these features, as well as the interaction among other users, are the groundwork where any educational software needs to be based on for the process of learning English as a second language.
Although the pedagogical software does not necessarily give users an added value, it needs to produce motivation in students through attractive and realistic activities. In other words, software must enhance communication skills in students through a collaborative network, where they can be urged to apply their knowledge as stated by Hmelo-Silver (2004).
As a final thought, in any software evaluation variables such as the amount of time it is used in class, participants’ perspectives, pedagogical advantages, updates of the software etc., should be considered in the interest of providing meaningful learning for the students, which eventually will allow them to develop the necessary skills to use the language properly. The results of such evaluation will determine the steps to follow.
Moreover, it is not recommended to integrate ICTs as a replacement for time in the classroom, but as a support, a tool to improve the quality of education and introduce students to real life situations empowering them to cope (Quillen, 2012). The success lies in the appropriate decisions that can come from proper evaluation of ICT- assisted education procedures, which can only derive from constant and adequate research.
Considering the results obtained through this investigation, including software in the classroom per se neither guarantees an improvement in the performance of students, nor does it demonstrate a tangible and significant progress in their professional preparation if it is not carried out under the correct conditions. Therefore, continuous evaluation is strongly suggested.