Data Analysis

Context

The students were researching an ancient civilisation and giving a 2 minute speech on 3 aspects of life for example food, clothing and weapons. They had effective modelling of the research process by their teacher in the activities prior to commencing research. A unit on middle ages was used to teach the research skills they would be using to do their own research on an Ancient Civilisation. The students’ work-books showed that information was presented not just as facts, but as explanations for why things were the way they were. A comparison chart was done between Medieval Times and Modern Australia. Dot points about weapons did not just mention the weapons, but who used them and why. Student work samples This is an example of an inquiry that falls on the inquiry learning spectrum between the highly structured, directed inquiry and open, free-ranging inquiry. Students were encouraged to seek information. Students were given the hands- on experience of a medieval incursion to spark their interest in history. The teacher chose the broad topic – ancient civilisations, but students are given choices within that topic. Inquiry learning is bounded by the curriculum  (Robins, 2005). Students were given a choice of the civilisation they would research, and within that civilisation they could choose the topics they would investigate. Student choices were made on the basis of interest and availability of information. The inquiry was bounded the SOSE Essential Learnings curriculum document (QCAR, 2007). Read the raw data – Interview with the Teacher  for more background. All names used in this report are pseudonyms.

Abstract:

This study looks at a class of year 6 students as they complete a research project on an ancient civilisation. As most research is completed on the internet, data is examined to identify search behaviours, depth of understanding and use of information. The project is examined as an example of inquiry learning and recommendations are made for future practice in the context of curriculum documents, and inquiry learning models in particular, inquiry using ICTs.

 

What sort of inquiry is this project?

The Australian Curriculum draft document for history year 6, asks students to identify questions to inform an historical inquiry (ACARA, 2012) . Inquiry learning has its roots in Constructivist theory and has sound support in educational research papers.  In this post I will review the literature on inquiry learning and describe the necessary elements for good inquiry which enable higher order thinking and deep understanding. I will then analyse the results of my inquiry research project and interpret results with reference to the literature.

The Constructivist theory of learning, proposed by Dewey in the early 20th century still has relevance today, as these ideas underpin 21st century skills, including inquiry learning. Constructivist theory is characterised by active, ongoing learning where new information is built on to existing knowledge and is used to construct deep understanding (Kuhlthau, Maniotes & Caspari, 2007).  Barell (2010), lists 21st century skills as critical thinking, problem-solving, collaboration, creativity, self-direction, leadership, adaptability, responsibility, global awareness and inquiry. Inquiry is characterised by good questions, purposeful investigation, critical thinking, drawing conclusions and reflection. Good inquiry units can be used with students of all ages and abilities (Barell, 2010).

To facilitate inquiry learning, teachers must provide an environment where learning is challenging, social and fun. Inquiry learning models consider the whole child and plan for cognitive, affective and behavioural stages. Kuhlthau’s ISP model, figure 1., illustrates the stages of the inquiry process and recognises the stages in the cognitive, affective and behavioural domains (Pickering-Thomas, 2004).

Figure 1. Kuhlthau (2004) Model of the ISP

 

Figure 1 – Kuhlthau’s Information Search Process retrieved 29/09/2012 from

http://comminfo.rutgers.edu/~kuhlthau/information_search_process.htm

Researchers agree on the principles of inquiry learning listed below.

  1. Students are actively engaged in the process. They have choices about the content, ways to learn and ways to share. Teachers and students share decision making. Students are involved in reflection and self-assessment often through journals or portfolios.
  2. Students learn by building on what they know and investigate authentic problems or real world problems.
  3. Students develop higher order thinking through guidance at crucial times, continuous assessment and feedback.
  4. Students have different ways and modes of learning.
  5. They learn through social interaction with others, collaboration and peer and teacher feedback.
  6. Instruction and experience is in accord with their cognitive development. Teachers revise, modify and elaborate plans.
  7. Questioning is at the centre. What do I already know? What more do I want to know? How do I find out? What did I learn? How do I use what I learned? What will I do next time? (Kuhlthau, 2007). A similar framework is KWHLAQ (Barell, 2010) figure 2 below.

(Barell, 2010; Callison, 2006; Harada, 2004; C. Kuhlthau, Maniotes, L., Caspari, A., 2007; Robins, 2005; Wilhelm, 2007)

Figure 2. Questioning framework for inquiry learning Barell (2010)

Results

Question 1 – Write down what you know about the topic.

Figure 3- Class Results for Question 1.

Analysis

The data in figure 3 shows that the number of facts peaked at questionnaire 2 when the students were halfway through the process of finding information. There were few explanations, with only 1 recorded in questionnaires 1 and 3, and 5 in questionnaire 2. This shows that few students were gaining a deeper understanding of the topic. There was only 1 conclusion which was written in the second questionnaire. This may be an indication of unreliable data as the students felt under pressure to complete questionnaires quickly and had little motivation to give complete answers. Kuiper (2005) found that students writing assignments have a higher motivation than when writing for a researcher who will not be grading their work. This concept is supported by viewing the student work sample of the speech. Example of Student’s public Speech. This student did not write any explanations or conclusions for question 1, but has four explanations and a conclusion in her speech. This difference between the data and the work sample may also be due to the presentation of the project as a speech. The students were going to be telling the class about the ancient civilization they had researched. To make the talk interesting, it needed to be more than just a collection of facts.

Quantitative Data – Number of facts recorded by seven students of different ability -students’ names are pseudonyms

Figure 4 – Number of facts recorded by tracked students

Analysis

Figure 4 Shows that for the high achieving students – Mary, Marie and Shane, the number of facts increased with each questionnaire. While the number of facts written by Shane appears to stay the same, overall the number of facts written by Shane increased as she wrote new facts for each questionnaire, where other students repeated facts given on previous questionnaires. Average students Margaret and Greg followed the class pattern with the number of facts peaking in questionnaire 2. Low achieving students Bradley and Jason wrote few facts overall indicating difficulty in accessing information and lack of motivation to write answers to the questionnaire.

Development of fact statements over the course of the inquiry – students’ names are pseudonyms

Figure 5 – Development of Fact Statements

Analysis

Figure 5 (using different students to the tracked group) shows that typically the number of facts and the quality of statements improved between the first questionnaire and the second.  For example Ellen wrote, “what they use for medicine” on questionnaire 1. On questionnaire 2, she extended this fact by adding “how advanced they were at curing disease”. Allen extended ‘built pyramids’ to ‘built pyramids from large blocks of stone’. There are also some indications that students did not achieve understanding of information. For example in questionnaire 3 Allen says “the pyramids were their castles”. James’s responses show that there is generally a lower quality of fact statement in questionnaire 3.

The other element that explains why students are not demonstrating deep understanding is the cognitive development of the students. Students are performing mental operations on a concrete level and are not capable of abstract thought until they are 12 years old. They should still be involved in inquiry to gain experience for later use. They should be able to recall, summarize, retell and extend. These students are on the cusp of being at an age where they can explore ideas from various sources and incorporate them into their own thinking (Kuhlthau, Maniotes & Caspari, 2007).

Question 2 – How Interested are you in this topic?

Figure 6 – Collation of data from Question 2

Analysis

Figure 6 shows that the students’ interest increased during the research task, and only a small number indicated they were not interested at all or only slightly interested. Students were not ‘greatly interested’ by the third questionnaire, possibly because the assessment had been done and they were ready to start their next topic.  However, only 3 students were “not much interested”. The reasons for this are that they were able to choose their own topic and were able to change if they chose to, and had a choice within the broad topic as well. The ability for students to make choices and change the topic of research has a positive effect on attitudes (Kuiper, 2005). Students were therefore able to pursue their own interests and were not all completing the same work. This is an important aspect of inquiry learning as when the students are interested and can relate the information to their own lives in some way, deeper understanding is gained. While the data gathered from question 1 does not prove that deeper understanding was gained, student work samples indicate that the students did more than just collect facts.

Question 6 on Questionnaire 2 – the affective domain

frustrated 1
overwhelmed 4
confused 2
confident 18

Figure 7 – Table showing the number of students for each emotive descriptor

Analysis

Question 6 on questionnaire 2 asked for students to rate how they were feeling according to the table. It can be seen that this class are quite confident in their abilities. By and large they are fluent readers and as such can read well on the Internet. They have access to computers and the internet at home and literate parents who support their learning. Those who were feeling overwhelmed and confused, felt that way because of the large amount of information they were finding in their searches and their inability to read much of it.  Some were frustrated due to time pressure. Slow internet and lack of time can affect student motivation (Kuiper, 2005). Five of the tracked students were feeling confident at this stage, indicating they may have skipped from the topic selection phase to the resource collection phase. Only Marie felt confused. As she is a high achieving student this would be expected as she is probably in the exploration stage of the ISP. Jason was feeling frustrated. As a low achieving student, searching the Web would have been very difficult for him. Students need time to read and reflect and guidance in making sense of information. Without guidance, the ISP can become a copying exercise with little real learning  (Kuhlthau, C. 2010). (note – students’ names are pseudonyms)

Guidance to find suitable Websites is essential at this stage. Students stop searching when they feel tired, bored or feel under time pressure. They are bored quickly by websites that have large amounts of information and few visual elements. Students are often confronted with information not written for them. They find it difficult to find good sites and they generally do not have enough patience to read descriptions of sites. Students’ characteristics affect quality of information found (Kuiper, 2005). The exploration stage is a crucial stage for teachers to provide support and guidance (Kuhlthau, Maniotes & Caspari, 2007). There is a vast difference between a routine search and a complex issue. I do not expect to see clear evidence of all of the stages of Kuhlthau’s ISP as this project is more of a routine search than a complex issue. Confusion and uncertainty increase during the exploration phase when using ICTs, as students find it more difficult to work through on their own. Most of the students are feeling confident indicating that they are not moving through the phases of Kuhlthau’s ISP, but have skipped from topic selection to resource collection. Interest increases with confidence as they move past the exploration phase. Expectations and instructions from teachers are an important element in fostering deeper learning. Time constraints did not allow opportunities to explore the topic and form a focus. Students were left alone to search. It is common for students to be left alone to search which results in narrow fact finding, copying and little real learning. Focus requires time to reflect and a new way of looking at something. (Kuhlthau, Maniotes & Caspari, 2007)

Data on Students topic choice and use of questions

Figure 8 Data from Survey 1 on topic choice, collaboration and use of questions

Figure 8 shows why they were interested and happy with the results. They were given opportunity to select their topic within limits. It is important to notice here, though, that they did not think of questions before they began to search and they did not collaborate. This would have helped them do a more focused search and have less of an issue of too much information. A good questioning framework is essential to get higher level answers and this may be another explanation as to why not many explanations or conclusions were given in answer to question 1.

Question 7 on Questionnaire 3 : Reflection on how I feel about my Research

unhappy  
confused
confident 10
happy 15

Figure 9 –Table of Responses to Question 7 on Questionnaire 3

Confident – I think it turned out ok. Happy – I’m really happy with how it turned out.

Figure 9 shows that by Questionnaire 3 at the end of the research task, all students were feeling either confident or happy indicating that they has been able to achieve a standard they were satisfied with. This may indicate that they had guidance at necessary points. They were able to present a speech where they were happy with the content, though some were disappointed with the delivery due to nerves. Confidence is positively correlated to an increase in knowledge. (Kuhlthau, Maniotes and Caspari, 2007)

Question 3 – How much do you know about this topic?

Figure 10 – Collation of data from Question 3

Analysis

Figure 10 shows that students felt their knowledge increased as they completed their research. By Questionnaire 3 most students felt they knew either “quite a bit” or “a great deal”.  It is interesting to compare this response, to the answers given in question 1. In questionnaires 2 & 3 for question 3, most students felt that they knew quite a bit, only 1 student felt they knew ‘not much’. However, some students did not write answers for question 1: ‘Write what you know about this topic’. This is further evidence of a lack of motivation to answer question 1 properly.

I will present the next two graphs together to allow comparison.

Question 4 – During your research project, what did you find easiest to do?

Figure 11 – Collation of data from Question 4

Figure 12 – Collation of data for Question 5

Analysis of Questions 4 and 5

Figures 11 and 12 show that students found it easiest to search for information using ICTS but also hardest. They found it easiest to summarise information into their own words, but also hardest. Anxiety over the public speech eased as their confidence in their information increased.

Students found ICT at school difficult and complained of sites being blocked and the internet being slow. They also found it difficult to read some of the information they found, and to summarise information into their own words or dot points. They were not allowed to cut and paste, having to write dot points into their notebooks, which meant they had to analyse information and choose what to write in their own words. Some expressed anxiety about having to do a public speech.

Internet Search Behaviours of Students

Figure 13 – Data from Survey 1 how do students search for information

Raw data from survey 1 was used to create figure 13. Figure 13 explains why students have difficulty finding specific information. They all use Google, but few use quotation marks, advanced search or Boolean operators (and, or, not). Quite a few students use key words, but further data collection would need to be done to ascertain the effectiveness of the key words. Researchers have found that difficulties occur around use of keywords for searches and students have problems extracting proper information. Search engines are unfamiliar and students mismatch keywords and misuse Boolean symbols. They retrieve data based on titles rather than content (Bowler, 2001; Hwang, 2011; Kuiper, 2005). Bowler (2001) and Kuiper (2005)  found that search skills are hampered when students do not demonstrate inquiry skills such as brainstorming, listing key concepts and formulating questions and do not understand how search engines work (Bowler, 2001; Kuiper, 2005). When these issues arise, students spend much time searching and collecting information and less time thinking about how to use information to answer questions. Searching for precise information makes high demands on students. Students complained of having difficulty finding specific information. Figure 14 gives an example of tracked student Marie’s answer to question 5.

Students need a great deal of help to search on-line. Being able to choose a topic within a broad category, as these students were able to do, and having the ability to change their choices has a positive impact on motivation and search behaviour (Kuiper, 2005).

Students in this study were given the tip by their teacher to add ‘for kids’ to their search terms. Further data collection would be needed to see if students used this tip, and if they found it useful. I have done a trial search to determine effectiveness.

Google Search – ancient Egypt

Screen shot 1

Google search – ancient Egypt for kids

Screen shot 2

Comparison of screen shot 1 and screen shot 2 indicate that this is an effective strategy. Adding ‘for kids’ produced Websites with information suitable for children.

Questions 4 and 5 were further analysed against the AASL standards for the 21st Century Learner (AASL, 2007).

Answers to Questions 4 and 5 compared to Standards for the 21st Century Learner (AASL, 2007)

Figure 15 Data showing answers Questions 4 and 5 compared to the AASL Standards for the 21st Century Learner

The data in figure 15 clearly supports Kuiper’s (2005) findings. The element that was the hardest to do, was ‘mastering technology tools for accessing information.’ 34 out of the 60 responses indicated that this was most difficult. ‘Creating new products that express new understandings e.g. note taking’, was the second most difficult. The students found it easiest to ‘find, evaluate and select appropriate sources’ and to ‘create new products to express new understanding”. Once they had completed the difficult task of note-taking, creating the speech was easy.  Once they had found Web-sites, they found it easy to decide whether the site had appropriate information. This is because their information literacy skills off-line are quite high. Information literacy skills are an integral part of inquiry learning.

Inquiry learning models use information from many different sources to help students develop information literacy.

Other Sources of Background information

Figure 16 – Data from Survey 1 for other sources of information that students use

Figure 16 shows that when choosing a topic for investigation, students used background information gained from watching movies and playing games. Their interest was stimulated by the images they had seen.

How Students Choose Books for Research

Figure 17 Data from Survey 1 for how students choose books

Books have been largely forgotten by students who just want to use Google. They sometimes choose books recommended by friends and that is all. Books should still be used first to build knowledge from a reputable source, so the students can be critical users of the Internet. Kuiper (2005) found that use and effect of technology is linked to the social context in which it is used. Survey findings shown in figure 20 indicate that this social element is missing for students.

Where did you learn to research on the internet?

Figure 20 – Data from Survey 1 Where did you learn to research in the internet?

This shows the students’ perception of where they learned to search. Apart from their parents, they felt they learned by just doing it by themselves. This means they have not had the social aspect of Internet use.

Research Behaviour at Home for comparison with Search Behaviours at School

Figure 21 Data from Survey 2 – Research behaviour at home

Raw data from survey 2 was used to create figure 21

Figure 21 shows that research behaviour at home is much the same as at school. The only exception is that frustration is lessened as internet speed is faster and fewer sites are blocked.  Research at home is still done without use of effective keywords and Boolean operators. Research has proven that students’ skills and understanding of key words and Boolean operators, and ability to assess relevance and reliability of information greatly effect information retrieved. Student characteristics of patience and motivation improve involvement in searching, but do not guarantee success on their own. Both at home and at school there is the influence of the Web environment, the language and tools used. Natural language is not effective for searching.

At school, teachers should consider font, use of illustrations and reading level of language on the site when selecting Websites. Characteristics of the task are important and are largely determined by the questioning framework used. The more specific the information required, the more difficult the search. Students actively engage with online information and communications networks (Blackall, 2005; Kuiper, 2005).

Question 6 on Questionnaire 3 : What did you learn in doing this research project?

Figure 22 – Collation of Data from Question 6 Questionnaire 3

Figure 22 shows that the most common answers to question 6 were non-specific facts, followed by facts. It would be hoped that more students would have learnt search skills or ICT skills. This shows, that students are not immersed in a holistic environment with all the characteristics of inquiry learning. Examples of each of the categories are shown in Figure 23.

Examples of responses to question 6

Figure 23 Examples of responses categorised for figure 22

Tracked Students Responses to Question 6 -students’ names are pseudonyms

Figure 24 Total of responses for tracked students

Only facts, non-specific facts and search skills are tallied in figure 24 as no other responses were made by the tracked students. These students showed a higher number of non-specific facts as did the class as a whole. There is no difference between low achievers and high achievers in types of responses.

Overall this inquiry project is on the more structured end of spectrum of inquiry learning. Results of data analysis are supported by information in the literature for this type of inquiry using the Internet as a source. Some suggestions for future action have been included here. In comparison to Kuhlthau’s ISP model, this project appears to skip the stages of topic exploration and focus formulation because it lacks a strong questioning framework. Future posts will make further recommendations for action beginning with planning for inquiry and use of questioning frameworks.

References

AASL. (2007). Standards for the 21st Century Learner, from http://www.ala.org/aasl/sites/ala.org.aasl/files/content/guidelinesandstandards/learningstandards/AASL_LearningStandards.pdf

ACARA. (2012). The Australian Curriculum – history-year 6  Retrieved 26/08/2012, from http://www.australiancurriculum.edu.au/Year6#learningarea=H

Ackerman, E., Hartman, K. (2005). Searching and Researching on the Internet and World Wide Web (4th ed.): Franklin, Beedle and Associates.

Barell, J. (2008). How do we Plan for Students’ Questions? Why are school buses always yellow? Teaching for inquiry PreK-5 (pp. 45-61): Corwin Press.

Barell, J. (2010). Problem-Based Learning: The Foundation for 21st Century Skills 21st century skills : rethinking how students learn (pp. 175- 199). Bloomington, IN: Solution Tree Press.

Blackall, C. (2005). Digital Literacy: how it affects teaching practices and networked learning futures-a proposal for action research. International journal of instructional technology and distance learning, 2(10), 1-11.

Bowler, L., Large, A., Rejskind, G. . (2001). Primary School Students, information literacy and the Web. Education for Information, 19(3), 201-223.

Callison, D. (2006). Information Inquiry : Concepts and Elements The blue book on information age inquiry, instruction  and literacy (pp. 3-16). Westport: Libraries Unlimited.

Harada, V., Yoshina, J. (2004). Chapter 1: identifying the inquiry based school Inquiry learning through librarian-teacher partnerships (pp. 1-10). Worthington: Linworth.

Hwang, G.-J. (2011). An information-summarizing instruction strategy for improving the web-based problem solving abilities of students. Australasian Journal of Educational Technology, 27(2), 290-306.

Kapitzke, C. (2003). Information literacy : a review and poststructural critique. Australasian Journal of Language and Literacy, 26(1), 53-66.

Kuhlthau, C. (2010). Guided Inquiry: School Libraries in the 21st Century. School Libraries Worldwide, 16(1), 1-12.

Kuhlthau, C., Maniotes, L., Caspari, A. (2007). The Theory and Research Basis for Guided Inquiry Guided Inquiry : Learning in the 21st Century (pp. 13-28). Connecticut: Libraries Unlimited.

Kuiper, E., Volman, M., Terwel, J. (2005). The Web as an Information Resource in K-12 Education: Strategies for Supporting Students in Searching and Processing Information. Review of Educational Research, 75(3), 285-328.

Lamb, A., Johnson, L., Smith, N. (1997). Wondering, Wiggling, and Weaving: A New Model for Project and Community Based Learning on the Web. Learning and Leading With Technology, 24(7), 6-13.

Lupton, M., Bruce, C. (2010). Chapter 1: Windows on Information Literacy Worlds Generic, Situated and Transformative Perspectives. In A. Lloyd, S. Talja & Charles Sturt University. Centre for Information Studies. (Eds.), Practising information literacy : bringing theories of learning, practice and information literacy together

(pp. 3-27). Wagga Wagga, N.S.W.: Centre for Information Studies.

NYC. (2012). New York City School Library System Information Fluency Continuum.  New York: New York City School Library System Retrieved from http://schools.nyc.gov/NR/rdonlyres/27A1E84E-65EB-4A54-80DF-51E28D34BF4F/0/InformationFluencyContinuum.pdf.

Pickering-Thomas, N. (2004). Chapter 3:From Library Skills to Information Skills Instruction: Kuhlthau’s Information Search Process (ISP) Information literacy and information skills instruction Applying research to practice in the school library media centre (pp. 27-43). Westport: Libraries Unlimited.

QCAR. (2007). Studies of Society and Environment Essential Learnings by the end of Year 7  Retrieved 23/09/2012, from http://www.qsa.qld.edu.au/downloads/early_middle/qcar_el_sose_yr7.pdf

Robins, J., Fagan, P., King, C. Ingram, K., Pierce, L. (2005). Beyond the Bird Unit. Teacher Librarian, 33(2), 8.

Schultz, J. (2007). The Future of SOSE? Integrative Inquiry is the answer. The Social Educator(December), 11-16.

Wilhelm, J. (2007). Asking the Guiding Question : Reframing the Existing Curriculum into Inquiry Units Engaging readers and writers with inquiry (pp. 41-71). New York: Scholastic.

This entry was posted in Analysis, my ISP. Bookmark the permalink.

2 Responses to Data Analysis

  1. bertstlstudy says:

    I have thoroughly enjoyed reading your ISP. I thought your use of the survey was an excellent addition as you were not the class teacher and this enabled you to gain more insight into the task. This was an approach that would have been beneficial for my project. Your findings were well explained and your use of graphs and figures to support your conclusions was effective. Including diagrams of the relevant models discussed was also well considered, especially for those of us who are visual learners. I enjoyed reading the comments by the students especially regarding their frustration with blocked sites and their frustration at not being able to interpret the large volumes of information they located. These are universal views shared across all year levels.

  2. jeanetteki says:

    Yes, it surprising what Education Queensland blocks. I have ‘volunteered’ to be involved in doing something about that to help our ICT co-ordinator. I am looking forward to learning another skill. I’m not sure what I’ll be doing yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s