WebQuest update

Among the components of the 2011 & 2012 offers of the course that I thought I wanted to retain for 2013, albeit with some changes, the WebQuest seemed to be a clear keeper. It models the use of a tried and tested method of making ICT integral to student-centred learning and teaching in the course. Based on feedback from students in 2012 it appears to have worked well as a way of exposing some issues that arise around technologies that trigger a variety of responses based on differing values, and it raised awareness about a technological issue that is likely to affect the lives of many students in the course and about which many of them, by their own account, had very limited knowledge before working through the WebQuest, which addressed several of the course objectives as revised for the 2013 offer: 

  • Demonstrate understanding of technology as a human activity
  • Demonstrate knowledge and skills relevant to the content of technology education
  • Demonstrate understanding of how ICT can be integrated to enhance learning in technology education
  • Demonstrate knowledge of course content using appropriate modes and conventions of expression with attention to spelling, grammar, punctuation and bibliographic referencing.

In marking student submissions in 2012 it became apparent that some of the instructions about the process and what was required for submission might have been clearer. Some students struggled to address the marking criteria within the 200 word limit for each of the 3 response segments. The responses had been deliberately specified as short in order to encourage students to focus on the key ideas and to limit the volume of reading and writing necessary for them to engage with the discussion forums. Short responses were also expected to assist the markers. 

Thus the work required to tidy up the WebQuest was relatively simple – clarify the instructions to assist students through the process, and simplify and clarify the marking criteria to improve the reliability of the activity as a method of assessing learning against the objectives. Checking and updating of links to online resources provided in the WebQuest was also required but that part of the renovation would be best done just before the start of the 2013 offer so that resources would be as up to date as possible. I also wanted to ensure that the WebQuest would display in a readable fashion on any device from a desktop computer to a smartphone so some work on the HTML and CSS might be required.

There had been some comments about students suggesting that verification by screenshot ought not be necessary for students attending class on campus and participating in discussion there. Although there is some validity in that comment, I was also aware that many students had commented in their assessment submissions that this was the first time they had learned how to grab screenshots and they thought it was a useful skill. On that basis I decided to retain that requirement as part of the effort to build and reinforce useful ICT skills.

My memories of where students appeared to have difficulty with the process and with the marking criteria were reasonably fresh so it was a relatively simple matter to work through the sections of the WebQuest making adjustments to address the known issues. Whether those will cause different issues will be revealed when the activity runs again in 2013. In the meantime the WebQuest content is tidied up, saving last minute checks to links just before semester begins. 

The structure of the WebQuest is based on that promoted by Bernie Dodge on his original WebQuest site. I built the pages in DreamWeaver with a simple stylesheet to handle the presentation. When I tweaked the site in 2012 I was thinking about mobile and included some media queries to deal with different screen sites. Changing page width in browsers on my iMac produced the expected adjustments to presentation, including rearrangement of navigation elements, but when I checked the updated site on my iPhone it preferred to present a ‘zoomed’ version with the full-size site in miniature. That was workable with ‘pinching’ but not what I wanted so I dug around my personal site and borrowed the ‘viewport’ code I’ve been using there. A few more tweaks to the media queries got that working well enough for now – more changes may follow when I have time and more knowledge – and opportunity to test on more devices. The tables that I used for the marking guide had multiple columns that did not squeeze down well for display in readable text on a small screen. Rather than try to replicate a previous effort to reformat a wide table of research supervision, I opted to rework the table into a vertical format.

The revised CSG WebQuest is now pretty much ready for use in 2013. It will be necessary to check and update links to resources and integrate it into the Moodle space for the course before the semester begins but, unless I have some brilliant idea for change, it is otherwise done.

What to retain?

One of the first decisions to be made in the process of course redevelopment is what to retain from what has gone before. In other words, I need to identify what should be kept because it worked reasonably well and what adjustments might be made to improve it for the future.

EDP4130 has a history that should not be ignored in planning for redevelopment. The first offer in 2011 borrowed much of its content and activities from EDU1471, which had been offered some years previously across 3 campuses but not online. The online offer required for EDP4130 was a significant difference that influenced planning and implementation of the LMS (Moodle) space. Based on that experience some further changes were made for the 2012 offer.

Thinking about how various parts of the course worked (or didn’t) can be informed by my own reflections on the experience, comments from colleagues who worked in one or other offer, records of activity in the LMS, and data from student evaluations of the course offers. Some of those sources are loose but should suffice for some guidance about what to change.

Because I developed EDU1471 immediately after spending a year at Purdue I designed the assessment to be distributed through the semester and linked closely with tutorial activities. That had multiple benefits with an on-campus course. Students saw relevance in attending classes that bore directly on assessment, much of the work was actually completed in class and small pieces of work submitted one week could be easily marked in time for return in class next week. That kept students attending and ensured that they knew how they stood as the semester progressed. The major assessment piece was a class project involving collaborative development of curriculum materials that were contributed to a shared resource pool, accompanied by an individual report on learning through the process. Students saw the assessment as relevant and manageable. From my perspective designing the key assessment items first was a useful way of specifiying expected outcomes and I was then able to design learning activities to support their achievement.

In developing EDP4130 for 2011 I retained as much as possible of the tutorial activities and related assessment from EDU1471. The WebQuest about infant formula that I had developed for EDU1471 seemed dated so I constructed a new one around Coal Seam Gas but did not include it in the assessment for 2011. I did retain an activity in which students developed design briefs, exchanged them for response, and then reported on that process. I was able to manage that for online students using a database in Moodle but handled the on-campus classes in the classical way. I retained a resource review activity and assessment from EDU1471 but worked it in Moodle using a database and a peer review assessment module. I added a quiz early in the semester to drive engagement with the curriculum documents and some relevant reading. The whole class curriculum project and report was retained essentially as it was configured for EDU1471 with online students working in forums with a database and wikis if desired.

The checklist approach in the peer review assessment produced high marks. That coupled with good marks on the other assessment items produced a distribution with more high grades than the faculty and university authorities liked so some change was indicated for 2012. Other issues arose around management of some activities, especially the large group curriculum development, for the online students. Again, some change was indicated for 2012. Overall students had a very positive response to the assessment pattern with evaluation comments running 12:1 in favour of the pattern of several small assessment pieces across the semester. Students also responded positively to what they saw as the relevance of the assessment activities to their learning and future work as teachers.

For 2012 I retained the quiz as a simple means of encouraging students to engage with key content. They recorded an average score of 12.5 out of 15 (83%) which some might regard as high but it caused me little concern if it meant that students became familiar with important course material. I also retained the design brief activity but, in deference to my understanding of the USQ push toward Digital First, I designed it to work entirely online for all students using a pair of databases in Moodle. I dropped the peer review activity around resources because it had inflated marks and tweaked the WebQuest about Coal Seam Gas to replace it as an assessment item. Student engagement with the optional because not for assessmentWebQuest in 2011 had been minimal but making it assessable was expected to encourage engagement with the important ideas about the values dimensions of technology. Because the whole class curriculum project had been awkward to manage online I made that item an individual curriculum development project and report with the option for students to work in small groups of up to 4 members. However, the requirement to contribute the materials to a shared pool of resources (collated through a Moodle database) remained. To retain the benefits of collaboration for all students I added requirements to demonstrate the effect of a Personal Learning Network (PLN) and of sharing work in progress through a Virtual Learning Design Studio (VLDS) hosted in the Mahara eportfolio system.

Student responses to the course in 2012 were mixed. Although the pattern of assessment was very similar to that in 2011 the changes resulted in a wider spread of grades and more disaffection among students, which seems consistent with students seldom complaining when they receive good grades. Some on-campus students expressed unhappiness about needing to complete certain tasks online, although others took the opportunity to reduce class attendance. Some students appeared to have difficulty with interpreting assessment requirements and criteria although others managed very well. Comments on student evaluations pointed to the need to ensure that criteria are clearer and that assessment details are easily found. Some careful thinking about the design of the Moodle space for the course will be needed. The final task was seen as large, though in reality it was no larger for a student than in previous years, and complex. Comments about needing to access multiple pages for information possibly related to information about the PLN and VLDS components and, if those are retained for 2013, care will be needed with designing clear pathways through the material. Comments in the actual assessment work submitted by students seemed to suggest that they found value in all parts of the activities, including the PLN and VLDS, but perhaps those comments were calculated to encourage the marker to reward them.

Although much of the course was designed around the assessment and related learning activities (directed reading assessed by the quiz, WebQuest over 3 to 4 weeks with a 600 word report, design brief exchange over 2 weeks with a 600 word report, curriculum materials and 1500 word report developed over 5 weeks) there were other resources provided to students. They included brief notes about the course content, recordings of presentations (in multiple formats – M4V [with video] and MP3 ) accompanied by PowerPoint slides and handouts, weekly guides to activities and links to other resources. How much use was made of these is difficult to determine because not all are trackable in Moodle and where material is provided in multiple formats it is difficult to aggregate records of access. However, examination of available activity reports suggests that some students access most of the material but many will have been selective, especially toward the end of semester when they were focused on completing assessment.

So, what have I learned and what should I retain for the 2013 offer?

Based on the student evaluation data there seems to be support for a distributed assessment pattern to spread the load, provide formative feedback through the semester, and avoid ‘high stakes’ items like the final piece worh 55% in 2011 and 2012. Students appreciate assessment that they see as relevant and for which the criteria are clear and instructions easy to follow. The quiz (in some form), WebQuest and design brief activities should be retained though some adjustments will be in order. The large final assignment seems not much favoured by students and the advent of systemic curriculum design initiatives like C2C and online sources of teaching materials like Teachers Pay Teachers suggests that in future teachers are much more likely to find themselves adopting and adapting existing or found materials rather than engaging in ab initio curriculum development. The assessment work in EDP4130 should probably recognise that shift and focus on the work of locating, curating, analysing and adapting curriculum materials rather than development from the blank page – if that ever really happened. There did seem to be value in the PLN and VLDS activity but the VLDS could be simplified by having students work in smaller, more manageable groups and both would benefit from being introduced from the start of semester as part of the professional approach to working rather than later as an assessment requirement.

There is probably a continuing need for some recorded material but front loading that early in the semester may encourage its use. By end of semester students are focused on completing assessment in all their courses rather than accessing new material that may not be directly related to assessment. Multiple formats that support access using mobile devices received some support in the student evaluations and should be retained. There were also requests for more easily printed material (PDF) and there might be value in looking at means to produce notes and other text material in multiple formats to support web access, printing (PDF) and eBook readers (ePub and mobi).

That seems like plenty to think about for now.

Redeveloping EDP4130 Technology Curriculum and Pedagogy

This semester I have some time set aside to work on redeveloping the course that I’ve been offering to 4th year Primary specialisation students in the BEd since 2011. The course, EDP4130 Technology Curriculum and Pedagogy, is intended to prepare students to teach the Technology Key Learning Area. It is offered on 3 campuses (2 in 2012 because of low numbers on one campus) and also entirely online.

For now in Queensland the Technology curriculum area is defined by Queensland Studies Authority documents based on the 2003 syllabus but the Australian Curriculum Assessment and Reporting Authority is working toward development of Australian Curriculum documents beginning with 2012 consultation around a draft shaping document.

I have good intentions of recording the development process here but how far that goes is yet to be seen. I had intended beginning this record last week after meeting with the Learning Innovation Teaching Enhancement (LITE) team to discuss my plans but was not well and also busy with other stuff. The LITE Team seemed to think that my existing course was in good shape. The notes they provided following the meeting included these comments:

This course is exemplary and references most of the AITSL and QCT standards relating to ICTs as well as the Good Practice Guidelines.  Learning activities and assessment tasks develop appropriate concepts and metacognitive skills.

Personally I might not rate it so highly. There are some parts of the course that appear to work well and some of those were borrowed from a previous course (EDU1471 Technology Education) that was offered from 2002 until 2006 to prepare students for the same curriculum area. Other parts of the course did not work as well as I had hoped in 2012 and some students expressed dissatisfaction with aspects of the course in the evaluations. Some of those were changes made from 2011 to 2012 to address issues with the distribution of grades and other matters from the 2011 offer. There is work to be done to improve on what was offered in 2011 and 2012.

In addition to that, the ACARA shaping document includes digital technologies as about half of the content alongside the more traditional Technology KLA ideas around design. The digital technologies element would introduce basic ideas about computer science, including the rudiments of programming, for which our graduates are not being prepared at present. It would be prudent for any redevelopment of the course to address those aspects and I’ve already made changes to course objectives to accommodate that.

I’m also interested in what can be done to move the pedagogy of the course forward. I’ve been providing recorded lectures but I’m not sure how much use those attract from students in a course with no final examination. Perhaps it’s time for a change in that department. It’s certainly time to think about more flexible access to course materials and activities with increasing numbers of students having access to mobile devices such as smartphones and tablets. I’m also interested in what might be done with gamification, specifically through the use of badges or similar approaches as part of the assessment regime.

That provides me with plenty to think about as I wrestle with redevelopment. One of the first things I’ll need to do is consider what has worked well and should be retained, perhaps with some adaptation, and what has not worked so well and should be dropped or subjected to major renovation. Beyond that I’ll need to think about what new developments might be possible in the time I have to work on this.