Tidying the assessment for EDP4130

With just a few weeks until the start of semester I’m working on my course materials and website for EDP4130 Technology Curriculum and Pedagogy. Consistent with my belief that students are most often most interested in what they need to do to successfully complete the course I’ve been giving a lot of attention to the design of the assessment. My approach might be described as some variation of assessment for learning in which I attempt to design assessment that will encourage students to engage with what I think is the important learning in the course.

Ideally I would design assessment around a series of mostly small tasks that would spread the workload for students and for markers, avoiding the tsunami of marking that usually mounts at mid-semester and the end. That approach would allow me to provide meaningful feedback through the semester so that students do not suffer nasty surprises on high stakes items and can take corrective action, if necessary, in time to make a worthwhile difference to their final grades. I’ve worked in courses where the assessment consisted of (almost) weekly tasks linked to learning activities that occur in class. That can often be arranged so that students complete much of the necessary work in class leaving relatively little demand on their time out of class. It usually also means that they have feedback within a week so that they can adjust their approach.

Unfortunately that approach to assessment is not acceptable right here and now so I am limited to 3 pieces of assessment though those may include multiple components. That allows linking of assessment to the learning activities as  I might want to but imposes restrictions on due dates so that feedback is less frequent than might be desirable to shape progress.

Given those constraints, in addition to the professional experience that must be satisfactorily completed for students to pass the course, the 3 assessment items are as follows:

Item Content

Short report about planning for the curation task including establishing a personal learning network
Simple online quiz about curriculum documents, readings, and introduction to Scratch

2 Short report about the CSG WebQuest
3 Short report about the digital technologies activities
Short report about a learning activity with design briefs
Curated collection of resources and associated report
Simple online quiz over the course 

I’m now in the process of working through those components to ensure that things are complete, adequately explained, and presented so that they will be accessible and clear to students.

I posted yesterday about finalising the digital technologies learning activities and now have both those and the related assessment requirements in place. Rather than directly assess the activities, though the quizzes may include some specific items about Scratch programming, I have opted to require students to write a reflective blog post for each of the four phases and share that with their personal learning network. Their reflections are to briefly describe the activity, make links to the Australian Curriculum documents, explain what they learned, and comment on potential application in primary classrooms. The actual assessable piece will be a report that summarises the content of those posts. I hope the result will be encouragement for students to think about what they are learning through the activities, both at the time they are working with them and toward the end of semester, and a piece of work that is manageable for all involved in the submission and marking process. 

Some months ago I posted about adjustments to the WebQuest about Coal Seam Gas that I developed for a previous offer. It had worked fairly well but feedback from students suggested that some changes might be in order.  Those changes were made and the CSG WebQuest is now tidied up with links to resources checked and should be ready for use.

I’ve used variations of the design brief activity since I first developed a technology education course in 2002. The first iterations were in classes on campus where students worked in groups one week to develop a design brief that could be managed by a group of their peers within a standard class period. The following week briefs were exchanged around groups and responses generated. After some debriefing discussion students wrote a short report for assessment. Since 2011, when EDP4130 was offered online as well as on campus, I have used the Moodle database to manage the exchange of briefs among individuals or small groups. Assessment is still based on a report about the activity and what was learned. For 2013 I’ve adjusted the report requirements and associated marking guide on the basis of feedback from 2012. Those elements are ready but I have yet to revise the materials that guide the activity. A significant part of the work for that may be to revise the instructions and illustrations for the database following our move from Moodle 1.9 to Moodle 2.x. 

I posted my early thinking about the content curation task some time ago. The basic requirements for the planning to be reported on in Task 1 and the collection and report to appear in the assignment are written up. What remains to be done there is to provide some introductory material about curation and PLNs. I’ve been collecting resources for that in my Diigo space but will need to find time to extract and present the key points.

Preparation for the quizzes is similarly incomplete. I have some material for the first quiz that I prepared in previous years. With a little updating and addition of some Scratch questions that will serve. The second quiz will need to have items developed as I put together the balance of the semester resources.

WebQuest update

Among the components of the 2011 & 2012 offers of the course that I thought I wanted to retain for 2013, albeit with some changes, the WebQuest seemed to be a clear keeper. It models the use of a tried and tested method of making ICT integral to student-centred learning and teaching in the course. Based on feedback from students in 2012 it appears to have worked well as a way of exposing some issues that arise around technologies that trigger a variety of responses based on differing values, and it raised awareness about a technological issue that is likely to affect the lives of many students in the course and about which many of them, by their own account, had very limited knowledge before working through the WebQuest, which addressed several of the course objectives as revised for the 2013 offer: 

  • Demonstrate understanding of technology as a human activity
  • Demonstrate knowledge and skills relevant to the content of technology education
  • Demonstrate understanding of how ICT can be integrated to enhance learning in technology education
  • Demonstrate knowledge of course content using appropriate modes and conventions of expression with attention to spelling, grammar, punctuation and bibliographic referencing.

In marking student submissions in 2012 it became apparent that some of the instructions about the process and what was required for submission might have been clearer. Some students struggled to address the marking criteria within the 200 word limit for each of the 3 response segments. The responses had been deliberately specified as short in order to encourage students to focus on the key ideas and to limit the volume of reading and writing necessary for them to engage with the discussion forums. Short responses were also expected to assist the markers. 

Thus the work required to tidy up the WebQuest was relatively simple – clarify the instructions to assist students through the process, and simplify and clarify the marking criteria to improve the reliability of the activity as a method of assessing learning against the objectives. Checking and updating of links to online resources provided in the WebQuest was also required but that part of the renovation would be best done just before the start of the 2013 offer so that resources would be as up to date as possible. I also wanted to ensure that the WebQuest would display in a readable fashion on any device from a desktop computer to a smartphone so some work on the HTML and CSS might be required.

There had been some comments about students suggesting that verification by screenshot ought not be necessary for students attending class on campus and participating in discussion there. Although there is some validity in that comment, I was also aware that many students had commented in their assessment submissions that this was the first time they had learned how to grab screenshots and they thought it was a useful skill. On that basis I decided to retain that requirement as part of the effort to build and reinforce useful ICT skills.

My memories of where students appeared to have difficulty with the process and with the marking criteria were reasonably fresh so it was a relatively simple matter to work through the sections of the WebQuest making adjustments to address the known issues. Whether those will cause different issues will be revealed when the activity runs again in 2013. In the meantime the WebQuest content is tidied up, saving last minute checks to links just before semester begins. 

The structure of the WebQuest is based on that promoted by Bernie Dodge on his original WebQuest site. I built the pages in DreamWeaver with a simple stylesheet to handle the presentation. When I tweaked the site in 2012 I was thinking about mobile and included some media queries to deal with different screen sites. Changing page width in browsers on my iMac produced the expected adjustments to presentation, including rearrangement of navigation elements, but when I checked the updated site on my iPhone it preferred to present a ‘zoomed’ version with the full-size site in miniature. That was workable with ‘pinching’ but not what I wanted so I dug around my personal site and borrowed the ‘viewport’ code I’ve been using there. A few more tweaks to the media queries got that working well enough for now – more changes may follow when I have time and more knowledge – and opportunity to test on more devices. The tables that I used for the marking guide had multiple columns that did not squeeze down well for display in readable text on a small screen. Rather than try to replicate a previous effort to reformat a wide table of research supervision, I opted to rework the table into a vertical format.

The revised CSG WebQuest is now pretty much ready for use in 2013. It will be necessary to check and update links to resources and integrate it into the Moodle space for the course before the semester begins but, unless I have some brilliant idea for change, it is otherwise done.