Traditional Assessment vs Authentic Assessment

Based on an article by John Mueller.

Authentic Assessment is defined as

"a form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills"

Traditional Assessment (TA) includes things such as multiple-choice tests, true-false, matching and so forth. The idea behind this is that students must possess a certain body of knowledge and skills, courses must teach this body of knowledge and skills, students must then be tested to see if the course is successful. So, the curriculum drives assessment - the body of knowledge comes first, that becomes the curriculum, and the tests see if students acquired the curriculum.

Authentic Assessment (AA) is built on the assumption that students should be able to perform tasks in the real world, courses must help students become proficient in those tasks. So, to determine the success of the course students perform meaningful tasks that replicate real world challenges. In this case assessment drives the curriculum - first determine what the tasks are that need to be performed, then develop a curriculum that enables students to perform the task well including the acquisition of essential skills and knowledge.

How does this work in subjects that are not practical, or do not require performance? For example how do we use AA in History, or Sociology for example? We ask students to perform tasks that replicate the challenges faced by people doing history, or conducting social research. In sociology the rubric may make reference to, for example, Sociological knowledge, Sociological thinking, and Sociological research skills.

It seems that TA is more useful for formative assessment, while AA is used for summative assessment. Students should be able to perform well in both types of test - TA provides a good complement to authentic assessment.

What are the attributes of TA and AA?

Traditional --------------------------------------------- Authentic

Selecting a Response ------------------------------------Performing a Task

Contrived -------------------------------------------------Real-life

Recall/Recognition ------------------------------- ------Construction/Application

Teacher-structured --------------------------------------Student-structured

Indirect Evidence ----------------------------------------Direct Evidence

AA will use verbs that are towards the top of Bloom's taxonomy - students will be asked to 'analyse', 'synthesise' and apply their learning. AA allows students more choice in what to focus on and what to present as evidence of their learning. TA is more prescriptive. There are often multiple routes to a good answer.

I'm not sure I like the term 'Authentic Assessment'. It suggests that TA is not authentic, which is wrong. TA has its uses in formative assessment. I'd prefer to use one of the alternative titles suggested at the bottom of the article - Alternative Assessment, or Performance Assessment.

Writing Effective Online Discussion Questions

This is based on the CREST+ model outlined by Lynn Akin and Diane Neal.

The ability to participate in group tasks is an important variable in the success of an online course. It encourages a sense of community and is more likely to improve engagement. Discussion forums provide a good way to encourage participation, but it requires skill in asking the right sort of questions. Akin and Neal argue that the CREST+ model provides a framework for creating effective questions which lead to greater participation and a higher level processing of the course material.

The CREST+ model looks at the Cognitive nature of question, the Reading basis, the Experiential possibilities and the Style and Type of question.

Cognitive Nature

There are a range of learning theories and models on which to base questions, such as Constructivism, Androgogy, Bloom's taxonomy. With a a Constructivist approach the students builds meaning based upon the course content. Questions can be structured to reflect increasing complexity. Gilly Salmon's Five Stage model of online learning uses this approach, increasing the student's interactivity and collaboration via carefully constructed questions at the different stages to facilitate the process. Knowles's Androgogy looks at how adults learn and proposes that they want to know why they are learning, need self-direction and want to be responsible for their own decisions, and they bring their life experiences to the course with them. Questions should be constructed that address these needs and help them to learn what will help them in their lives. Bloom's taxonomy, updated by Anderson, ranks enquiry types into a hierarchy. Each level builds upon the other and the student moves to complex understanding and knowledge. The types of question that could be asked would be based on the different levels and where the student was at on the hierarchy. The hierarchy is shown below, with its updated version.



Each level has a set of terms that can be used to build questions, which are available from a wide range of sources online.

In summary the first step in building questions is for the tutor to decide the best type of question based upon the cognitive needs of the students and the desired learning outcomes. The aim is to encourage participation and engagement from the outset. Higher participation and engagement leads to increased cognitive presence, which enables students to construct meaning through sustained communication, and to engage in critical reflective thinking.

Reading

Many courses will have a text book which is a shared resource for the students. Initial questioning can be based upon the shared textbook. It is important to scaffold the questions so that student, online at least, can arrive at more complex understanding together. So for example the forums would be separated, first would be one which concentrates on more basic understanding, before moving on to another forum which requires more complex thinking and critical reflection. Students can learn from each other about how they came to their conclusions, and can learn why others might not be arriving at the same answer.

Questions can also be based on a wider reading of relevant literature. Students would be instructed to find alternative viewpoints and arguments, to share their findings, and resources and citations. It also encourages participation and collaboration, and engages students in finding our about current ideas and research in their field of study.

You should also try to incorporate questions that do not rely on a text. Use videos or podcasts and sound recordings, graphics and images, webquest, scenarios provided by the tutor.

Experiential Element

This is based on Knowles's Androgogy, and constructivist views. Adult students bring a lifetime of experience with them (well, all students do, naturally, but this refers to more mature and varied experience that is often not there in younger students). The tutor should tap into this by providing discussion forums which are based upon the experiences of students and where they can share those experiences, and ask each other questions. They will create their own meanings based upon their prior experiences and peer generated questions can help to build new knowledge. It also increases the sense of community, and builds the students' social presence.

Style and Type of Question

In this case the 'style' of question refers to the students answering questions in pairs or groups. Then changing pairs or groups to discuss the question further. One advantage of this is that it reduces the number of posts in a forum. It also involves collaboration which again will enhance the feeling of community.

Different types of question could include: Metacognitive questions, in which students question their own knowledge, make connections between former and current problems, and reflect on the process of solving problems. Follow-up questions in which students consider different perspectives, provide clarification of thoughts, identify outcomes and answer the 'so what' question within the discussion. Student-created questions can provide thought provoking questions, and puts the student in charge of their own learning. Evaluation and Reflection questions allow students to reflect on the course so far, or any section of the course. For example the 'one-minute' assessment in which students write something they learned form the session and one thing they struggled with. This can then form the basis of a discussion. The tutor may want to allow anonymous contributions.

Finally

The discussion forum should be structured. Students need to know when the discussion is open and when it closes. They should know etiquette and protocols expected in an online discussion.  In summary, the tutor should decide the cognitive value of the question, then whether it should be literature based or not. Once this is established, decide whether it should be an experience-based question, then design the style and type of the question, before deciding the parameters for the structure of the question within the discussion forum.

photo credit: Horia Varlan via photopin cc

Lecture Capture

There are moves here at Leicester Uni to use lecture capture. A blog post by Mark Smithers suggests why this might be a bad idea. First, a couple of plus points for lecture capture:
The capture can be broken down into smaller chunks, so that it's not just simply a re-run of the lecture. It can also have subtitles added, and questions can be provided to engage the student in some active participation rather than simply passively watching.

It is useful for recording guest lecturers and visiting subject matter experts.

So, on to the bad points made by Smithers in his blog. Lecture capture perpetuates a passive and outdated mode of teaching. It is using 21st century technology to present 1000 year old pedagogy.

Lectures are a certain length often to suit the timetabling requirements of a particular building, rather than for any pedagogical reason - is there any need for example for lectures to be 1 or 2 hours long? Furthermore no meaningful learning can occur in a lecture.

What's the alternative? Use video technology to record short desktop pieces that are about 10 minutes long, and which develop a particular point. Or any sort of content that gets across information and ideas efficiently. This fits with the attention span of students, and enables them to study in their own time. Research at Bath University, where they have used lecture capture, suggests that the students spend around 10 minutes looking at the capture. This suggests they are skimming for particular content, and it ties in with evidence for people's attention span when learning. However, Bath uses the Panopto software which has good searching and note taking facilities.

Overall - I suppose it's like any technology in education, it can be used badly, and it can be used well. Perhaps money could be better spent on staff development that encourages different methods of delivery, and more engaging ways to deliver lectures.

photo credit: I, Timmy via photopin cc