SPANISH ORAL LANGUAGE ASSESSMENT

SPANISH ORAL LANGUAGE ASSESSMENT
Jill Jegerski | Department of Modern Languages | April 8, 2011
INTRODUCTION
Basic Spanish program at CSI
• Three-course Gen. Ed. sequence: SPN 113, 114, 213
• Approximately 800 students total per semester
• Most sections are taught by part-time faculty
Jegerski
Language Assessment
2
PRIOR ASSESSMENT METHOD: FORUMS
Previously, we used threaded discussion boards
• Students recorded answers using a separate
program and posted them in the forum
• Instructors downloaded individual answer files
• Mean score for SPN 114 assessments was 86.1%
• Higher than mean course grades for SPN 114
• We know oral proficiency is the last of the four
language skills to develop
• We questioned the validity of the assessment
method
Jegerski
Language Assessment
3
PRIOR ASSESSMENT METHOD: FORUMS
Validity may have been undermined by several factors:
• No mechanism for timing students’ oral responses
• Students could playback and record over their
answers
• Test items were the same and appeared in the same
order for all students
• Instructors used an external grading sheet
Jegerski
Language Assessment
4
PRIOR ASSESSMENT METHOD: FORUMS
Other issues relevant to outcomes assessment:
• No way to compile or analyze scores
• No way to examine individual test items
• No way to monitor the use of rubrics by
instructors
→New testing software acquired Fall 2010: OWL
Jegerski
Language Assessment
5
NEW ASSESSMENT METHOD: OWL
OWL testing software: Validity
• Professors specify time limits for oral
responses
• One attempt only for each item
• Test items are randomly selected from banks
of questions
• Grading module (including our rubrics) is
built in to the program
Jegerski
Language Assessment
6
OWL: STUDENT VIEW
Jegerski
Language Assessment
7
OWL: STUDENT VIEW
VIDEO SCRIPT:
Hola ¿Cómo está? Me llamo Teresa. Soy de Oaxaca, México. Mi estación preferida del año es el otoño. Durante los meses
de otoño no hace mucho calor, hace fresco. Algunas veces llueve, pero no siempre. También me gusta el otoño porque en
esta estación celebramos una de las fiestas más tradicionales de mi país: el día de muertos. El día de muertos se celebra el
uno de noviembre. En esta fiesta celebramos a nuestros parientes muertos. Hay muchas decoraciones típicas de esta
celebración, como el papel picado… o las calacas… y más importante: los altares. Los altares se montan en honor de los
parientes muertos. En ellos ponemos decoraciones y también ofrendas de comida. Normalmente ponemos la comida
favorita de nuestros parientes y también la comida típica de la celebración… Algunas de las comidas típicas de esta fiesta
son: el pan de muerto… Y las calaveritas de azúcar. Si algún día viene usted a México durante el día de muertos, pida estas
comidas típicas, cómalas y visite los altares. Estoy deseando conocerla. Escriba pronto. Hasta luego.
Jegerski
Language Assessment
8
OWL: INSTRUCTOR VIEW
Jegerski
Language Assessment
9
OWL: INSTRUCTOR VIEW
Jegerski
Language Assessment
10
NEW ASSESSMENT METHOD: OWL
Owl testing software: Outcomes assessment
• Test scores are easily compiled for
analysis
• Scores for individual test items are also
available
• The use of rubrics is easily monitored
because the rubric is part of the score
Jegerski
Language Assessment
11
NEW ASSESSMENT METHOD: OWL
Owl testing software: Results from pilot
• Mean scores were 74.2% (vs. 86.1%)
Jegerski
Language Assessment
12
FUTURE DIRECTIONS
1. Goal: Achieve a minimum level of oral proficiency
(ACTFL Intermediate Low)
2. Objectives: Specific language skills (e.g., talk about
self, family, and daily activities; express personal
preferences; ask questions)
3. Measurement: Assessment of oral proficiency
4. Evaluation: Compile and analyze results using OWL
5. Modifications: More emphasis on speaking in the
curriculum, possibly modify expectations for oral
proficiency
Jegerski
Language Assessment
13
THANK YOU!
Jill Jegerski | Department of Modern Languages