Assessment of upper secondary national tests in English varies considerably between teachers (e.g., Tengberg & Skar, 2016; Borger, 2021; Frisch, 2021). This is problematic as it reduces the validity of grades across teachers. The Swedish upper secondary course curriculum (Lp21) involves assessment criteria that focus on clarity and fluency, coherence, variation and adaptation to different contexts. These rubrics appear in grading criteria for the different skills (receptive, productive and interactive), with only small differences in phrasing distinguishing grades. For example, considering variation in written production in English 5, the criteria state ‘some variation’ for grades E and C and ‘varied’ for grade A, without specifying what types of variation are required at different levels. Assessing pupils’ texts is thus a matter of interpretation. Furthermore, assessment is a multifactorial process, where variation is just one of several criteria used to assess written production, which may explain some of the differences in grades reported by Tengberg and Skar (2016). In this talk we aim to focus on one area of assessment: variation in written production. We present an interview study of experienced teachers’ holistic and analytical assessment of an English 5 student text responding to a National Test writing assignment. We asked teachers to assess texts on individual rubrics and also provide overall grades and explain their reasoning behind the assessment. We aim to illustrate that the variation in grades seen by Tengberg and Skar in reading comprehension also applies to written production and argue for the need for clearer assessment criteria. 

This study introduces an approach to providing corrective feedback to L2 learners termed analogy-based corrective feedback that is motivated by analogical learning theories and syntactic alignment in dialogue. Learners are presented with a structurally similar synonymous version of their output where the erroneous form is corrected, and they must decode the analogy-based feedback to understand the correction. A quasi-experimental classroom-based study was conducted with upper secondary Swedish EFL learners (N = 49) to investigate the effectiveness of corrective feedback varying in mode (inductive exemplar-based or deductive rule-based) on English subject-verb agreement. Explicit correction, metalinguistic, and analogy-based corrective feedback, all explicitly providing evidence of error and including reformulation prompts, were assessed by timed and untimed grammaticality judgment and sentence completion tasks in a between-groups pretest, posttest, delayed posttest design with a control group. Results indicate significant delayed gains for all feedback types on the untimed grammaticality judgment task for ungrammatical items. No clear advantage was seen for rule-based or exemplar-based CF. Descriptive statistics indicate different trends over successive testing times, where analogy-based feedback often led to lowest performance on the immediate posttest but showed improvement on the delayed posttest, unlike the other two CF types.

 This paper investigates the effect of spatial semantic elaboration strategies on young and older participants' performance and preferences in a route drawing task where participants read route instructions and then drew the route on a map. In particular, an elaborated form of spatial perspective that communicated the route instruction in both egocentric and allocentric perspectives was compared against just the egocentric or allocentric perspective instructions individually.

Additionally, route instruction granularity was varied to compare elaborated hierarchical instructions which communicated goal and landmark information with flat instructions. The results of the experiment showed that older participants performed best with least confusion in the allocentric perspective with flat granularity, but that they benefited from semantic elaboration when less optimal spatial strategies were used, unlike young participants who performed best with the mixed (i.e., elaborated) perspective and flat granularity. The experiment showed that older participants actually suffered from what might be information overload when excess semantic elaboration was provided, as was likely the case with the use of the mixed perspective, which confused them considerably. In addition, hierarchical granularity was only beneficial in confusing spatial perspectives, while it detracted from older participants' performance in their optimal perspective, showing that there is a fine balance to be struck between beneficial semantic elaboration and information overload.

 This paper describes exploratory research into automatically describing geo-referenced information to blind

people. The goal is to produce texts giving an overview describing the spatial layout, and a central concern of such texts

is that they employ an appropriate linguistic reference frame which enables blind hearers to ground the information. We

hypothesise that (1) directly perceivable reference frames are easier to ground and also that (2) spatial descriptions

drawn from composite reference frame systems composed of more than one reference frame are easier to ground. An

experiment exploring text comprehension on a range of texts employing different reference frame systems is presented.

The main results indicate that the second hypothesis is supported. A prototype of a natural language generation system

which generates texts describing geo-referenced information from data is described.

In this paper we investigate the effect of level of understanding revealed by feedback in the form of clarification requests from a route follower on a route giver’s spatial perspective choice in their response in route instruction dialogues. In an experiment varying the level of understanding displayed by route follower clarification requests (the independent variable), route giver perspective switching in response to this feedback is investigated. Three levels of understanding displayed by feedback are investigated: (1) low-level clarification requests indicating that the instruction was not processed, (2) semantic-level clarification requests indicating that the spatial direction given in the instruction could not be resolved as the speaker of the clarification request could not interpret which perspective was intended, and (3) high-level feedback which indicates that the route giver’s instruction was understood but which note an obstacle to following the instruction. Results show that perspective choice, which is a conceptual feature of language use, is sensitive to perceived level of addressee understanding. We found that route givers consistently switch perspectives in responding to semantic-level clarification requests but not in response to low-level ones, and also that switching occurs more for high-level feedback than for low-level feedback. We address how dialogue systems can take advantage of these findings by modelling our results in an Information State model of dialogue, presenting update rules for response generation which account for our findings and also update rules which enable generation of the feedback themselves.

When conveying information about spatial situations and goals, speakers adapt flexibly to their addressee in order to reach the communicative goal efficiently and effortlessly. Our aim is to equip a dialogue system with the abilities required for such a natural, adaptive dialogue. In this paper we investigate the strategies people use to convey route information in relation to a map by presenting two parallel studies involving human–human and human–computer interaction. We compare the instructions given to a human interaction partner with those given to a dialogue system which reacts by basic verbal responses and dynamic visualization of the route in the map. The language produced by human route givers is analyzed with respect to a range of communicative as well as cognitively crucial features, particularly perspective choice and references to locations across levels of granularity. Results reveal that speakers produce systematically different instructions with respect to these features, depending on the nature of the interaction partner, human or dialogue system. Our further analysis of clarification and reference resolution strategies produced by human route followers provides insights into dialogue strategies that future systems should be equipped with.

We present Atlas.txt, a novel data-to-text natural language generation system which enables access to geo-referenced information like online census data. We first discuss initial findings from an accessibility study on geo-referenced data and outline needs requirements for visually-impaired users of such data. We then present work towards realising our data-to-text system and indicate how it aims to address this issue.

In this paper we describe content determination issues involved in the Atlas.txt project, which aims to automatically describe georeferenced information such as census data as text for the visually-impaired (VI). Texts communicating geo-referenced census information contain census data abstractions and their corresponding geographic references. Because visually impaired users find interpreting geographic references hard, we hypothesized that an introduction message about the underlying geography should help the users to interpret the geographic references easily. We performed user studies to design and evaluate the introduction message. An initial evaluation study with several sighted users and one partially sighted user showed that an introduction message is certainly preferred by most participants. Many of them used an introduction message themselves when they described maps textually. But the study also showed that the introduction message made no difference when the participants were asked to draw maps using the information in the textual descriptions.