Evaluating the impact of embedded Insessional academic language and literacies provision

In this short piece I aim to provide the context and thinking behind the development of an ongoing collaborative scholarship project that the Insessional teaching team in Leeds are working on.  

Here is the summary of the project, as outlined for ethical approval: 

English for Academic Purposes (EAP) teaching is now embedded across a range of Schools in the University at PGT level, with EAP teachers working alongside subject academics to support students for whom English is an additional language to develop their understanding of disciplinary academic discourse and literacy requirements. 

This research aims to evaluate and measure the impact of this intervention, with different researchers in the research group approaching the question from a variety of angles in order to build a rich picture that takes into account the range of perspectives involved in this aspect of taught student education. 

This research is a longitudinal study that will make use of a highly mixed methods approach to data collection and analysis. Whilst there is one overarching research question, the project will be composed of a range of more localised investigations, carried out by those with insider knowledge. The sum of these different projects will enable a rich description of how local enactments and differing disciplinary concerns and understandings around how knowledge is created and communicated interact with the human participants. The research will consider what, if any, generalisations can be made across a whole University or whether individuals and their contexts are key. 

Insessional English for Academic Purposes (EAP) is, on the whole, a hidden practice. This is true on several levels. Whilst those of us who work in the field are aware that Insessional teaching happens in most institutions, it is not easy to see how, where, why or in what format this takes place. Whilst we can get a sense of the different kinds of programme structures, syllabus and approaches taken on presessional courses by looking at the websites of specific EAP centres (see, for example, the one for Leeds), Insessional programming remains hidden from view in part because it is not something that students apply to directly. 

Many presessional programmes, although not strictly credit bearing, do now go through the formal scrutiny and quality evaluations of external examining; a significant number of institutions, at least in the UK, are also accredited by BALEAP – the global forum for EAP professionals. Again, this enables a level of transparency and public sharing of practices. The EAP community generally feels it has a good sense of what makes a ‘good’ presessional programme – again this is clearly defined in BALEAP accreditation documentation. 

Insessional provision, however, is not so clearly defined or visible to external scrutiny. This may, in part, be because some teaching that could be defined as Insessional is so embedded within a particular programme that it is not labelled as ‘insessional’ but is subject to external examiner scrutiny as part of a credit bearing module. It then becomes hidden to EAP practitioners wishing to learn from other contexts. In other cases, Insessional teaching is not credit bearing, is not part of a formally approved programme or module and so is not visible through course searches or websites. We may be able to find short texts explaining to students that provision is available; students are then given further information on arrival, possibly by proactively contacting the EAP centre themselves.  

Furthermore, Insessional teaching is often not called Insessional teaching. The term itself is relatively meaningless to those who are not long-term UK-based EAP practitioners; outside the UK other language is used to describe similar work (e.g. writing across the curriculum); to staff and students in other disciplines, Insessional does not really clearly define the learning that would take place. For further exploration of the difficulty of choosing a name to describe our work, please read Michelle Evans’ blog. 

One result of the occluded nature of Insessional work is that it can frequently feel that we are practising in isolation and without a clear or strong understanding of what good practice in Insessional work looks like. BALEAP does not (yet?) accredit Insessional provision and, often working outside a universities QA structure, it remains unscrutinised against professional standards. There is also very little published work that explores Insessional practice. A notable exception is the open access edited volume available on the BALEAP website. Expanding searches beyond the term ‘insessional’ provides literature in similar areas from Australia and the United States. As a team we are building a reference list to share with others interested in developing an understanding of the variation and types of practice that might come under the umbrella of ‘Insessional’. 

The second level of Insessional invisibility is around evaluation of the impact it does or does not have. It is this lacuna that, as a team of practitioners, we are currently attempting to consider within our own contexts. 

Insessional teaching, by its very nature, involves working alongside and in conjunction with disciplinary teaching, working on, with and through academic language which is inextricable from the disciplinary learning that takes place. If it is working well, it should be almost impossible to separate the learning that takes place in an Insessional class from the learning that takes place across the rest of the curriculum. It should support, develop and enhance in tandem with all other knowledge building. It is, therefore, incredibly difficult to find ways to extract and evaluate any specific impact that Insessional EAP teaching may or may not have on the learning that takes place. 

The EAP community, via BALEAP’s discussion list, often raises the question of tracking students (from presessional and beyond). In fact, the development of a better system of tracking was one of the key areas for improvement suggested in our last BALEAP Accreditation visit in 2019. A good system, that tracks and measures (?), evaluates (?), demonstrates (?) progress or impact of EAP teaching seems to remain somewhat of a holy grail. If any centres do have an excellent system for doing this in place, we would love to hear from you. Responses to requests for suggestions on the BALEAP discussion list seem to be quite scarce! Some of the difficulty in establishing effective tracking mechanisms lies in the systems used in our institutions. As students tend not to ‘belong’ to EAP centres beyond a presessional course, we do not have direct or easy access to their records.  

However, it also seems clear that we (EAP practitioners) are not always necessarily clear as to what we are supposed to be looking for. This is not a question that is specific to EAP but is very pertinent to our Insessional context. On this website and through the scholarship that we produce through our collaborative project we hope to share some of our thinking and developing understanding around what questions we should be asking ourselves when we try to consider ‘impact’.  

We have tussled with what we mean by ‘impact’, whether and how it could be measured (quantitively as well as qualitatively) and how this is different to evaluation. Which areas of student work do we look at for any impact we may have? Can we see impact on staff? Where and how? What is the impact of disciplinary Insessional teaching on ourselves? On our other work, including presessional design? And how do we manage and address the multiple ethical questions that arise when trying to consider the impact of our own work on others? Working online and through lockdown has, inevitably, delayed much of our scholarship work and thrown additional questions and uncertainties into an already chaotic and confusing area of investigation. 

This area of our website will provide an overview of the various investigations we are undertaking as part of this overarching scholarship project. We also aim to share some of the findings and some of the struggles involved in our attempts to demonstrate impact (including a general dislike of the ways that impact currently seems to be understood in UK universities!) 

This project is very much a collaborative, iterative and shared endeavour. We hope that by sharing our progress on this site we will also open the discussions to others (and not only EAP practitioners). Please do comment or send us your thoughts and suggestions.