Reflections on a crisis – opportunities for a teaching overhaul

Written by Angela Attwood and Olivia Maynard, with reflections from Marcus Munafò

Beyond the immediate impact on people’s lives and livelihoods, the SARS-CoV-2 pandemic has caused a great deal of disruption in how we work. The burden on academics, particularly with respect to teaching, has been considerable. But are there positives that we can take from this situation?

Academia can be surprisingly conservative – we have ways of working that we are reluctant to change. While undergraduate courses may have been tweaked in response to student feedback, they remain largely unchanged from the courses available in the 1990s. Yet over this same period the ways that young people digest knowledge has changed radically. Today’s undergraduates are digital natives, used to receiving content in very different (and more flexible) ways.

Once we knew the pandemic would force us to move to online teaching, and that we’d be delivering our third-year optional psychology unit on ‘Drug Use and Addiction’ online, we knew we had to take the opportunity to completely overhaul our course and update our pedagogy.

We started by identifying key principles that would inform the redesign of the course. As we outline below, we aimed to: ensure clarity, maximise engagement, facilitate presence, tackle the “valuable but missable” problem of live sessions, and be flexible.

Our redesigned course followed a flipped lecture format, whereby asynchronous material was delivered ahead of an online live (synchronous) session. This flipped approach is known to have pedagogical benefits over traditional didactic lectures. This was a substantial structural change to our course, but throughout we tried to avoid reinventing the wheel! Rather we wanted to create a course that was pedagogically sound, based on current evidence, and shaped by our key principles.

The feedback so far from students has been overwhelmingly positive (perhaps even more so than in previous years!) and we strongly believe our principles have been key to the success of the course. We therefore want to expand on each principle and share what we have learned so far, in case this is helpful for others also faced with the daunting task of complete course redesign.

Principle 1: Ensuring clarity

More than anything else, it was essential that students understood what they needed to do and why they need to do it.

What we did:

  • Created a consistent structure. We had folders for each sub-unit (previously lectures, but “sub-unit” better captures the granular nature of the content). Released weekly, these contained all teaching material (e.g., pre-recorded mini “lectures”, reading, etc.) for that week.
  • Ensured requirements were clear. Each sub-unit started with a “roadmap”, including a summary of the sub-unit, intended learning outcomes, and an ordered list of tasks for completion, with an estimate of the time required for each.
  • Clarified the importance of each task. We labelled these as either as CORE or RECOMMENDED. This allowed students flexibility, as they could choose to leave or return to RECOMMENDED items.
  • Provided guidance notes for all academic reading (i.e., journal articles, book chapters). This included an overview (why it was chosen), any focussed reading (particularly useful for long review articles), and key “take home” messages *.

* This was an unexpected “win” as our discussion board inbox was significantly quieter this year. Many questions in previous years asked how to make notes on or read journal articles in the context of the course. The number of these questions received at the point of writing is zero!

Example sub-unit structure from one of the “roadmaps”

Student feedback

“The structure for the sub-units is SO helpful, really like how it tells you how much time each activity is going to take.”

“The pre-recordings are a very good length, and the little summary of everything we are doing for the sub-unit with the timings is incredibly helpful.”

Principle 2: Maximising engagement

Students are spending more time working at home, due to local or national restrictions, or limits on campus study space. This means that as well as material needing to be high quality, it also needs to be interesting and engaging. We focussed on material that was digestible, offered various methods of delivery, and gave students flexibility in how they structured their own learning.

What we did:

  • Lectures recorded into bitesize chunks (ideally of no more than 20 minutes each). This reduced the burden associated with listening to each lecture and provided students with more flexibility when it came to organising their learning.
  • Academic reading was supplemented with additional materials (e.g., videos, podcasts, websites). This allowed students to explore areas of personal interest more deeply if they wished to.
  • Student-led activities (e.g., interview their friends, own literature searches, evaluate websites, mini-experiments). This provided opportunities for students to again explore areas of personal interest more deeply, in a range of different ways.
  • Student choice (e.g., choosing a drug they were interested in, and activities that could be aligned to build a “portfolio” of materials specific to their drug of choice). This fed through to assessment where they could answer the question on any drug they wanted.

Student feedback

“I really like the sub-unit structure. As someone who doesn’t learn best unless there is a range of different learning stimuli in combination (e.g., lecture content, reading, visual cues like videos/stats graphs etc.) I find the subunits are so interesting and they help me to focus my energy onto the task at hand and stops me getting distracted.”

“I really enjoyed the variety and the fact it wasn’t just hours and hours of straight lectures which can get really dull! :)”

Principle 3: Facilitating presence

Working through material posted on a website can be isolating. It’s important to create a sense of community in online settings.

What we did:

  • Used software that enabled student interaction and reflection throughout the week (e.g., Padlet, Mentimeter). We made sure at least one of these was present in each sub-unit, and encouraged students to communicate with each other as well as ourselves.
  • Recorded “reflection” lectures between ourselves (lecturers) or invited guests. This ensured that students saw our faces during the week, as well as a range of different contributions from the wider academic community.
  • Held weekly live sessions on Zoom to reflect on the week’s teaching. Although not strictly necessary, we both attended all live sessions to maximise our interaction with the students, and encouraged students to have their webcams on during these sessions (about half did).
  • Held weekly live drop-in sessions (in addition to core live sessions) to answer questions and chat. This provided further opportunities to interact directly with ourselves and other students in real time.
  • Used Zoom functions in live sessions – including breakout rooms – to give students a chance to talk to each other. We also used Zoom polling to ensure that all students had an opportunity to contribute, even if they didn’t feel like talking.
  • Emailed the cohort regularly with additional opportunities, talks etc. relevant to the course. This created a sense of the wider academic community that they are part of, and the ongoing research activity relevant to the course.

Student feedback

“I really enjoyed the smaller rooms when on Zoom to talk to others in small groups of 5. Found it a lot easier to talk in these smaller groups than larger ones. I also liked the multiple-choice questions that you can present on the screen to see how everyone else is doing in terms of the sub units and the current work.”

Principle 4: Tackling the “valuable but missable” problem for live sessions

One of the biggest risks to any live session are technical issues. This created a “valuable but missable” paradox – we didn’t want to deliver core material during live sessions (so they could be missable if a student had Internet issues), but the sessions also had to be seen as valuable (or students might not attend!)

What we did:

  • Constructed live sessions to be “skill building” (e.g., essay planning, argument building, debating skills, evidence synthesis and critique). These were designed to be valuable across the course as a whole, but any one could be missed with limited impact on assessment.
  • Created different formats for the live sessions to make sure these were seen as valuable, but also interesting and engaging (e.g., discussion on how to answer a mock essay question, multiple choice quiz, hot topic debates).

Student feedback

“I really liked the quiz session last time, it made me think about the information I absorbed in the subunits, but equally I loved the debate. Practice essay questions are also very useful because I am struggling with planning my essays in general.”

“I think the live sessions have been very beneficial in a number of ways related to our essays, overall course understanding and guiding areas for reading.”

Principle 5: Being flexible (we are learning too!)

Co-design with end-users is vital for the best end-product. We allowed time to ask for student feedback, and space to respond to it.

What we did:

  • Created polls that allowed students to vote on upcoming content (e.g., what question would be discussed in live sessions; what format of live sessions they find most helpful).
  • Kept aspects of the course only partially developed (e.g., live session format) so that we had scope to be responsive to feedback.
  • Continually asked for student feedback, via short polls and surveys on specific questions (e.g. ‘What should the format of the live sessions be?’, ‘How long should we stay in breakout groups for’) as well as asking for stop-start-continue feedback on the course as a whole, via an online survey that students could complete at any point during the course.

So, what does the future hold?

While we all hope ‘normal’ life will resume soon, the reality is that the world will not be quite the same post-pandemic. Much like many businesses that are planning to retain positive elements of home working, we should be open to retaining elements of our new ways of teaching. The crisis of the pandemic has created an opportunity to fundamentally overhaul and modernise the way that we teach that would have been unthinkable in a ‘normal’ year. And it seems to have worked – to quote a student, “It’s better than face to face teaching” (emphasis added).

We agree that these new ways of teaching are better – not just for students but for academics too. The recorded asynchronous material will stay current for 2 or 3 years (and perhaps longer for more introductory courses), meaning that if we retain this overall structure, our workload will be less next year. At the same time, many of the various synchronous elements can return to a face-to-face format, ensuring we spend more time in small groups, doing interactive work which both students and academics (certainly ourselves) find more engaging and fulfilling.

While our model is certainly not perfect – it had to be developed rapidly under considerable pressure – it’s a start, and offers a glimpse of the future.

Maximum cigarette pack size: a neglected aspect of tobacco control

Written by Anna Blackwell, Senior Research Associate

The manufacturing or importing of packs of cigarettes with fewer than 20 cigarettes per pack was prohibited in the UK when the EU Tobacco Products Directive and standardised packaging legislation were fully implemented in May 2017. This change was aimed at reducing the affordability of cigarettes and thereby discouraging young people from smoking. This directive also required the removal of branding and established a standard shape and dark green colour for packaging, including pictorial health warnings, which prevented the use of packaging for promotion and reduced its appeal.

However, the tobacco industry has been able to exploit loopholes in recent packaging regulations, including the absence of a regulated maximum pack size, by introducing non-standard and larger pack sizes to the market to distinguish products. This is a public health concern given evidence that larger pack sizes are linked to increased smoking, and could undermine existing tobacco control success.

In a recent Addiction Opinion and Debate paper, we proposed that a cap on cigarette pack size should be introduced; a pragmatic solution would be to permit only a single pack size of 20, which is now the minimum in many countries. This approach would reduce the number of cigarettes in packs in several countries such as Australia – where packs up to sizes of 50 are available – and prevent larger sizes being introduced elsewhere.

Capping cigarette pack size therefore has the potential to both reduce smoking and prevent increased smoking. While the health benefits of reducing smoking alone are small, it may have important indirect effects on health through its role in facilitating quitting. Those smoking fewer cigarettes per day are more likely to attempt to quit and succeed in doing so. Trials of smoking-reduction interventions have also found that these can lead to increased quitting when combined with nicotine replacement therapy.

Our Opinion and Debate paper drew on evidence from a range of sources including industry documents and analyses, population surveys, intervention trials and Mendelian randomization analyses. Together these suggest that consumption increases with larger pack size, and cessation increases with reduced consumption. However, direct experimental evidence is not currently available to determine whether pack size influences the amount of tobacco consumed, or whether the association is due to other factors.

People who want to quit may be using smaller packs as a method of self-control, and smokers who successfully cut down and later quit may be more motivated to do so. Cost is also an important factor and larger packs may be linked to increased smoking because they have a lower cost per cigarette. Further research is needed to determine whether the associations between pack size, smoking and cessation are causal to estimate the impact of policies to cap cigarette pack size.

Commentaries on our Opinion and Debate paper, published in the May 2020 Issue of Addiction highlight the need to understand the mechanisms for the associations observed between pack size and smoking in order to identify the optimal cigarette pack size. Although mandating packs of 20 is a pragmatic approach, pack size regulation needs to achieve a compromise between tobacco affordability and smokers’ self-regulation. Nevertheless, the policy debate should start now to address this neglected aspect of tobacco control.

To find out more visit the Behaviour Change by Design website or follow us on Twitter @BehavChangeDsgn @BristolTARG

Leaving the Lab: Rising to the Challenge of Remote Research

Written by Angela Attwood and Maddy Dyer

COVID-19 impact on research

The coronavirus (COVID-19) pandemic has forced millions of us to embrace remote working, and researchers are no exception. Universities are closed and face-to-face research with human participants has been temporarily halted. This has created challenges for our research, and laboratory and field studies are particularly affected.

As part of a large research group at the University of Bristol, we had to respond to this new situation and develop contingency plans for our research. Our first step was to review ongoing research and identify which studies could be suspended. We were fortunate on two counts: 1) we were able to put data collection on hold for many of our studies because there was enough flexibility in our planned completion times, and 2) we were able to stay busy with other work (such as analysing or writing up data from completed studies or developing new grant applications).

In all honesty, we would probably have stopped there with our contingency plans if it hadn’t been for one study that did not have this flexibility. This study investigates smokers’ experiences of switching to e-cigarettes, and requires participants to vape and complete various tasks across a two-week period. If we weren’t able to deliver on this study by a fixed deadline of the end of 2020, the funding would be withdrawn and one of our research staff would have been out of work. This forced us to think again…

Challenges and opportunities

There have been more challenges than opportunities in the context of the COVID-19 outbreak. However, one thing this global crisis has encouraged is innovation and creativity. We responded to the pressing need to complete this project by adapting our protocol so that the study could be run remotely (with no face-to-face communication). Some studies can move onto online platforms (and some of our changes include the use of online surveys), but our study involves participation over a two-week period with multiple “visits” and the use of electronic cigarettes. This required substantial adaption of the study methods (see below for some examples), but we were able to produce a comprehensive revision that retained the necessary components to ensure valid testing of our original research question.

We had to overcome several practical challenges, such as how to screen participants for smoking status and pregnancy. In the laboratory, we typically verify smoking status using a carbon monoxide (CO) breath monitor – equipment that cannot be used during lockdown. To overcome this, we replaced the CO monitor with a cotinine urine test, which verifies smoking status by detecting a metabolite of nicotine in urine. These are dipstick tests that participants can take themselves and we verify the outcome via a video call by asking them to show us the used dipstick.

Another challenge was how to safely deliver the e-cigarettes, e-liquids, and screening tests to participants. We are doing this via post (using pre-paid postage), with carefully constructed information packs and cleaning instructions. All test sessions that collect primary outcome data are now taking place online. This includes a cue reactivity procedure that participants are led through via pre-recorded instructions that link to our online study materials. We are also exploiting ecological momentary assessment methods (daily messaging via mobile phones) to collect real-time data across the test weeks, and all face-to-face communication has been replaced with phone and video calls. We worked closely with our faculty research ethics committee and university IT services as we developed this protocol to ensure any new ideas were feasible and ethically sound (or to identify problems early and seek alternative solutions).

Our aim was to complete a project that otherwise would not have been possible. However, the important learning point was that in developing essentially what was an “emergency response” protocol, we have unlocked other important benefits. Before the COVID-19 outbreak, our biggest challenge was recruiting participants (we require smokers who are willing to abstain from smoking for one week!). As with all university-based research, we often rely on opportunistic recruitment that means recruiting from the local area (i.e., people who can easily attend the laboratory sessions during university opening hours), and our samples often comprise a relatively high number of students. This not only means we have difficulty recruiting, but that our samples are not always representative, and our results may not generalise to the wider population. This new model of working means we have no geographical restriction (as long as the post delivers and there is Internet provision – we can collect data!), hugely improving our reach and the diversity in our participant samples.

Another benefit is that data are collected in more naturalistic settings (although this comes with a loss of control that needs to be considered or may not be appropriate for some studies). For studies that require participants to attend multiple sessions, it is also likely there will be lower attrition (i.e., fewer drop-outs) as there is less burden on participants to travel to a testing laboratory.

This has not been an easy transition (although we will certainly be well equipped to do it again if we need to). It has been time consuming, and some aspects of the study were simply not possible in the context of the fixed time constraints and funding in place, and without the laboratory facilities. The utility of this approach needs to be considered on a case by case basis. But, for our study, it was doable. We are only at the start of this process – the study will be running throughout 2020 and we look forward to the ongoing challenge and reflecting on how we can optimise this process in future.

The important take-home message is that remote research is not something we will discard after the COVID-19 restrictions are lifted. We will refine these methods and embrace the benefits they offer. Remote research will not be an emergency response option only, but instead it will be an integral part of our research toolkit.

If you are interested in finding out more, please visit our website:

http://www.bristol.ac.uk/psychology/research/brain/targ/participants/smoker-experience-ecigs/

You can also follow us on Twitter: @BristolTARG @AngelaAttwood @MaddyLDyer

 

Improving the way research is done: the UK Reproducibility Network

Written by Natalie Hunter, Graduate Trainee – Research Landscape at Wellcome Trust

 

As a graduate at Wellcome, I get the opportunity to be involved in so many exciting initiatives, including working with the people aiming to tackle some of the biggest challenges in science and research.

 

Attending the first annual meeting of the UK Reproducibility Network (UKRN) last Friday is a great example of this. UKRN is a grassroots, researcher-led organisation with the aim of improving scientific integrity, with a particular focus on the reproducibility of research. There’s been a lot written about the so-called ‘reproducibility crisis’ in research recently, but it essentially boils down to this: were the experiments conducted in a non-biased way, blinded where possible? Have all of the results been reported – including negative ones? Were appropriate controls used? Could the study be repeated by somebody else and the same results be found? However, it has become increasingly obvious that a number of papers, some with very influential results, have not been conducted to this standard. For example in some areas such as preclinical cancer research,  studies estimate as many as 70-90% of papers have irreproducible results. This links to the wider problem of research culture we’ve identified an­­d are working on here at Wellcome.

 

UKRN was founded to increase standards in research. Lead by four researchers – Marcus Munafò (Psychologist, University of Bristol), Laura Fortunato (Anthropologist, University of Oxford), Chris Chambers (Neuroscientist, University of Cardiff) and Malcolm Macleod (Neuroscientist, University of Edinburgh), the network is made up of a steering committee and two groups: Local Network Leads and Stakeholders.

 

The Local Network Leads are researchers, each representing a university. Together, they provide on-the-ground support within universities to help other researchers and policymakers navigate this complex area. UKRN wants local network leads to create excitement and buzz about doing high quality research. They support for example local ‘Reproducibilitea journal clubs’, where researchers collectively review research papers and discuss methodological issues.

 

The Stakeholders group is made up of representatives from research-related organisations, including Wellcome, UKRI, MRC, Nature, PLOS, JISC, UK Research Integrity Office, Universities UK and many more. Each organisation contributes small grants to UKRN, including Wellcome. By engaging these two key groups, UKRN aims to achieve both a bottom-up and top-down influence on UK research culture.

 

I attended the second half of the meeting, which was specifically for the Stakeholders. The focus of this meeting was the work plan for the year ahead. Discussion was lively, ranging from the responsibility of funders like Wellcome to those of journals such as Nature, data sharing and open access policies, the value of ‘metaresearch’ – or ‘research on research’ – and expanding discussions on these issues beyond biomedical science. The incentive structures within academia came up frequently as a key cause for concern. With publishing in a high-impact journal still seen as the key measure of success in science, despite intitives such as DORA, the concern is that pressure to publish could lead researchers to behave less than perfectly. As one attendee said, this is a “systemic issue, which needs a systemic solution”. That’s where UKRN’s strength lies. They have managed to get so many key stakeholders in a room at once to discuss these issues and commit to finding solutions that it feels like real change is on the horizon.

 

The potential of UKRN is exciting, and there is a sense that it is capitalising on a cultural moment in science right now; those involved feel there is a real appetite for change from a number of directions. But it’s important to remember it’s a very small organisation, with an administrator as the only paid member of staff – everybody else is involved on a purely voluntary basis. There’s only so much an organisation like that can achieve in a year. However the plans UKRN have set out for the next year are bold and ambitious: it will for example continue ongoing work (funded through the Wellcome Research on Research scheme) on linking the registered reports system with funding decisions; it will plan a large conference for next year, bringing together relevant parties for extended conversations and workshops; it will continue to grow its network and build an evidence base for improving research integrity.

 

So, watch this space – and the UKRN twitter account – as this network will only continue to grow and develop its influence over the coming year, with support from Wellcome and other funders.

Understanding Anorexia – Promoting Life through Prevention

An essay by Caitlin Lloyd.

Emma was an anxious child, always worrying. At thirteen, her anxiety became centered on interactions at school – she was terrified of being judged negatively by classmates. Around this time Emma began dieting, intending to lose just a small amount of weight. It turned out she could do so relatively easily, and enjoyed the sense of achievement resulting from the numbers on the scale going down. Her diet continued, becoming more and more extreme. Emma’s weight plummeted.

Eight years later, having had two inpatient hospital admissions, Emma maintains a dangerously low body weight, achieved by setting strict rules around eating. A daily calorie limit is followed, and foods containing fat and sugar avoided. Eating takes place only at certain times, and each mouthful must be chewed ten times before swallowing. Any deviation from these rules, and the day is ruined.

Emma retook two years at school, falling behind her peers, but secured a place at Durham University to study mathematics. It is difficult to concentrate on her work though, because all Emma can think about is food: what she has eaten; and what she will eat. Her focus on food makes it hard to maintain friendships, and Emma has few. Emma spends university holidays with her family, the time dominated by arguments over food.

Sometimes Emma wishes things were different. But that means eating more, which feels impossible. Deviating from the rules makes Emma unbearably anxious. No amount of support can dispel the intense fear of becoming fat, or feelings of self-disgust that accompany weight-gain.

Emma is fictional but typical of someone with anorexia nervosa, an eating disorder characterised by persistent starvation in the context of a low weight and fear of weight-gain. In the UK it is estimated that as many as one in 25 women will experience anorexia in the course of their lifetime. Men develop anorexia too; roughly one in ten people with anorexia is male.

Anorexia usually develops during adolescence, and has many adverse yet long-lasting physical and mental health consequences. Starvation compromises the function of almost all major organ systems, and feelings of despair increase the risk of suicide; anorexia has the highest death rate of any mental health disorder.

Full recovery from anorexia is a lengthy process, and unfortunately not common. Treatments exist but not one is consistently effective. Fewer than half of those diagnosed with anorexia make a full recovery, and relapse rates are high – around 30-40% of people fall back into the disorder’s grip following initial recovery. For some, weight-gain is sustained, but a strict diet and overconcern with eating and weight remains, severely impacting quality of life.

The difficulty treating anorexia makes effective prevention vital. For this we need to target the factors that cause anorexia, requiring knowledge of what those factors are. My research investigates whether anxiety disorders play a causal role in anorexia development, to help us understand whether it would be beneficial to address anxiety in young people to prevent eating disorders.

It has long been suggested that the starvation of anorexia reduces anxiety. This would make dieting helpful (in this narrow sense) to those experiencing anxiety symptoms, encouraging the dieting to continue. Anxiety disorders and anorexia often co-occur. But correlation is not causation, and determining cause-and-effect is notoriously challenging.

As an example, for anxiety to cause anorexia development, anxiety must precede anorexia. Existing findings support this, however studies have tended to ask people with anorexia to recall the time before their illness developed. Experiencing anorexia may affect memory recall; to try and explain how their anorexia developed, someone with anorexia might believe themselves to have been more anxious in childhood than they actually were. In this case the conclusion that anxiety causes anorexia may be invalid. Many sources of potential error exist in research, meaning that many findings could be inaccurate, at least to some degree.

Different research methods have different strengths and limitations, and are thus prone to different biases. This can be used to our advantage: if findings across studies of different research methods point to the same conclusion, we can be more confident the conclusion is correct. I am using a variety of research methods, each designed to minimise the potential for erroneous conclusions, to determine the role of anxiety in anorexia. If a causal role is supported across the different studies, trialing interventions designed to reduce anxiety for eating disorder prevention is encouraged. If not, the search for other factors to target for improved eating disorder prevention continues.

We are at an early stage in understanding anorexia, but we do know that many people with the illness become ill at a young age, with their whole lives ahead – like Emma. My research matters because it aims to stop people losing their lives, and quality of life, to anorexia.

 

Does schizophrenia influence cannabis use? How to report the influence of disease liability on outcomes in Mendelian randomization studies

The recent Nature Neuroscience paper by Pasman et al entitled “GWAS of lifetime cannabis use reveals new risk loci, genetic overlap with psychiatric traits, and a causal influence of schizophrenia” (see below) provides important and novel insights into the aetiology of cannabis use and its relationship with mental health. However – in its title and elsewhere – it subtly misrepresents what the Mendelian randomization (MR) [1] analyses it presents actually show. MR analyses are increasingly being reported as demonstrating the effect of a disease (in this case schizophrenia) on the outcome, through using genome wide significant variants associated with risk of the disease on the outcome (which can be a behavior, such as cannabis smoking in the present paper, a measured trait or a second disease).

MR analyses are often carried out using summary data, where the exposure and outcome GWAS come from separate samples. In such analyses interpretation is not to apparent effects of the disease itself, but to the phenotypic effects of genetic liability to that disease. Typically, only a tiny proportion of participants in the outcome GWAS datasets will actually have experienced the disease – in this case particularly so given the low participation rate of people with schizophrenia in most studies in the general population. Indeed, MR studies can be carried out in datasets where there are no individuals with the outcome (e.g. datasets collected amongst an age group in whom the outcome will have occurred very rarely, if ever). Such analysis may reveal apparent, but impossible, effects of the disease on outcome phenotypes. To use MR analyses to investigate the causal effect of a disease on outcomes would require individual-level data with recorded disease events and subsequent follow-up. Analytical approaches to such data have, as yet, not been published.

The widespread misrepresentations of such MR studies have important implications, not just in terms of how the results are interpreted, but also how they are applied. One valuable contribution of MR studies is that they can identify modifiable exposures that can be the target of interventions. If it is recognized that what is being shown is an effect of liability to disease on an outcome, then interventions targeting the mechanisms of this liability would have benefits even in individuals who are unlikely to go on to develop the disease, including those at low risk of the disease for other reasons. For example, targeting breast cancer liability may have benefits in men if this liability influenced diseases that are common in men. If, however, it is the disease itself which has the effect, then the interventions would be targeted at those likely to develop disease: only women, in the case of breast cancer liability. It may be that schizophrenia does indeed lead to cannabis use, but the analyses reported by Pasman et al show only that liability to schizophrenia leads to cannabis use.

The point is a subtle one – we have both used similar language in the past in articles reporting MR analyses on which we are authors. Indeed, one of us (MM) was an author on the Pasman et al paper (and contributed principally to the MR analyses and their interpretation) but failed to suggest the correct phrasing.  Fortunately, the title and discussion will be changed to address this problem so that the enduring version of the paper captures this importance nuance (unfortunately, the original headline has already been repeated elsewhere [2]). However, it is a widespread and underappreciated point of interpretation in MR studies, and we feel that this presents a useful opportunity to highlight it. It also illustrates that methodologies, and the interpretation of the results they generate, continue to evolve, illustrating the need to interpret past work (including our own!) through the lens of current approaches.

[1] Davey Smith G, Ebrahim S. ‘Mendelian randomization’: can genetic epidemiology contribute to understanding environmental determinants of disease?  Int J Epidemiology 2003;32:1-22.

[2] Andrae LC. Cannabis use and schizophrenia: Chicken or egg?  Sci Transl Med 2018;10:eaav0342.

The House of Commons Science and Technology Committee reports on e-cigarettes

Written by Jasmine Khouja, PhD Student, Tobacco and Alcohol Research Group

Today sees the publication of a report on electronic cigarettes (e-cigarettes) by the House of Commons Science and Technology Committee. This compiles evidence from over 100 pieces of written submissions and five oral sessions, and highlights key issues around reducing harm, promoting smoking cessation and effectively regulating e-cigarettes. Since the report is quite long, we’ve tried to extract the main messages.

The report takes a relatively positive stance on e-cigarettes, encouraging use for smoking cessation and suggesting a more accepting approach to e-cigarettes in public spaces. This is in contrast to other countries, such as Australia, where a ban is in place due to the lack of long-term research on the health impact of using e-cigarettes.

Reducing harm

The general consensus from a variety of sources is that e-cigarettes are less harmful than combustible cigarettes. However, a frequent theme is that this does not mean that e-cigarettes are ‘safe’, and the report is careful to emphasise that e-cigarettes are not completely harmless. The relative harm of heat-not-burn tobacco products compared to combustible cigarettes is less clear. There is a lack of independent evidence as the majority of data on the safety and emissions of these product has come from Philip Morris, a major tobacco company.

The long-term effects of using e-cigarettes are currently unknown. It is difficult to assess the comparative harm of e-cigarettes without also measuring the effects of prior smoking, since there are very few long-term e-cigarette users who have never smoked. Exposure to second-hand e-cigarette vapour has been similarly difficult to assess, but since potentially harmful compounds emitted are present only at very low levels second-hand vapour is unlikely to be harmful.

E-cigarettes have become a popular tool for quitting smoking and an estimated 16,000 to 22,000 people who would not have quit using alternative products or willpower alone have successfully quit each year by using e-cigarettes. Although these figures are promising, there is a lack of high-quality evidence from randomised control trials showing how effective e-cigarettes are when quitting smoking. Other evidence has been inconclusive due to the low quality of some studies.

Despite fears that e-cigarettes may act as a ‘gateway’ to smoking, current evidence does not show that using e-cigarettes causes people to start smoking. Although there is a link between e-cigarette use and subsequent smoking initiation, very few never smokers regularly use e-cigarettes, so any causal link would have a limited impact on smoking rates.

Smoking cessation

Providing e-cigarettes on prescription could encourage smokers to try e-cigarettes without barriers such as money as well as give them more confidence in the product being less harmful than cigarettes. The report concludes that e-cigarettes should be available to those in NHS mental health services given high rates of smoking in this group.

NHS England were unable to provide evidence for how they were addressing this issue. They were unable to provide a representative because there is no one individual responsible centrally with ‘oversight’ of e-cigarette policies across NHS mental health trusts. The report criticises this, stating it was concerning and that a position should be created as a matter of urgency.

E-cigarettes are generally prohibited in closed spaces such as workplaces, public transport and restaurants and vapers are usually encouraged to vape outside within designated ‘smoking’ areas. Since second-hand vapour is unlikely to be harmful, these policies may be more harmful than beneficial; frequently exposing vapers to cigarettes and cigarette smoke may increase the likelihood that they will relapse to smoking.

Regulation

E-cigarettes are currently regulated under the Tobacco Products Directive (TPD; see our previous blog) if you want to learn more about these regulations). As part of this directive, the Medicines and Healthcare products Regulatory Agency must be notified before any e-cigarette or e-liquid can be sold in the UK.

Four key criticisms of the TPD were identified in the report: i) unnecessary limits on nicotine strength of refill liquids which may lead to failed quit attempts, ii) unnecessary tank size restrictions which may lead to failed quit attempts, iii) blocking advertising the relative harm-reduction of e-cigarettes which may discourage quit attempts, and iv) the ineffective notification scheme for e-cigarette ingredients which slows innovation.

Some TPD regulations are optional and give freedom to governments to be as restrictive as they feel necessary. Scotland has been more restrictive than England in their regulations by banning certain advertising of vapour products. Currently, health claims are banned from all media advertising of e-cigarettes without a medical license (of which none is currently available). The Advertising Standards Authority is currently reviewing the legislation on e-cigarette advertising and health claims and are considering allowing this in the future.

Unsurprisingly, there is uncertainty about the future regulation of e-cigarettes due to Brexit. Regulation of e-cigarettes may change after leaving the European Union and it is unclear what these changes may be or what potential impact increased flexibility in regulating e-cigarettes could have.

Conclusions

The report is comprehensive and raises some interesting questions particularly about the lack of NHS involvement in developing strategies for smoking cessation that utilise e-cigarettes. It will be interesting to see if the NHS responds to these criticisms by taking action. I am also interested to see what Brexit will mean for the regulation of e-cigarettes in the UK, given the criticisms of TPD regulations.

The full report can be accessed here: [The House of Commons Science and Technology Committee reports on e-cigarettes]

 

Why I took part in the “Preregistration Challenge”

By Sarah Peters

The preregistration of study protocols has a long history in clinical trials, but is a more recent innovation in many other areas. The hope is that it will help counter the “reproducibility crisis” in psychological science – the failure of many published findings to replicate reliably. Here I discuss my experience with the Open Science Framework “Preregistration challenge”, and argue for more widespread adoption of preregistering reports.

There is an ongoing methodological crisis in psychological science – the reproducibility crisis refers to the failure of many scientific findings to be replicated. The Reproducibility Project, a recent initiative led by Professor Brian Nosek at the University of Virginia, aimed to identify the scale of this crisis. A large collaboration between 270 project members reran 100 published psychological experiments, and found that just 36% of the initial findings were replicated. Similarly, some classic textbook experiments have proven difficult to replicate, and publication bias – whereby positive findings are more likely to be published and negative findings to be dismissed – plagues the field.

Given this, scientists are exploring how to improve the way we conduct research and thereby improve the quality of what we produce. One suggestion is to preregister our research question, methods and analysis plan in advance of data collection. It is hoped that public preregistration will limit analytical flexibility and post hoc hypothesising, thereby improving the transparency and robustness of research findings.

Curious about the benefits of preregistration, and to see how it differed from the way I’d previously conducted my research, my colleagues and I published a preregistration for a recent study on Open Science Framework (OSF). We were interested in whether Cognitive Bias Modification, a psychological intervention designed to shift the emotional interpretation of faces, would impact clinically-relevant outcomes. We also entered the study into OSF’s (ongoing!) Preregistration Challenge, which offers the chance to win a $1,000 prize to 1,000 researchers who go from preregistration to publication.

Preregistering our study did require a greater time commitment prior to running it, but thinking about our predictions, design, and analyses meant that we could spot any potential issues and improve our experimental design before we collected data (i.e., before it was too late!). As a preregistration is public and cannot be changed after it’s published, it forced us to think more carefully about our decisions. For example, thinking more carefully about whether our data would truly answer our question made us wonder whether the emotional biases we wanted to study might be more prominent when an individual is under stress, so we decided to include another task to measure this. Also, by knowing which statistical analyses we would conduct before recruiting participants we could ensure that our study was adequately powered and would meet the assumptions of the planned analyses.

Initially I was concerned that this approach could be limiting. What if we found something interesting that we hadn’t expected and wanted to run additional analyses to probe it? But a preregistered report doesn’t prevent that – it simply means that you would (honestly and transparently!) report those analyses as exploratory. This protection against HARKing (hypothesising after the results are known) is important; separating analyses as planned versus exploratory can prevent overconfidence in weaker findings and the publication of attractive, but uncertain, positive findings.

Following data collection, we went back to our preregistration. It was here that our earlier time investment paid off; once our data were cleaned we could immediately run our planned analyses, and much of the manuscript writing (introduction and methods) was already done. We also ran a number of exploratory analyses, such as whether our results were moderated by participants’ anxiety scores. We subsequently published our findings in the academic journal Royal Society Open Science, and were thrilled to receive one of the latest $1,000 Preregistration Challenge prizes for bringing our study from preregistration to publication!

While interpreting findings and making discoveries is an important aim of scientific research, it is just as important to continuously scrutinise the scientific method. As a scientist, there is no question that seeing data can influence my decisions and interpretations. However, the adoption of preregistration can eliminate this, make the process easier in the long term, and improve research quality overall.

Professor Nosek and other members of the Reproducibility Project argue that, “Progress in science is marked by reducing uncertainty about nature”. But, if scientific findings have not or cannot be replicated, we can’t be certain that they exist. Preregistration is a simple change to the way we do research that can help to halt the reproducibility crisis and produce effective and credible science.

Read more about how to take part in the Preregistration Challenge here.

See Peters et al.’s preregistration here, and the published study here.

Sarah Peters can be contacted via email at: s.peters@bristol.ac.uk.

Can cognitive interventions change our perception from negative to positive, and might that be useful in treating depression?

By Sarah Peters

Have you ever walked away from a social interaction feeling uncomfortable or anxious? Maybe you felt the person you were talking to disliked you, or perhaps they said something negative and it was all you could remember about the interaction. We all occasionally focus on the negative rather than the positive, and sometimes ruminate over a negative event, but a consistent tendency to perceive even ambiguous or neutral words, faces, and interactions as negative (a negative bias), may play a causal role in the onset and rate of relapse in depression.

A growing field of psychological interventions known as cognitive bias modification (CBM) propose that by modifying these negative biases it may be possible to intervene prior to the onset of depression, or prevent the risk of subsequent depressive episodes for individuals in remission. Given that worldwide access to proven psychological and pharmacological treatments for mood disorders is limited, and that in countries like the UK public treatment for depression is plagued by long waiting lists, high costs, side effects, and low overall response rates, there is a need for effective treatments which are inexpensive, and both quick and easy to deliver. We thought that CBM might hold promise here, so we ran a proof of principle trial for a newly developed CBM intervention that shifts the interpretation of faces from negative to positive (a demonstration version of the training procedure can be seen here). Proof of principle trials test an intervention in a non-patient sample, which is important to help us understand a technique’s potential prior to testing it in a clinical population – we need to have a good idea that an intervention is going to work before we give it to people seeking treatment!

In this study, we had two specific aims. Firstly, we aimed to replicate previous findings to confirm that this task could indeed shift the emotional interpretation of faces. Secondly, we were interested in whether this shift in interpretation would impact on clinically-relevant outcomes: a) self-reported mood symptoms, and b) a battery of mood-relevant cognitive tasks. Among these were self-report questionnaires of depressive and anxious symptoms, the interpretation of ambiguous scenarios, and an inventory of daily stressful events (e.g., did you “wait too long in a queue,” and “how much stress did this cause you on a scale of 0 to 7”). The cognitive tasks included a dot probe task to measure selective attention towards negative (versus neutral) emotional words, a motivation for rewards task which has been shown to measure anhedonia (the loss of pleasure in previously enjoyed activities), and a measure of stress-reactivity (whereby individuals complete a simple task under two conditions: safe and under stress). This final task was included because it is thought that the negative biases we were interested in modifying are more pronounced when an individual is under stress.

We collected all of our self-report and cognitive measures at baseline (prior to CBM), after which participants underwent eight sessions (in one week) of either CBM or a control version of the task (which does not shift emotional interpretation). We then collected all of our measures again (after CBM). In order to be as sure of our results as possible, there were a number of critical study design features we used. Our design, hypotheses, and statistical analyses were pre-registered online prior to collecting data (this meant that we couldn’t fish around in our data until we found something promising, then re-write our hypotheses to make that result seem stronger). We also powered our study to be able to detect an effect of our CBM procedure. This meant running a statistical calculation to ensure we had enough participants to be convinced by any significant findings, and their potential to be clinically useful. This told us we needed 104 individuals split evenly between groups. Finally, our study was randomised (participants were randomly allocated to the intervention group or the control group), controlled (one group underwent an identical “placebo” procedure), and double-blind (only an individual who played no role in recruitment or participant contact knew which group any one participant was in).

So, what did we actually find? While the intervention successfully shifted the interpretation of facial expressions (from negative to positive), there was only inconclusive evidence of improved mood and the CBM procedure failed to impact most measures. There was some evidence in our predicted direction that daily stressful events were perceived as less stressful by those in the intervention group post-CBM, and weaker evidence for decreased anhedonia in the intervention group. In an exploratory analysis, we also found some evidence that results in the stress-reactivity task were moderated by baseline anxiety scores – for this task, the effects of CBM were only seen in individuals who had higher baseline anxiety scores. However, exploratory findings like this need to be treated with caution.

Therefore, as is often the case in scientific research, our results were not entirely clear. However, there are a few limitations and directions for future research that might explain and help us to interpret our findings. Our proof of principle study only considered effects in healthy individuals. Although these individuals were clearly amenable to training, and may indeed have symptoms of depression or anxiety without a clinical diagnosis, our observation that more anxious individuals appeared to be more affected by the intervention warrants research in clinical populations. In fact, a reasonable parallel to the effects observed in this study may be working memory training, which does not transfer well to other cognitive operations in healthy samples, but shows promise as a tool for general cognitive improvement in impaired populations.

Future research is also needed to disambiguate the tentative self-report stress and cognitive anhedonia effects observed here. One possibility, for example, is that the 104 participants we recruited were not enough to detect an effect of transference from CBM training to other measures (the size of which is unknown). Given the complexity of any mechanism through which a computerised task could shift the perception of faces and then influence behaviour, it is likely that a larger sample is necessary. While it could be argued that if such a large group of individuals is warranted to detect an effect, that effect is likely too small to be clinically useful, we would argue that even tiny effects can indeed be meaningful (e.g., cancer intervention studies often identify very small effects which can have a meaningful impact at a population level).

Another explanation for our small effects is that while one week was long enough to induce a change in bias, it may not have been long enough to observe corresponding changes in mood. For instance, positive interpretation alone may not be enough – it may be that individuals need to go out into the world and use this new framework to have personal, positive experiences that gradually improve mood, and this process may take longer than one week.

Overall, this CBM procedure may have limited impact on clinically-relevant symptoms. However, the small effects observed still warrant future study in larger and clinical samples. Given the large impact and cost of mood disorders on the one hand, and the relatively low cost of providing CBM training on the other, clarifying whether even small effects exist is likely worthwhile. Even if this procedure fails to result in clinical improvement, documenting and understanding the different steps in going from basic scientific experimentation to intervening in clinical samples is crucial for both the scientific field and the general public to know. The current study is part of a body of research which should encourage all individuals who are directly or indirectly impacted by depression or other mood disorders. Novel approaches towards understanding, preventing, and treating these disorders are constantly being investigated, meaning that we can be hopeful for a reduction in the devastating impact they currently have in the not so distant future.

Read the published study here

Sarah Peters can be contacted via email at: s.peters@bristol.ac.uk 

Does calorie and unit information influence our drinking behaviour?

By Olivia Maynard

Over the past two years we’ve invited hundreds of people into the lab to drink beer. Unfortunately, we weren’t there to socialise; this was in the name of science. We wanted to know whether giving people information about the number of units and or calories in their beer influenced how much they drank and their perceptions of drinking.

There are strong arguments for including this information: providing unit information may increase knowledge about alcohol consumption and calorie information may help drinkers choose lower calorie (and as a result lower unit) beverages. However, we also wondered whether there might be some unintended consequences of providing this information, particularly for those who are highly motivated to drink. What if unit information simply allows these drinkers to choose higher strength drinks and calorie information only discourages them from eating more, not drinking less? What if discussion around mandatory unit and calorie labelling is distracting us from the bigger issues: health warnings, minimum unit pricing, improving treatment for alcohol dependence and stopping alcohol advertising to young people, to name a few?

So, with this healthy level of scepticism, we set about inviting 264 regular alcohol consumers (mostly undergraduate students) to attend a lab session where they were given some beer and completed some taste ratings. What participants didn’t know was that they had been randomly assigned to one of four conditions. One group had information about the calorie and unit content of the beers, one group just got calorie information, another had just unit information, and the final group got no information at all. As well as measuring how much beer they drank, we also asked participants to reflect on the likely impact of unit and calorie information on their drinking behaviour.

You can read all the results in our (open access) paper that was published this week in the journal Alcohol and Alcoholism. If you want the concise version: we found no evidence that either unit or calorie information influenced how much beer people consumed and we found a lot of variation in the amount people drank.

However, it was our analysis of participants’ thoughts on unit and calorie information that proved vital to understanding what was going on here. Our participants told us that their main motivation for drinking alcohol was usually to get drunk; where unit information was perceived as being helpful, this was to help them choose the highest strength drink. Unit and calorie information was seen as distracting from the social aspect of drinking, and although some participants felt that calorie information might reduce consumption, most thought it would affect others, not themselves. Some people thought that calorie information could be misused by encouraging people to eat less (to compensate), rather than drink less.

It’s interesting that even though the unit and calorie information was very visible in our study (on a piece of paper, presented for 10 minutes), those who had received this information were still very inaccurate when it came to reporting how many units and calories were in their drinks. They basically didn’t seem to have read or engaged with it. If they’re not reading it in this context, is it likely that drinkers will read this information when it’s printed in tiny font on the back of the bottle?

So, what does this all mean for any plans to introduce unit and calorie information? Our study only really tells us about the potential impact of unit and calorie information among young adults (many of whom were students) who tend to drink to get drunk. However, our findings do call into question whether mandatory unit and calorie labelling on its own would reduce how much people drink, and also highlights potential negative unintended consequences of providing this information.

Despite some of these potential unintended consequences, there are still reasons to include unit and calorie information, if only because it’s a consumer right (you know how many calories are in just about everything else you consume). However, perhaps more effort needs to be placed on making this information more engaging and embedding it into public understanding of recommended drinking levels. Coincidentally, an analysis of the public’s awareness of new national alcohol guidelines was also published yesterday. This report argues that although the public have a relatively high awareness of what the guidelines are, they should be put into context by increasing the public’s awareness of the links between alcohol and cancer. Perhaps using health messages such as ‘Drinking alcohol regularly is linked to long-term risks such as cancer’, alongside unit and calorie information, might result in more meaningful changes in attitudes and behaviours around drinking. I feel another study coming on….

Olivia Maynard can be found on Twitter at @OliviaMaynard17