PhD Studentship: What Do Effect Sizes Mean in Real Terms? Benchmarking Effect Sizes in Behavioural Research

A fully-funded studentship is available to work alongside the team working on the BR-UK project (https://www.linkedin.com/24/behavioural-research-uk/) at the University of Sheffield. The aim of the project is to use a technique called ‘benchmarking’ to identify how effect sizes of varying magnitudes correspond to real world behaviour and outcomes.

Behavioural researchers typically evaluate the effect of interventions (e.g., a campaign to promote physical activity or encourage recycling) by evaluating whether behaviour or outcomes following the intervention are statistically different from before the intervention or among a comparison condition. Because statistical significance (or not) depends on the size of the sample and provides little information about the magnitude of the effect, researchers have started to move toward reporting the size of the difference using metrics like Cohen’s d, eta2, Pearson’s r, which is often interpreted qualitatively as, for example, small, medium, or large (Cohen, 1992).

However, while effect sizes are useful (e.g., for comparing interventions, estimating the likely effect of an intervention in a new context), they provide little information on the actual change in behaviour and / or outcomes that might be expected because of an intervention. In this sense, the real-world implications of behavioural interventions are hidden from those who might want to use them (e.g., policy makers). For example, what does an effect of d = 0.20 of an intervention designed to promote physical activity mean in terms of changes in physical activity and associated outcomes?

Benchmarking involves identifying how effect sizes of varying magnitudes correspond to real world behaviour and outcomes. For example, Wright et al. (2021) used the standard deviations reported in large-scale studies that used accelerometer data to chart the distribution of physical activity in a population (e.g., Clark et al., 2019) to convert effect sizes from reviews of various interventions designed to promote physical activity. The headline finding was that median effect size reported by reviews of interventions (d+ = .21) was equivalent to an increase of 1,320 steps per day, 15.6 additional minutes of daily moderate/vigorous physical activity, and a 4.3% increase in the proportion of participants meeting WHO guidelines for physical activity.

This studentship will extend benchmarking to new contexts, working with each of the four themes with BR-UK to benchmark effect sizes associated with interventions relevant to: (i) Environment and sustainability, (ii) health and well-being, (iii) resilient communities, and (iv) organisations, markets, and the economy. We will also consider whether it is valuable to extend benchmarking to (i) draw analogies with commonly encountered / understood metrics and (ii) show how small effects accumulate to have important outcomes.

REQUIREMENTS:

A master’s degree (or relevant research experience) in Psychology or a related discipline and at least a 2:1 at BSc level. It will also be an advantage if candidates have good knowledge of meta-analysis and quantitative statistics.

All applications to be received 12 noon UK time 24 April 2024.

Information and enquiries via email to harriet.baird@sheffield.ac.uk

Advert information