Research Associate (Emote VR Voicer AI)

The School of Digital Arts is a purpose built, interdisciplinary school at one of the UK's leading universities. Offering industry and research informed courses and specialist spaces with the latest technologies. The School of Digital Arts is a proud part of Manchester Metropolitan University. We build on the creative, science, tech and business strengths of a university whose research is rated as ‘world-leading' and is changing the way we live, work, learn and play.

AI systems are increasingly able to detect a speaker's emotions, leading to a new affective channel that can be explored in art. The controls available in standard Virtual Reality (VR) can be supplemented with speech recognition, natural language processing, and sentiment analysis. We aim to embody this potential in the front end of the Emote VR Voicer interface, which would translate detected meanings of vocal utterances to the morphing of abstract 3D animated shapes, enabling a radical new aesthetic experience. We are using iterative design cycles and ultimately aim to develop an interface that will improve the participant's wellbeing. Candidates do not need prior arts-sector experience, only openness to interdisciplinary collaboration.

Working within the School of Digital Arts (SODA) you will join state-of-the-art research on the AHRC funded Emote VR Voicer project, to contribute to the development of an intelligently responsive VR app that incorporates speech recognition and meaning classification. You will be responsible for the AI development part, which will consist of making the system intelligent enough to listen to live mic input and classify to emotion (happy, sad, calm or angry) aiming for a success rate of approximately 80% or above. In addition, you will make the model able to analyse language and classify to the same emotions based on that.

Key requirements for our system are minimum latency to allow the visual response to be immediate and a good success rate that will work on most visitors. You will research, use and fine-tune existing models that will work on spoken or sung voice input. Once your model works in Python you will re-code it, potentially as a plugin, to be easily usable in Unity. No previous art experience is required but previous experience of interdisciplinary working is welcomed.

About the role:

You will be working closely within a small project team consisting of artists, a psychologist and AI researchers in an iterative development cycle. Your AI programming skills will be utilised to train and fine tune two AI models, one able to detect and tag emotional meaning from audio, and one classifying emotion based on the spoken/sung words. The outputs of these will steer real-time visuals based on four dominant emotions in the Unity Games engine. You will also be involved in writing up the research for publication. You will gain co-authorship on interdisciplinary publications, portfolio-quality work and exposure to exhibitions.

The job will be for 2.5 days per week (0.5 FTE) on a fixed-term basis for a maximum of 6 months. The working pattern will be partially on-campus and partially remote depending on project stage and flexible, or caring-friendly arrangements are welcomed where possible.

About you

Key skills:

• An excellent understanding of and experience with programming using Python for machine learning, C# and C++.

Essential skills and experience:

• A PhD in computer science, AI, software engineering or a similar technical field, or equivalent professional experience.
• Experience with machine learning and AI.
• Experience with writing and co-writing research papers.
• Experience with speech recognition (e.g. Whisper), sentiment analysis, or emotion classification.
• Experience with Image and/or audio-based AI development and familiar with libraries such as: SciPy, TensorFlow, PyTorch, Hugging Face.
• Experience with audio-based projects.
• Experience with real-time system optimisation (e.g. low-latency audio/visual feedback in VR).
• Experience with data backup systems.
• Experience with working in interdisciplinary teams.
• Excellent communication and interpersonal skills.
• Creative problem-solving skills.
• Self-motivation and capable of independent research.
• Excellent ability to work to deadlines.

Desirable:

• Experience with user testing or co-design methods, in arts or health settings.
• Familiarity with research projects productivity timelines.
• Knowledge of the peer review process for research projects and journal articles.
• Sensitive to nuances in visual aesthetics.

To apply, please submit your CV, a cover letter explaining how you meet the criteria and include a link to previous relevant work (for example a GitHub link) and two named references via our application portal (available on the university's website, accessed by the 'Apply' button). If you would like to discuss the role, please email Adinda at: A.vant.Klooster@mmu.ac.uk.

Manchester Metropolitan University fosters an inclusive culture of belonging that promotes equity and celebrates diversity. We value a diverse workforce for the innovation and diversity of thought it brings and welcome applications from all local and international communities, including Black, Asian, and Minority Ethnic backgrounds, disabled people, and LGBTQ+ individuals.

We support a range of flexible working arrangements, including hybrid and tailored schedules, which can be discussed with your line manager. If you require reasonable adjustments during the recruitment process or in your role, please let us know so we can provide appropriate support.

Our commitment to inclusivity includes mentoring programmes, accessibility resources, and professional development opportunities to empower and support underrepresented groups.

Manchester Met is a Disability Confident Leader and, under this scheme, aims to offer an interview to disabled people who apply for the role and meet the essential criteria as listed in the Job Description for that vacancy.