PhD Studentship - FSE PhD President's Doctoral Scholarship - Neuromorphic Systems for Low-latency and Efficient Multimodal Processing

This 4-year PhD studentship is open to both Home (UK) and international applicants. The successful candidate will receive an annual stipend at the UKRI minimum rate (currently £21,805 per year) plus £1,000/annum enhancement, and a research training support grant of up to £20,000 for the full 4-year programme. This project is in competition for funding with other projects. Usually, the projects that receives the best applicants will be successful.

Modern AI systems achieve remarkable performance on tasks ranging from language modelling to computer vision. However, their large size, and significant data requirements, make them costly to train and deploy, with AI systems set to consume 3% of global energy supply by 2030. Neuromorphic systems aim to solve this challenge, by taking inspiration from the brain, to develop algorithms and computational substrates that can achieve intelligent processing on a similar power (approx. 20W).

However, while modern AI systems have benefited from access to large volumes of high-quality and labelled data, these resources do not exist in the neuromorphic domain, limiting overall progress. The proposed supervisory team are addressing this gap and are currently collecting the world’s first phonetically rich multi-modal neuromorphic dataset, containing audio and video recordings of spoken language (ieeexplore.ieee.org/abstract/document/11043887). [ieeexplore.ieee.org] The unique contribution captures data using neuromorphic sensors allowing high temporal accuracy, while simultaneously introducing sparsityto increasing overall efficiency. 

However, while neuromorphic processors can exploit these data structures, specialised algorithms are required to perform tasks such as classification and reasoning, which are severely lacking in the community.

The project is structured into 3 equally weighted research streams to deliver new fundamental knowledge through harnessing the capabilities of neuromorphic sensors to better capture and process speech (Streams 1 & 2), and generate impact through applications of this technology in society (stream 3):

Stream 1: Algorithmic exploitation of neuromorphic audio-visual speech data, to address the major computational bottlenecks in speech recognition and speech production analysis, e.g. minimising data movement using neuromorphic algorithms and chips such as Intel’s Loihi2.

Stream 2: Developing language understanding through neuromorphic representations. Speech sounds are produced using coordinated movements of articulators to achieve specific acoustic targets (e.g. fast movements of the tongue and lips). These are challenging to process using conventional techniques, as individuals can produce the same sound in multiple ways, and the same articulatory movement in different individuals can lead to different sounds.

Stream 3: Impact through real-world applications of low-power speech processing, for tasks such as lip reading for the hearing impaired, and language understanding. The goal will be to develop technology suitable for deployment at the edge, such as in mobile devices and hearing aids, and with potential for embedding in augmented reality scenarios.

Please see here for further details.

Please contact the main project supervisor, Dr. Luca Peres, to learn more about the project and discuss whether it may be suitable for you.

To be considered for this project you must submit a formal application through our online application portal by following the instructions provided here. Please note that you must select programme FSE PhD President’s Doctoral Scholarship:

If you already have an applicant account, use this link to open an application for FSE PhD President's Doctoral Scholarship. If you do not yet have an applicant account, follow the account setup instructions and then use this link to open an application for the correct programme.

Advert information