Month: May 2021

Visual mental imagery is the quasi-perceptual experience of “seeing in the mind’s eye.” While it has been well established that there is a strong relationship between imagery and perception (in terms of subjective experience), in terms of neural representations, this relationship remains insufficiently understood.

In a recent article, researchers from NESTOR Project 1, at Maastricht University, exploit high spatial resolution of functional magnetic resonance imaging (fMRI) at 7 Tesla, uncovering the retinotopic organization of early visual cortex and combining it with machine-learning techniques, to investigate whether visual imagery of letter shapes preserves the topographic organization of perceived shapes.

Six subjects imagined four different letter shapes which were recovered from the fMRI (BOLD) signal. These findings may eventually be utilized for the development of content-based BCI letter-speller systems.

Mario Senden, Thomas C. Emmerling, Rick van Hoof, Martin A. Frost & Rainer Goebel. Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception. Brain Structure and Function 224, pages 1167–1183 (2019). https://doi.org/10.1007/s00429-019-01828-6

The human brain contains many neurons, the activity of which, when measured, respond differently to specific types of visual input. These neurons can be divided in retinotopic and category-specific regions and have been the focus of a large body of functional magnetic resonance imaging (fMRI) research. Studying these regions requires accurate localization of their cortical location, hence researchers traditionally perform functional localizer scans to identify these regions in each individual.

However, it is not always possible to conduct these localizer scans. Researchers from NESTOR Project 1 have recently published a probabilistic map of the visual brain, detailing the functional location and variability of visual regions. This atlas can help identify the loci of visual areas in healthy subjects as well as populations (e.g., blind people, infants) in which functional localizers cannot be run.

A Probabilistic Functional Atlas of Human Occipito-Temporal Visual Cortex. Mona Rosenke, Rick van Hoof, Job van den Hurk, Kalanit Grill-Spector, Rainer Goebel. Cerebral Cortex, Volume 31, Issue 1, January 2021

https://doi.org/10.1093/cercor/bhaa246

In this exciting episode of the popular children’s TV series, ‘Klaas Kan Alles,’ Klaas’s challenge is to drive around a race track in a go-kart, without using normal vision! A visit to the AI department at Radboud University might offer the magical solution: a mobile phosphene vision simulator! https://www.youtube.com/watch?v=1YMs6jXUs0s

For a visual prosthesis to be useful in daily life, the system relies on image processing to ensure that maximally relevant information is conveyed, e.g. allowing the blind neuroprosthesis user to recognise people and objects. Extraction of the most useful features of a visual scene is a non-trivial task, and the definition of what is ‘useful’ for a user is strongly context-dependent (e.g. navigation, reading, and social interactions are three very different tasks that require different types of information to be conveyed). Despite rapid advancements in deep learning, it is challenging to develop a general, automated preprocessing strategy that is suitable for use in a variety of contexts. In this recent publication, we present a novel deep learning approach that optimizes the phosphene generation process in an end-to-end fashion. In this approach, both the delivery of stimulation to generate phsophene images (phosphene encoding), as well as the interpretation of these phosphene images (phosphene decoding), are modelled using a deep neural network. The proposed model includes a highly adjustable simulation module of prosthetic vision. All components are trained in a single loop, with the goal of finding an optimally interpretable phosphene encoding which can then be decoded to obtain the original input. In computational validation experiments, we show that such an approach is able to automatically find a task-specific stimulation protocol, which can be tailored to specific constraints, such as stimulation on a sparse subset of electrodes. This approach is highly modular and could be used to dynamically optimize prosthetic vision for everyday tasks and to meet the requirements of the end user.

Jaap de Ruyter van Steveninck, Umut Güçlü, Richard van Wezel, Marcel van Gerven. doi: https://doi.org/10.1101/2020.12.19.423601

We developed a mobile simulator of phosphene vision, to allow the general public to experience how artificially induced phosphene vision would look like for blind users of a visual prosthesis. This setup allows us to evaluate, compare, and optimize different signal processing algorithms that are used to generate phosphene vision, by carrying out tests on individuals with normal vision. In this demo, we show how intelligent algorithms can improve the quality of perception with prosthetic vision with an image processing pipeline that allows for accurate emotion expression recognition.

C. J. M. Bollen, U. Güçlü, R. J. A. van Wezel, M. A. J. van Gerven and Y. Güçlütürk, “Simulating neuroprosthetic vision for emotion recognition,” 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2019, pp. 85-87, https://doi.org/10.1109/ACIIW.2019.8925229