Timbre in the brain

Title: Timbre in the brain
Authors: Vinoo ALLURI, International Institute of Information Technology, Hyderabad
Conference:  Timbre is a Many-Splendored Thing — Research Keynote Lecture, July 6th, 2018.
Source URL: https://www.mcgill.ca/timbre2018/program

Timbre in the Brain

Vinoo Alluri

Cognitive Science Lab, International Institute of Information Technology,

Hyderabad, Telangana, India

Background

Timbre, in its very nature, is abstract. The brain, as we all know, is the most complex system that exists. So, one can only imagine the challenges that can be encountered while investigating timbre processing in the brain. The central focus of timbre research from a neuroscientific point of view, until recently, has been typically on brain responses to changes in acoustic structure or related to sound source categorization (ex: violin vs piano) (Giordano et al., 2014; Halpern, Zatorre, Bouffard, & Johnson, 2004; Overath, Kumar, von Kriegstein, & Griffiths, 2008). While these studies have successfully identified brain regions participating in processing of individual feature manipulations or categorization, they have relied on the controlled auditory paradigm, wherein the sounds are typically presented in isolation and/or synthesized and manipulated artificially. Thus, as a result, such settings paint but only an incomplete picture of the neural underpinnings of the perceptual processes involved in timbre processing, as the results do not necessarily translate to how the human brain processes, in parallel, the multitude of musical features when listening to, for example, a recording of real orchestral music. The aim of the keynote is to present recent neural studies that have attempted to address timbre perception in more ecological settings and touch upon the reliability and validity of the results, and also to highlight the challenges encountered in such settings and future directions.

Timbre in the naturalistic paradigm

Studying music listening as a continuous process using naturalistic stimuli could provide more accurate accounts of the processing of individual musical features in the brain. The novel naturalistic paradigm is one such setting that is more ecological than hitherto encountered settings, both stimulus-wise and task-wise, as participants freely listen to real music without performing any other task while their brain activity is being recorded. This approach of combining functional Magnetic Resonance Imaging (fMRI), Music Information Retrieval (MIR), and multivariate analysis, was first introduced by Alluri et al., (2012) and has recently been acknowledged as a new trend in the field of neuromusicology (Neuhaus, 2017). The results revealed that timbre processing, in addition to auditory regions, was associated with large-scale brain networks in musicians. These regions particularly encompassed cognitive regions previously linked with processing cognitive and attentional load. Additionally, the recruitment of the somatomotor areas of the brain also indicates that listening to natural timbres triggers action- perception mechanisms in musicians' brains. These results reflect the dual representative nature of timbre in the brain, as acoustic structure being processed in the auditory cortices, and as sound-source identification (as reflected by mirror neuron regions and those related to attention).

A fundamental problem that plagues the field of neuroscience is replicability and validity of the results, more so when novel paradigms and methods are introduced. A subsequent replication study performed by the same group revealed high replicability of the neural correlates of timbre pertaining to auditory areas (Burunat et al., 2016). Interestingly, increasing sample size of participants revealed more similar large-scale patterns in both musicians and non-musicians (Alluri et al., in prep). To test the generalizability of the results, a similar approach was used to build encoding models, wherein timbral features were used to predict brain activity and the robustness of the created models were tested across different stimuli (Alluri et al., 2013). As a result, activations in brain regions that were associated with timbre processing were predicted to a significant degree with different stimulus sets and varying participant pools. Decoding studies, which aim to predict stimulus features from brain responses, also revealed that timbral features could be decoded more accurately than the rhythmic and tonal ones (Toiviainen, Alluri, Brattico, Wallentin, & Vuust, 2014). However, several challenges remain concerning the underlying perceptual dimensions of timbre, and their acoustic correlates in such naturalistic settings, let alone methodological challenges in the novel paradigm. The talk is aimed at addressing some of these issues and open avenues of looking at timbre in a holistic sense.

Conclusions

Timbre processing, in the context of music, takes a different identity than hitherto put forth definitions in previous experimental settings. Timbre contributes to the putative emotional content of music (Brattico, Brattico, & Vuust, 2017), which is one of the fundamental driving forces behind the field of cognitive neurosciences of music. Hence, now is the time, due to technological and methodological advances in understanding human perception, to revisit timbre with rigour.

Acknowledgments

I would like to thank Prof. Stephen McAdams for setting the standard in research that one aims at achieving and his encouragement. I extend my sincere gratitude to Prof. Elvira Brattico for showing me the door to neuroscience. And, last but not the least, Prof. Petri Toiviainen, my beacon, without whom this wonderful journey would not have been possible, and for providing never-ending support both professionally and personally.

References

Alluri, V., Toiviainen, P., Jääskeläinen, I. P., Glerean, E., Sams, M., & Brattico, E. (2012). Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm. NeuroImage, 59(4), 3677–3689. http://doi.org/10.1016/j.neuroimage.2011.11.019

Alluri, V., Toiviainen, P., Lund, T. E., Wallentin, M., Vuust, P., Nandi, A. K., ... Brattico, E. (2013). From Vivaldi to Beatles and back: predicting lateralized brain responses to music. NeuroImage, 83, 627–36. http://doi.org/10.1016/j.neuroimage.2013.06.064

Brattico, P., Brattico, E., & Vuust, P. (2017). Global sensory qualities and aesthetic experience in music. Frontiers in Neuroscience, 11(APR), 1–13. http://doi.org/10.3389/fnins.2017.00159

Burunat, I., Toiviainen, P., Alluri, V., Bogert, B., Ristaniemi, T., Sams, M., & Brattico, E. (2016). The reliability of continuous brain responses during naturalistic listening to music. NeuroImage, 124, 224–231. http://doi.org/10.1016/j.neuroimage.2015.09.005

Giordano, B. L., Pernet, C., Charest, I., Belizaire, G., Zatorre, R. J., & Belin, P. (2014). Automatic domain-general processing of sound source identity in the left posterior middle frontal gyrus. Cortex, 58, 170–185. http://doi.org/10.1016/j.cortex.2014.06.005

Halpern, A. R., Zatorre, R. J., Bouffard, M., & Johnson, J. A. (2004). Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia, 42(9), 1281–1292. http://doi.org/10.1016/j.neuropsychologia.2003.12.017

Neuhaus, C. (2017). Methods in neuromusicology: Principles, trends, examples and the pros and cons. In A. Schneider (Ed.), Studies in Musical Acoustics and Psychoacoustics (pp. 341-374). Cham, Switzerland: Springer Nature. http://doi.org/10.1007/978-3-319-47292-8

Overath, T., Kumar, S., von Kriegstein, K., & Griffiths, T. D. (2008). Encoding of Spectral Correlation over Time in Auditory Cortex. Journal of Neuroscience, 28(49), 13268–13273. http://doi.org/10.1523/JNEUROSCI.4596-08.2008

Toiviainen, P., Alluri, V., Brattico, E., Wallentin, M., & Vuust, P. (2014). Capturing the musical brain with Lasso: Dynamic decoding of musical features from fMRI data. NeuroImage, 88, 170–180.