Although Fernanda De La Torre still has a few years left in her graduate studies, she is already dreaming big about what the future holds.
“I dream of opening a school one day where I could bring this world of understanding of cognition and perception to places that would never have contact with it,” he says.
It’s this kind of ambitious thinking that has brought De La Torre, a doctoral student in MIT’s Department of Brain and Cognitive Sciences, to this point. A recent recipient of the prestigious Paul and Daisy Soros Fellowship for New Americans, De La Torre has found MIT a supportive and creative research environment that has allowed her to delve into the cutting-edge science of artificial intelligence. But she is still driven by an innate curiosity about the human imagination and a desire to bring that knowledge to the communities in which she grew up.
An unconventional path to neuroscience
De La Torre’s first exposure to neuroscience was not in the classroom, but in his everyday life. As a child, she watched her younger sister struggle with epilepsy. At age 12, she crossed the United States illegally from Mexico to reunite with her mother, exposing her to a whole new language and culture. Once in the United States, he had to contend with his mother’s changing personality in the midst of an abusive relationship. “All these different things I was seeing around me made me want to better understand how psychology works,” says De La Torre, “to understand how the mind works and how it is that we can all be in the same environment and feel things very different.”
But finding an outlet for this intellectual curiosity was a challenge. As an undocumented immigrant, his access to financial aid was limited. His high school was also underfunded and lacked electives. Mentors along the way, however, encouraged the aspiring scientist, and through a program at his school, he was able to take community college courses to meet basic education requirements.
It took inspiring dedication to his education, but De La Torre made it to Kansas State University for his undergraduate studies, where he majored in computer science and mathematics. At Kansas State, he was able to get his first real taste of research. “I was fascinated by the questions they were asking me and all this space I hadn’t found,” De La Torre says of her experience working in a visual cognition lab and discovering the field of computational neuroscience.
Although Kansas State did not have a dedicated neuroscience program, her research experience in cognition led her to a machine learning lab led by William Hsu, a professor of computer science. There, De La Torre fell in love with the possibilities of using computing to model the human brain. Hsu’s support also convinced her that a career in science was a possibility. “He always made me feel like I was capable of tackling big questions,” he says fondly.
With the confidence passed down to him at Kansas State, De La Torre came to MIT in 2019 as a post-baccalaureate student in the lab of Tomaso Poggio, the Eugene McDermott Professor of Brain and Cognitive Sciences and a McGovern Scholar Institute for Brain Research. . With Poggio, also director of the Center for Brains, Minds and Machines, De La Torre began work on deep learning theory, an area of machine learning focused on how artificial neural networks modeled on the brain can learn to recognize patterns and learn.
“It’s a very interesting question because we’re starting to use them everywhere,” says De La Torre of neural networks, listing examples from self-driving cars to medicine. “But at the same time, we don’t fully understand how these networks can go from knowing nothing and just being a bunch of numbers to producing things that make sense.”
His post-bac experience was De La Torre’s first real opportunity to apply the technical computer skills he developed as an undergraduate to neuroscience. It was also the first time he could fully focus on research. “This was the first time I had access to health insurance and a stable salary. That in itself was kind of life-changing,” he says. “But in terms of the research, it was very intimidating at first. I was anxious and not sure I belonged here.”
Fortunately, De La Torre says he was able to overcome these insecurities, both through a growing unabashed enthusiasm for the field and the support of Poggio and his other colleagues in MIT’s Department of Brain and Cognitive Sciences. When the opportunity presented itself to apply to the department’s PhD program, he jumped at it. “It was just knowing that these kinds of mentors are here and that they care about their students,” De La Torre says of her decision to stay at MIT for graduate school. “That was really meaningful.”
Expanding notions of reality and imagination
In his two years so far in the graduate program, De La Torre’s work has expanded the understanding of neural networks and their applications to the study of the human brain. Working with Guangyu Robert Yang, a research associate at the McGovern Institute and assistant professor in the departments of Brain and Cognitive Sciences and Electrical Engineering and Computer Science, she is engaged in what she describes as more philosophical questions about how a sense of self as an independent being. She is interested in how this self-awareness develops and why it can be useful.
De La Torre’s main advisor, however, is Professor Josh McDermott, who runs the Computational Hearing Lab. With McDermott, De La Torre is trying to understand how the brain integrates sight and sound. Although combining sensory input may seem like a basic process, there are many unanswered questions about how our brains combine multiple signals into a coherent impression, or perception, of the world. Many of the questions are raised by audiovisual illusions in which what we hear changes what we see. For example, if one watches a video of two discs sliding past each other, but the clip contains the sound of a collision, the brain will perceive the discs as bouncing, rather than passing through each other. Given an ambiguous image, this simple auditory cue is all that is needed to create a different perception of reality.
“There’s something interesting about how our brain is getting two signals that are telling us different things, and yet we have to combine them somehow to make sense of the world,” he says.
De La Torre is using behavioral experiments to investigate how the human brain makes sense of multisensory cues to construct a particular perception. To do this, he has created several scenes of interacting objects in 3D space over different sounds, asking research participants to describe the features of the scene. For example, in one experiment, he combines images of a block moving across a surface at different speeds with various scraping sounds, asking participants to estimate how rough the surface is. Eventually, he hopes to bring the experiment to virtual reality, where participants will physically press the blocks in response to how hard they perceive the surface to be, rather than reporting what they experience.
Once he has collected data, he will move on to the modeling phase of the research, assessing whether multisensory neural networks perceive illusions the way humans do. “What we want to do is model exactly what’s going on,” says De La Torre. “How is it that we are receiving these two signals, integrating them, and at the same time using all our prior knowledge and inferences from physics to really make sense of the world?”
While his two lines of inquiry with Yang and McDermott may seem different, he sees clear connections between the two. Both projects seek to understand what artificial neural networks are capable of and what they tell us about the brain. At a more fundamental level, he says, the way the brain perceives the world from different sensory signals could be part of what gives people a sense of self. Sensory perception consists of constructing a unitary and cohesive sense of the world from multiple sources of sensory data. Similarly, he argues, “the sense of self is really a combination of actions, plans, goals, emotions, all these different things that are their own components, but somehow create a unitary being.”
It’s an apt sentiment for De La Torre, who has been working to make sense of and integrate different aspects of his own life. Working in the computational hearing lab, for example, she has begun experimenting with combining electronic music with folk music from her native Mexico, connecting her “two worlds,” as she puts it. Having the space to undertake these kinds of intellectual explorations, and the colleagues who encourage it, is one of De La Torre’s favorite parts of MIT.
“Beyond the teachers, there are also many students whose way of thinking amazes me,” he says. “I see a lot of kindness and excitement about science and a little bit of, it’s not nerdiness, but a love of very special things, and I love that.”