To navigate the daily tasks, our brains have evolved efficient and effective mechanisms to allow us to (1) focus on selected feature and locations, and (2) ignore the unselected information. At the same time, we also use various compensation systems to keep our visual world stable and consistent although we move our eyes and bodies frequently. We study how our attentional and perceptual systems operate with psychophysical method, neurophysiological systems (e.g. SSVEP, EEG, fMRI), and computational modelling.

We used steady-state visual evoked potential (SSVEP) to discover that visual attention can precisely adjust its focus based on spatial size tuning.

During saccadic eye movements, human observers are insensitive to object displacement, termed Saccadic Suppression of Displacement (SSD). We reported an analogy case in which auditory blanking facilitates the recovery of the saccadic suppression.

We build conceptual models and designed psychophysical experiments to test these models. The results are used to guide physiological experiment design to understanding underlying neural mechanisms.

Publications

• Chen, G., Hatori, Y., Tseng, C.H., & Shioiri, S. (2025). Adaptive focus: Investigating size tuning in visual attention using SSVEP. Journal of Vision, 25, 6, 1-8.
• Chow, D., Ma, JL., Shioiri, S., Tseng, C.H. (2025). Audio blanking effect of saccadic suppression of target displacement. Vision Science Society Meeting, Florida, USA.

---

We study how body motion cues are perceived and can be used to facilitate the design of the next-generation communication technology. DIEMA-A is the first diverse, intercultural, and high-quality database that contains bodily expression of emotion and affective states. We are able to investigate how contextual information modulates our expression and interpretation of body languages, with human studies, brain imaging experiments, and AI machine learning.

We collected data at (A) Japan and (B) Taiwan to provide inter-cultural contexts for researchers interested in cultural modulations. Professional performers use their full body to act their self-chosen scenario, providing a rich database for the research community.

We reduced the motion videos into dynamic displays (biological motion) so that participants are not affected by environmental cues unrelated to motion (e.g. lab space, gender, etc). We conducted online experiments to evaluate the database and to examine cultural differences.

We conducted a fMRI study and found stronger activation in the bilateral posterior insula and lingual gyrus, regions associated with interoception, empathy, and body-based semantic processing. The bilateral superior temporal sulcus, a region involved in social perception and interpreting social interactions conveyed through BM, was also an area for further exploration.

We attempted a deep learning algorithm using Spatio-Temporal Graphic Convolution Networks (ST-GCN) to recognize human emotions from motion capture data. Our model simulates human observers’ difficulty in recognizing “Disgust” but did not have the human bias to go for “Neutral” when uncertainty arises. Our goal is to use AI to automatically understand and generate body motions for applications in entertainments and communication tools.

Publications

• Miao C, Tseng, C.H., Fujiwara, K., Schneider, V., Kitamura, Y. (2025). Asian Emotional Body Movement Database: Diverse Intercultural E-Motion Database of Asian Performers (DIEM-A). 13th International Conference on Affective Computing and Intelligent Interaction, Canberra, Australia.
• Tanaka, K., Tseng, C.H., Cheng, M., Murai, Y., Sun, S., (2025) Reading the Body: Neural Representations of Emotion and Social Meaning in Naturalistic Human Motion, Annual Meeting of Vision Science Society, Florida, U.S.A.
• Cheng, M., Dumay, D., Higashiyama, S., Tseng, C.H., Komura, T., Kitamura, Y. (2024) Decoding Emotion from Motion: A Comparison of Spatio-Temporal Graph Convolutional Networks (ST-GCN) and Human Accuracy, Asia Pacific Conference on Vision (APCV), Singapore.

---

Walking is one of our most natural daily actions. We discovered that walkers use step synchronization as a form of non-verbal social communication. Walking side-by-side, even without verbal communication, is sufficient to alter the social relation between two strangers. We also discovered that pairs with a better first impression had greater synchronization in their steps. In addition to social relation, personal traits are also important. Female pairs, compared to male pairs, exhibited higher walking synchrony in this experiment. There is also an age effect – older participants tend to synchronize with their partners more in walking. Participants with lower autistic tendencies synchronize better than pairs of higher autistic tendencies.

Publications

• Cheng, M., Kato, M., and Tseng, C.H., (2017) Gender and autistic traits modulate implicit motor synchrony. PLOS One, 10(5).
• Cheng, M., Kato, M., Saunders, J., Tseng, C.H. (2020). Paired walkers with better first impression synchronize better, PLOS One, 15(2), e0227880

---

Team flow happens when team members get “in the zone” to accomplish a task together. Great teams experience this psychological phenomenon, from athletes to musicians and even professional work teams. When teamwork reaches the team flow level, one can observe the team perform in harmony, surpassing their limits. We used an EEG to measure the brain activity from 10 teams while they played a music video game together. It enables us to discover team flow’s unique signature: increased beta and gamma brain waves in the middle temporal cortex, a type of brain activity linked to information processing. Teammates also had more synchronized brain activity during the team flow state compared to the regular teamwork state.

Publications

• Shehata, M., Cheng, M., Leung, A., Tsuchiya, N., Wu, D. A., Tseng, C. H., ... & Shimojo, S. (2021). Team Flow Is a Unique Brain State Associated with Enhanced Information Integration and Interbrain Synchrony. Eneuro, 8(5).

---

Social communication is vital for our well-being and team productivity. The recent outbreak of COVID-19 has made online sociality a significant and central part of our life. We now use online systems for various kinds of business and leisure activities (e.g. classes, work meetings, social parties, etc). However, for many people, online communication has more barriers and is less satisfactory than in-person communication. One possible reason is that it is harder to feel “together,” i.e. ittaikan (一体感), with other people online, and it is harder to form “oneness” for an online group/team. We combine psychophysical behavioral experiments and machine learning techniques in order to decode the psychological states of individual members (e.g. engaged, help-needed) and the team (e.g. togetherness, co-presence).

We propose to see MA as the tension network which holds the bars together and apart at the same time. The black bars represent different categories of events, temporal moments, spatial location, or subjectivity. MA helps to keep the configuration integrity by balancing the pushes and pulls in this structure, which generates a holistic experience from holding the differences together.

Publications

• Wang, G. Y., Nagata, H., Hatori, Y., Sato, Y., Tseng, C. H., & Shioiri, S. (2023). Detecting Learners' Hint Inquiry Behaviors in e-Learning Environment by using Facial Expressions. In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 331-335).
• Tseng, C.H., Cheng, M., Matout, H., Fujita, K., Kitamura, Y., Shioiri, S., Bachrach, A. (2021), MA and togetherness (ittaikan) in the narratives of dancers and spectators: sharing an uncertain space, Japanese Psychological Research, 63(4), 421-433.

---

Infants experience rapid growth and prolific brain development, which becomes critical foundations for later social and cognitive functions. We established the first infant research group in the Tohoku Region via interdisciplinary research strengths available at Tohoku University. Our core members from psychological development, neuro-linguistics, and public health address questions about infant development and their implications later in life. The ultimate goal is to provide evidence-base suggestions to inform better public health policies.

Our ongoing topics include infant learning and its relationship with infant sleep. We use objective experiments and subjective parents’ report to capture the pre-verbal infants’ sleep characteristics.

We examine infants’ social preferences via an eye-tracker, which enables us to measure the baby’s gaze patterns.

We attach a sleep watch measuring sleep-wake cycle to the infants’ wrist or ankle (Micro Motionlogger) by recording the time-series data of physical activity levels and ambient light.

We create visual-audio rule sequences and present to infants repetitively until they have learned it (indicated by their reduced looking time). The relation between visual-audio can be (1) arbitrary, (2) related in object level, or (3) relevant in semantic level (e.g. emotion).

Publications

• Tsui, Ma, Ho, Chow, and Tseng. (2016) Bimodal emotion congruency is critical to preverbal infants’ abstract rule learning. Developmental Science, 1-12.
• Tseng, Chow, Ma, and Ding. (2018) Preverbal infants utilize cross-modal semantic congruency in artificial grammar acquisition, Scientific Report, 8:12707.
• Chow, Ma, Tseng. (2024). Social and Communicative not a Pre-requisite: Preverbal Infants Learn an Abstract Rule only from Congruent audio-visual dynamic pitch-height patterns. Journal of Experimental Child Psych., 248, 106046.

---

An increase online activities is no doubt the most profound behavioral change during the Covid-19. In the cyber world, we quickly establish the group social identities (e.g. the partner, co-workers, boss, client) with minimum information (e.g. online ID, photo/image only). We discovered a “partner-advantage” which refers to the higher processing priority endowed to events or stimuli associated with our partners compared to that to strangers. We are expanding it from human-human to human-robot and human-avatar in order to characterize the social relationships between human beings and other agents.

Publications

• Tseng, C. H., Jingling, L., & Cheng, M. (2022). Social affiliation is sufficient to provoke the partner-advantage. Scientific Reports, 12(1), 21293
• Cheng, M., Tseng, C.H. (2019). Saliency at first sight: instant identity referential advantage toward a newly-met partner. Cognitive Research: Principles and Implications, 4(1), 1-18.

---

Our ability to tell where the true vertical (i.e. verticality perception) is critical because it secures our safe interaction with the world (e.g. holding a cup of coffee without spilling it) and our posture maintenance. This ability is supported by multiple sense: visual, auditory, tactile, proprioceptive, and vestibular systems. We identified a new illusion in the natural environment where this ability is compromised – that is when our body pitch and body motion occur at the same time. We designed experiments to look for the sources of errors. The results indicated that this illusion is not from a single sensory system, but came at the sensory integration stage. When there is a conflict between multiple senses, our brain tries to make sense and thus an illusion. This sensory conflict in VR has been a challenge to create comforts, and this series of study has a potential to help solve the fundamental issue of VR discomfort.

Publications

• Tseng, Chow, and Spillman. (2013) Falling skyscrapers: when cross-modal perception of verticality fails. Psychological Science, 24(7), 1341-7.
• Tseng, Chow, Spillmann, Oxner, & Sakurai. (2022). Body Pitch Together with Translational Body Motion Biases the Subjective Haptic Vertical. Multisensory Research, 36(1), 1-29.

---