Maryam Sadat Mirzaei

Interested in

About

Research Scientist @ RIKEN Center of Advanced Intelligence Project

My research interests lie in the field of Natural Language Processing and Human-AI communication. The primary focus of my research is to make use of machine learning methods to understand and analyze the human-human conversation with the aim to design agents that can smoothly communicate with humans. I also actively work in using NLP and automatic speech recognition (ASR) technologies in developing intelligent CALL systems to facilitate teaching and learning foreign languages.

Resume

Education

Ph.D. in Informatics

Mar 2017

Kyoto University, Kyoto, Japan

Title: Partial & Synchronized Caption to Foster Second Language Listening based on Automatic Speech Recognition Clues

Supervisor: Prof. Tatsuya Kawahara

M.Sc. in Informatics

Mar 2014

Kyoto University, Kyoto, Japan

B.A. in Foreign Language Education and Translation Studies

Jan 2009

Teacher Training University, Tehran, Iran

Work Experience

Research Scientist

Feb 2021 - Present

Language Information Access Technology Team, RIKEN AIP, Tokyo, Japan

  • Role: AI-Mediated Human-Human Communication

Post-doc Researcher

Apr 2020 - Feb 2021

Language Information Access Technology Team, RIKEN AIP, Tokyo, Japan

  • Role: AI-Mediated Human-Human Communication

Post-doc Researcher / Guest Research Associate (Kyoto Uni)

Jun 2017 - Mar 2020

Human-AI Communication Team, RIKEN AIP, Kyoto, Japan

  • Role: Conversation Envisioning Project
  • Role: Intelligent Story Creation Project

Research Assistant

Apr 2012 - Mar 2017

Audio & Speech Processing Lab, Kyoto University, Kyoto, Japan

  • Role: Partial and Synchronized Caption project

Research Assistant

Oct 2010 - Mar 2012

Grad. School of Human & Environment Studies, Kyoto University, Kyoto, Japan

  • Role: Theories and practices of developing CALL systems
  • Role: The use of digital games such as MMORPG and virtual worlds in language education technology

English Instructor

Jul 2006 - Sep 2010

Iran Language Institute (ILI), Tehran, Iran

Publications

Peer-Reviewed Articles

Mirzaei, M. S., Meshgi, K., and Sekine, S. (2023) What is the real intention behind this question? Dataset collection and intention classification, ACL’23, Toronto, Canada. (submitted).

Mirzaei, M. S., Meshgi, K. (2022). Self-regulated listening experience with smart captioning, EuroCALL'22, Reykjavík, Iceland.

Mirzaei, M. S., Meshgi, K., and Sekine, S. (2022). Is this Question Real? Dataset Collection on Perceived Intentions and Implicit Attack Detection, WWW'22, Lyon, France.

Meshgi, K., Mirzaei, M. S., and Sekine, S. (2022). Q-Learning Scheduler for Multi Task Learning using Histogram of Task Uncertainty, ACL'22, Rep4NLP workshop, Dublin, Ireland.

Meshgi, K., Mirzaei, M. S., and Sekine, S. (2022). Uncertainty Regularized Multi-Task Learning, ACL'22, WASSA workshop, Dublin, Ireland.

Peterson, M., White, J., Mirzaei, M. S., & Wang, Q. (2022). A review of research on the application of digital games in foreign language education. Research Anthology on Developments in Gamification and Game-Based Learning, 1948-1971.

Mirzaei, M. S., Meshgi, K. (2021). Adaptive Listening Difficulty Detection for L2 Learners Through Moderating ASR Resources, Interspeech'21, Brno, Czechia.

Mirzaei, M. S., Meshgi, K. (2021). Facilitating L2 listening through automatic detection of speech and lexical difficulties, EuroCALL'21, Paris, France.

Mirzaei, M. S., Meshgi, K., and Nishida, T. (2021). Promoting common ground building in L2 cross-cultural conversations, EuroCALL'21, Paris, France.

Meshgi, K., Mirzaei, M. S., and Oba S. (2020). Adversarial Semi-Supervised Multi-Domain Tracking, ACCV'20, Kyoto, Japan.

Meshgi, K., Mirzaei, M. S., and Oba S. (2020). Leveraging Tacit Information Embedded in CNN Layers for Visual Tracking, ACCV'20, Kyoto, Japan.

Mirzaei, M. S., Meshgi, K., and Nishida, T. (2020). Sentence Complexity as an Indicator of L2 Learner's Listening Difficulty, EuroCALL'20, Copenhagen, Denmark.

Mirzaei, M. S., Meshgi, K., and Nishida, T. (2020). A Situation Creation System to Enable Experiential Learning in Virtual Worlds for Developing Cross-Cultural Competencies, EuroCALL'20, Copenhagen, Denmark.

Mirzaei, M. S., Meshgi, K., Frigo, E., and Nishida, T. (2020). AnimGAN: A Spatiotemporally-Conditioned Generative Adversarial Network for Character Animation, ICIP'20, Abu Dhabi, UAE.

Mirzaei, M. S., Zhang, Q., Van der Struijk, S., and Nishida, T. (2019). Towards Conversation Envisioning for Cognitive Robots. In Vallverdú, J., & Müller C.V. (Eds.). Blended Cognition. The Robotic Challenge, Springer Verlag.

Mirzaei, M. S., Meshgi, K., and Nishida, T. (2019). Collaborative Learning through Story Envisioning in Virtual Reality, EuroCALL'19, Louvain-la-Neuve, Belgium.

Mirzaei, M. S., Meshgi, K., and Nishida, T. (2019). Learner-Adaptive Partial and Synchronized Caption for L2 Listening Skill Development, EuroCALL'19, Louvain-la-Neuve, Belgium.

Peterson, M., Mirzaei, M. S., and White, J. (2019). Application of Digital Games in Foreign Language Education: A Review of the Research. New Technological Applications for Foreign and Second Language Learning and Teaching. IGI Global.

Meshgi, K., Mirzaei, M. S., and Sekine, S. (2019). Uncertainty Regularized Multi-Task Learning for Text Classification, ACML'19, WSL Workshop, Nagoya, Japan.

Zhang, Q., Mirzaei, M. S., Huang, H., and Nishida, T. (2019). Story Envisioning Framework for Visualized Collective Storytelling in Conversation. HCII'19, Florida, USA.

Peterson, M., Qiao, W., and Mirzaei M. S. (2019). The Use of Network-Based Virtual Worlds in Second Language Education: A Research Review. Assessing the Effectiveness of Virtual Technologies in Foreign and Second Language Instruction. IGI Global, pp.1-25. DOI: 10.4018/978-1-5225-7286-2.ch00.

Meshgi, K., Mirzaei, M. S., and Oba, S. (2019). Long and Short Memory Balancing in Visual Co-Tracking using Q-Learning, ICIP'19, Taipei, Taiwan.

Mirzaei, M. S., Zhang, Q., Van der Struijk, S., and Nishida, T. (2018). Language learning through conversation envisioning in virtual reality, EuroCALL'18, Jyväskylä, Finland

Mirzaei, M. S., and Meshgi, K. (2018). Automatic Scaffolding for L2 Listeners by Leveraging Natural Language Processing, EuroCALL'18, Jyväskylä, Finland.

Meshgi, K., and Mirzaei, M. S. (2018). A comprehensive word difficulty index for L2 listening. ExLing'18, France, Paris.

Van der Struijk, S., Huang, H., Mirzaei, M. S., and Nishida, T. (2018). FACSvatar: An Open Source Modular Framework for Real-Time FACS based Facial Animation. IVA'18, Sydney, Australia.

Meshgi, K., Mirzaei, M. S., and Oba, S. (2018). Information-Maximizing Sampling to Promote Tracking-by-Detection, ICIP'18, Athens, Greece.

Mirzaei, M. S., Zhang, Q., and Nishida, T. (2018). Conversation Envisioning to Train Inter-Cultural Interactions, HCII'18, Las Vegas, USA.

Mirzaei, M. S., Zhang, Q., Van der Struijk, S., and Nishida, T. (2018). Conversation Envisioning Framework for Situated Conversation. IEA/AIE'18, Montreal, Canada.

Mirzaei, M. S., Meshgi, K., and Kawahara, T. (2018). Exploiting Automatic Speech Recognition Errors to Enhance Partial and Synchronized Caption for Facilitating Second Language Listening. Computer Speech and Language Journal, Elsevier.

Meshgi, K., Mirzaei, M. S., Oba, S., and Ishii, S. (2017). Efficient Asymmetric Co-Tracking using Uncertainty Sampling, ICSIPA'17, Kuching, Malaysia.

Meshgi, K., Mirzaei, M. S., Oba, S. and Ishii, S. (2017) Active Collaborative Ensemble Tracking, AVSS'17, Lecce, Italy.

Mirzaei, M. S., Meshgi, K., and Kawahara, T. (2017). Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill. ReCALL Journal, Cambridge University Press.

Mirzaei, M. S., Meshgi, K., and Kawahara, T. (2017). Listening difficulty detection to foster second language listening with the Partial and Synchronized Caption system. EuroCALL'17, Southampton, U.K.

Mirzaei, M. S., and Kawahara, T. (2017). Detecting listening difficulty for second language learners using Automatic Speech Recognition errors. Interspeech'17, SLaTE workshop, Stockholm, Sweden.

Mirzaei, M. S., Meshgi, K., and Kawahara, T. (2016). Automatic Speech Recognition Errors as a Predictor of L2 Listening Difficulties. Coling'16, CL4LC workshop, Osaka, Japan.

Mirzaei, M. S., Meshgi, K., and Kawahara, T. (2016). Leveraging automatic speech recognition errors to detect challenging speech segments in TED talks. EUROCALL'16, Limmasol, Cyprus.

Mirzaei M. S. (2016). Using Automatic Speech Recognition Technology to Assist L2 Listeners. LET-56, Tokyo, Japan.

Mirzaei, M. S., Meshgi, K., Akita, Y., and Kawahara, T. (2015). Errors in automatic speech recognition versus difficulties in second language listening. EuroCALL'15, Padova, Italy.

Mirzaei, M. S., and Kawahara, T. (2015). ASR Technology to Empower Partial and Synchronized Caption for L2 Listening Development. Interspeech'15, SLaTE workshop, Leipzig, Germany.

Mirzaei, M. S. (2015). Automatic Speech Recognition Errors Enrich Partial and Synchronized Caption to Develop Listening Skill. LET-55 Osaka, Japan.

Mirzaei, M. S., Akita, Y., and Kawahara, T. (2014). Partial and Synchronized Captioning: A New Tool for Second Language Listening Development. EuroCALL'14, Groningen, The Netherlands.

Mirzaei, M. S., Akita, Y., and Kawahara, T. (2014). Partial and Synchronized Caption Generation to Develop Second Language Listening Skill. ICCE'14, NLP-TEA workshop, Nara, Japan.

Mirzaei, M.S., and Kawahara, T. (2014). Listen More, Read Less: Partial and Synchronized Captions to Train L2 Listening using TED Talks. LET-54, Fukouka, Japan.


Talks

Toward Adaptive Partial and Synchronized Caption to Facilitate L2 Listening, in FLEAT'19, Tokyo, Japan.

Virtual reality to advance sociocultural knowledge for language learners, in WorldCALL'18, Concepción, Chile.

A Data-driven Approach to Generate Partial and Synchronized Caption for Second Language Listeners, In WorldCALL'18, Concepción, Chile.

Partial and Synchronized Caption Generation to Enhance the Listening Comprehension Skills of Second Language Learners. In 101 Joint SLP Conf., Tokyo, Japan.

Listen More, Read Less: The Use of Technology in Generating Partial and Aligned Captions for Foreign Language Learners. Kansai Joint Speech Seminar 2013, Nara, Japan.

Contact

Location:

Mitsui Building 15F, 1-4-1 Nihonbashi,Chuo-ku, Tokyo