Research Experience

My primary research interest is in the field of Applied Machine Learning & Deep Learning with focus on Speech Recognition, Natural Language Processing, Human Computer Interaction, Conversational Agents (Chatbot), Computer Vision and Internet of Things.

Publications

mTOVA: A Multilingual Task Orientented Virtual Assistant for Human Computer Communication.

The 5th IEEE International Conference on Telecommunications and Photonics (ICTP) 2023

DOI: 10.1109/ICTP60248.2023.10490454

Authors: Sabbir Hossain Ujjal *, A F M Mahfuzul Kabir*, Dr. Mohammad Ariful Haque [* Equal Contribution]

Abstract: A task-oriented virtual assistant (VA) refers to an artificial intelligence-driven system that can assist users to perform daily activities. The utilisation of deep learning algorithms has enabled VAs to attain noteworthy advancements in high-resource languages, such as English. However, languages with limited resources, such as Bengali, have not encountered significant advancements in this domain. In this paper, we propose a multilingual task oriented voice-to-voice conversational agent, which is capable of proficiently managing diverse tasks such as weather forecast, date & time query, hospital and blood bank search etc in both Bengali and English language. Our developed system can understand voice command using Automatic Speech Recognition (ASR) and Natural Language Understanding Unit (NLU). Then the system generates an appropriate reply by gathering information from the internet via APIs, data retrieval techniques and by employing dialogue management. Finally, Natural Language Generator (NLG) and Text to Speech (TTS) techniques are used to construct and deliver proper response. We integrated all the units using RASA framework and python script. Our developed ASR system has an average word error rate of 13% and NLU system has an intent and entity extraction accuracy of 93% and 96.2% respectively. The overall action prediction accuracy of our developed system is 99.4%

Read Pre-Print

Research Experience

Undergraduate Thesis

Title: Development of an end-to-end voice controlled multilingual conversational agent using deep learning and natural language processing.

Principal Investigator: Dr. Mohammad Ariful Haque (Professor, Dept. of EEE, BUET)

Brief summary: The primary objective of this research endeavor is to design a voice-controlled conversational agent capable of operating in multiple languages, with a specific focus on Bengali and English. This entails the utilization of deep learning methodologies, including semi-supervised learning and transformer-based natural language processing techniques. The resulting system empowers users to interact through voice commands and receive automated spoken responses, thereby streamlining task-oriented communication.

Publications: mTOVA: A Multilingual Task Orientented Virtual Assistant for Human Computer Communication.

The 5th IEEE International Conference on Telecommunications and Photonics (ICTP)

Authors:

Sabbir Hossain Ujjal *, A F M Mahfuzul Kabir*, Dr. Mohammad Ariful Haque

[* Equal Contribution]

Demonstration: