HCI-BCI RESEARCH @ IIT KHARAGPUR

From imagination to impact …

View related publications

Ongoing Projects

Brain Computing Interface Augmented Human Computer Interaction

Human brain is one of the most wondrous organs that distinguish humans from all of other organisms. The brain does not just control the organs, but also can think and remember. This ability to feel, adapt, reason, remember and communicate makes human a social being.

Typically, people with disabilities have limited opportunities to socialize and pursue social and leisure activities that most people enjoy. Around 15% of the world’s population lives with some form of disability, and this requires a call to change, to empower people with disabilities. With the recent advancement in brain-computer interface (BCI) technology, researchers are exploring if BCI can be augmented with human computer interaction (HCI) to give a new hope of restoring independence to disabled individuals. Hence, we propose to work with a fourth generation hands-free or touch-free text-entry system based on the neural potentials, for the task of communication for people with less mobility. Here, the BCI system translates the intentions of the user, left and right hand motor imagery signals, captured using an EEG device into device commands with no involvement of peripherals. The BCI system also helps us to analyse cognitive loads endured by the user and allow us to determine, what can be an ideal text-entry mechanism or what should the graphical user interface (GUI) be like so that the user in question faces lesser pressure, or may even enjoy the task of composing texts. The use of BCI augmented HCI for a text-entry mechanism will become even more paramount for people with limited mobility.

Motor Imagery based BCI system

Read more on this topic

Physiological Measurement and Human Performance Monitoring

Human brain is a central coordinating system and it has been thoroughly studied about its working mechanism. With the advancement and mellowness of new edge information technology (IT), it is now possible to amalgam the brain and IT in many application areas, other than in medical domain.

One such promising application area is the brain computing interface. With such technology in hand, it has become easy to monitor human behaviour before assigning them to critical tasks. Effective prevention of dangerous events is possible if a risk assessment process is considered in advance, particularly the role of man in accident dynamics. Out of several factors influencing human errors, two of them are of prime importance are: vigilance and mental workload. Vigilance or sustained attention is an act of careful observation of critical or rare events whose negligence may lead to some undesirable circumstances. In this regard, we propose to use Event Related Potentials (ERPs) of EEG signals to indicate level of vigilance. Mental Workload, on the other hand, is the demand placed on humans for mental resources while performing a task. One way to increase the performance of the operator is to constantly monitor their performance and then to take a strategy to avoid risk due to human resources. This influences the practical ramifications in many automated systems designed to forewarn monitors of impending, untoward events in the work world. Further, it would be more effective, if it can be accomplished in ground situation when operators are in operations as an instantaneous, online process without any uncomfortable setup that affects operators' normal operation.

Overview of human performance monitoring system

Read more on this topic

Adaptive E-learning System

Adaptive e-learning is an educational method which uses computers as interactive teaching devices. Computers adapt the presentation of educational material according to students' cognitive state and learning needs, as indicated by their responses to questions and cognitive behaviours during learning.

Existing approaches assumes student’s cognitive state by analysing their response to tasks and activities during learning. If the e-learning system can comprehend student’s cognitive state precisely by considering their brain signals while learning, then it can make a significant improvement in building an adaptive e-learning system. In the direction of satisfying this goal, our research objective is to augment Brain Computing Interface (BCI) with an e-learning system to make it as an adaptive e-learning system. To achieve this task, the common BCI steps such as channel selection, feature extraction, feature selection and translation algorithm has to be processed to measure the student’s cognitive state. Finally, depending on the cognitive load level of the student, the e-course content has to be changed or revised, so that the user can enjoy his learning.

Adaptive e-learning using BCI

Read more on this topic

Research Data: Data Collection Protocols

Protocol followed for three-class motor imagery EEG data collection

Experimental Design

The experiments were conducted in an electrically isolated HCI-BCI laboratory at IIT Kharagpur. Users were seated in a chair with their arms and legs extended, resting on a desk and resting on a footrest respectively. We made sure that there was no background noise while the data was recorded. Subjects were asked to avoid unnecessary movements to minimise artifacts. We used a 64-channel EEG device, a product from RMS-India, to acquire motor imagery data. We used two systems, one to display the below mentioned experimental interface and the other to collect motor imagery EEG signals.

  • The subjects were asked to perform imagery left hand, right hand and tongue according to a cue displayed on the screen.
  • Each experiment includes several runs (at least 2) and trials (at least 20).
  • Each trial includes the following order. The first 3s of the procedure will be quite and at t=3s an acoustic stimulus and a cross “+” will be displayed to indicate the beginning of trial.
  • Then at t=4s an arrow to the left, right, or sphere in the centre will be displayed for 1s and at the same time the subject will be asked to imagine a left hand, right hand, and tongue movements respectively, till the cross appears on the screen.
  • The cross "+" disappears at t=8s. Therefore, each trial takes 8s totally. Each of three cues will be displayed 20 times for each run in a randomised order.
  • There was 5-10 minute gap between each run.
Cues displayed to collect motor imagery data.
Metadata:
  • No of Subjects: 5 (3 male and 2 female)
  • Age range: 26-32

For each subject, we have noted down their last night sleep duration. Besides, they were also enquired about their recent illness and medicine consumptions. Each run took about 10-15 minutes. The entire session took around 30-40 minutes. After the session, we asked the subjects to fill out a form consisting of few questions such as could they imagine moving their hands and tongue according to the cue appeared, do they feel irritated on doing the experiment, do they feel drowsy, etc.

Subjects Gender Age Session Sleeping hours Medicine Consumption
S1 M 28 FN ~8 hours No
S2 F 29 FN ~6 hours No
S3 F 31 AN 8 hours No
S4 M 26 AN 6 hours No
S5 M 32 FN ~7 hours No

Read more on this topic

Protocol followed for EEG data collection to determine vigilance

Experimental Design

The experiment was carried out in a quiet and isolated room with maintained room temperature, where each participant was seated comfortably. A large 20-inch monitor was placed approximately 65cm away from the participants for presenting visual stimuli. Initially, each participant was asked to relax for duration of ten minutes. Thereafter, in the next five minutes we asked participants to fill subjective questionnaires. After this, a five minutes’ instruction, demonstration and practice session was arranged for each participant, which was followed by the actual EEG data recording of 20 minutes. During the data recording the participants were asked to avoid, as far as possible, movement of any kind except for responding to the target stimuli. This experiment was conducted once with each participant. Moreover, only a single participant’s data was recorded in a day. The data recording was done between 7:00 AM - 10:00 AM as per the availability of the participants.

Timeline

Vigilance Task: We used the popular Mackworth Clock Test for the vigilance detection experiment. In this task, the task of the participant is to monitor a red pointer moving in front of a black background in steps like the seconds’ hand of a clock and responds when the clock randomly skips a beat. In a session, whenever the clock skips a beat, the user has to promptly respond by pressing the "space bar" key.

Metadata:
  • No of Subjects: 10 (8 male and 2 female)
  • Age range: 24-32

Participants comprising of research scholars and post-graduate students available on campus at IIT Kharagpur were randomly selected for this study. The participants’ were having normal intelligence with no history of mental. Each participant had normal or corrected-to-normal vision and was right handed. Besides, the participants were not sleep deprived; they had no deviations from their usual circadian cycle, and they took no medicine or alcohol. They were asked to refrain from having tea or coffee three hours before the experiment. Informed consent from all participants was taken before conducting the experiment. Appropriate certificate of approval was also obtained from the Institutional Ethical committee at IIT Kharagpur.

Subject Gender Age Sleeping hours Medicine Intake Tea/ Coffee Consumption
S1 M 28 ~8 hours No No
S2 F 29 ~6 hours No No
S3 F 31 8 hours No No
S4 M 26 6 hours No No
S5 M 32 ~7 hours No No
S6 M 27 10 hours No No
S7 M 28 7 hours No No
S8 M 30 5 hours No No
S9 M 32 8 hours No No
S10 M 24 6 hours No No

Read more on this topic

Data acquisition of three-class Cognitive load data for Adaptive E-learning

Experimental Design

Our main objective is to build Adaptive E-learning using BCI. The E-learning system should adapt itself based on user’s cognitive load, which will be measured from his or her brain signal. So we collect our cognitive load data from different subjects during their reading of different readability level texts. Images given below are used texts for one trial of data collection. Level 1 text has high readability index, so data collected during reading of this text is considered as low level cognitive load data. Same way, data collected during reading of level 2 and level 3 texts are considered as medium level and high level cognitive load data respectively.

Adaptive e-learning level 1 Adaptive e-learning level 2Adaptive e-learning level 3

Experimental Setup

The experiments were conducted in controlled environment in an electrically isolated BCI laboratory at IIT Kharagpur. We used R.M.S. EEG device and its 64 channel brain cap for this data collection. To minimize any of the artifacts generated, the subjects were asked to avoid any unnecessary physical movements and their hands were placed in a fixed position. Subjects were requested to refrain from excessive blinking of eye lid.

  1. Each subject read a paragraph written in English language, appearing on the screen in front of him or her.
  2. This experiment has included 6 trials for each subject.
  3. Each trial included the following order. The first 6s of the procedure was quite, subject did nothing and at t=7s one paragraph has been displayed and subject started reading. This indicates the beginning of trial. The paragraph has been displayed for 1 minute.
  4. After this, two more paragraphs of different level loads have been appeared in the screen for 1 minute each.
  5. Totally, after 3minutes and 7s screen was black and this indicates the end of trial.
  6. Same way one subject has gone through 6 trials.
  7. There was a 10 minute gap between each trial. The total time taken for each trial is given in the table below. Data
  8. Total time taken for each session that is for each subject is 1 hour 30 minutes.
Metadata:

Eight healthy male and three healthy female volunteers engaged in graduate study participated in the experiment with age ranging between 24 to 29 years. All subjects had normal or corrected to normal eye-sight. All subjects were given adequate practice and idea of the experiments. Ethical Clearance was taken for collecting EEG data from them. Room temperature was controlled to ensure adequate comfort during the experiments.

Participants Gender Age Medically fit? Time of data collection
S1 M 26 Yes FN
S2 M 27 Yes AN
S3 M 29 Yes FN
S4 M 24 Yes FN
S5 M 26 Yes AN
S7 M 29 Yes FN
S8 M 31 Yes FN
S9 F 27 Yes FN
S10 F 29 Yes FN
S11 F 28 Yes AN

Read more on this topic

Data Repository

Related to different directions of BCI-HCI reserach, a large amount of data has been collected in our BCI-HCI Lab, IIT Kharagpur. Further, the implementation of some of our work is published now for the benefits of budding reserachers in the area. Please click for Data and Code.

Research Team

Dr. Debasis Samanta

Dr. Debasis Samanta
Associate Professor
Department of Computer Science & Engineering
IIT Kharagpur

Dr. Pabitra Mitra

Dr. Pabitra Mitra
Associate Professor
Department of Computer Science & Engineering
IIT Kharagpur

Dr. Monalisa Sarma

Dr. Monalisa Sarma
Assistant Professor
Subir Chowdhury School of Quality and Reliability
IIT Kharagpur

Research Scholars

S. R. Sreeja

S. R. Sreeja
Department of Computer Science & Engineering, IIT Kharagpur
Topic: Text Entry System Development using Motor Imagery EEG Signals
Supervisors: Dr. Debasis Samanta, Dr. Pabitra Mitra

Shabnam Samima

Shabnam Samima
Subir Chowdhury School of Quality and Reliability, IIT Kharagpur
Topic: Human Performance Prediction using Brain Computing Interfaces
Supervisor: Dr. Monalisa Sarma

Anushri Saha

Anushri Saha
Department of Computer Science & Engineering, IIT Kharagpur
Topic: Adaptive E-learning System using Brain Computing Interface
Supervisors: Dr. Debasis Samanta

Post-Graduate Student

Prachi Agarwal

Prachi Agarwal
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.

Lt. Col. Baljeet Singh Cheema

Lt. Col. Baljeet Singh Cheema
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.

Joytirmoy Rabha

Joytirmoy Rabha
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.

Under-Graduate Students

Arnab Bir

Arnab Bir
Department: Mathematics
Year: Fourth
Area of Research: BCI/HCI, Machine Learning.

Vikash

Vikash
Department: Computer Science & Engineering
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.

Sanjit

Sanjit
Department of Computer Science & Engineering, IIT Kharagpur
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.

Nagarjuna

Nagarjuna
Department of Computer Science & Engineering, IIT Kharagpur
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.