The main objective of the project is to investigate the application of brain computing technology to find an alternative and better approach for various day-to-day utility. It includes:
Human brain is one of the most wondrous organs that distinguish humans from all of other organisms. The brain does not just control the organs, but also can think and remember. This ability to feel, adapt, reason, remember and communicate makes human a social being.
Typically, people with disabilities have limited opportunities to socialize and pursue social and leisure activities that most people enjoy. Around 15% of the world’s population lives with some form of disability, and this requires a call to change, to empower people with disabilities. With the recent advancement in brain-computer interface (BCI) technology, researchers are exploring if BCI can be augmented with human computer interaction (HCI) to give a new hope of restoring independence to disabled individuals. Hence, we propose to work with a fourth generation hands-free or touch-free text-entry system based on the neural potentials, for the task of communication for people with less mobility. Here, the BCI system translates the intentions of the user, left and right hand motor imagery signals, captured using an EEG device into device commands with no involvement of peripherals. The BCI system also helps us to analyse cognitive loads endured by the user and allow us to determine, what can be an ideal text-entry mechanism or what should the graphical user interface (GUI) be like so that the user in question faces lesser pressure, or may even enjoy the task of composing texts. The use of BCI augmented HCI for a text-entry mechanism will become even more paramount for people with limited mobility.
Human brain is a central coordinating system and it has been thoroughly studied about its working mechanism. With the advancement and mellowness of new edge information technology (IT), it is now possible to amalgam the brain and IT in many application areas, other than in medical domain.
One such promising application area is the brain computing interface. With such technology in hand, it has become easy to monitor human behaviour before assigning them to critical tasks. Effective prevention of dangerous events is possible if a risk assessment process is considered in advance, particularly the role of man in accident dynamics. Out of several factors influencing human errors, two of them are of prime importance are: vigilance and mental workload. Vigilance or sustained attention is an act of careful observation of critical or rare events whose negligence may lead to some undesirable circumstances. In this regard, we propose to use Event Related Potentials (ERPs) of EEG signals to indicate level of vigilance. Mental Workload, on the other hand, is the demand placed on humans for mental resources while performing a task. One way to increase the performance of the operator is to constantly monitor their performance and then to take a strategy to avoid risk due to human resources. This influences the practical ramifications in many automated systems designed to forewarn monitors of impending, untoward events in the work world. Further, it would be more effective, if it can be accomplished in ground situation when operators are in operations as an instantaneous, online process without any uncomfortable setup that affects operators' normal operation.
Adaptive e-learning is an educational method which uses computers as interactive teaching devices. Computers adapt the presentation of educational material according to students' cognitive state and learning needs, as indicated by their responses to questions and cognitive behaviours during learning.
Existing approaches assumes student’s cognitive state by analysing their response to tasks and activities during learning. If the e-learning system can comprehend student’s cognitive state precisely by considering their brain signals while learning, then it can make a significant improvement in building an adaptive e-learning system. In the direction of satisfying this goal, our research objective is to augment Brain Computing Interface (BCI) with an e-learning system to make it as an adaptive e-learning system. To achieve this task, the common BCI steps such as channel selection, feature extraction, feature selection and translation algorithm has to be processed to measure the student’s cognitive state. Finally, depending on the cognitive load level of the student, the e-course content has to be changed or revised, so that the user can enjoy his learning.
The experiments were conducted in an electrically isolated HCI-BCI laboratory at IIT Kharagpur. Users were seated in a chair with their arms and legs extended, resting on a desk and resting on a footrest respectively. We made sure that there was no background noise while the data was recorded. Subjects were asked to avoid unnecessary movements to minimise artifacts. We used a 64-channel EEG device, a product from RMS-India, to acquire motor imagery data. We used two systems, one to display the below mentioned experimental interface and the other to collect motor imagery EEG signals.
For each subject, we have noted down their last night sleep duration. Besides, they were also enquired about their recent illness and medicine consumptions. Each run took about 10-15 minutes. The entire session took around 30-40 minutes. After the session, we asked the subjects to fill out a form consisting of few questions such as could they imagine moving their hands and tongue according to the cue appeared, do they feel irritated on doing the experiment, do they feel drowsy, etc.
Subjects | Gender | Age | Session | Sleeping hours | Medicine Consumption |
---|---|---|---|---|---|
S1 | M | 28 | FN | ~8 hours | No |
S2 | F | 29 | FN | ~6 hours | No |
S3 | F | 31 | AN | 8 hours | No |
S4 | M | 26 | AN | 6 hours | No |
S5 | M | 32 | FN | ~7 hours | No |
The experiment was carried out in a quiet and isolated room with maintained room temperature, where each participant was seated comfortably. A large 20-inch monitor was placed approximately 65cm away from the participants for presenting visual stimuli. Initially, each participant was asked to relax for duration of ten minutes. Thereafter, in the next five minutes we asked participants to fill subjective questionnaires. After this, a five minutes’ instruction, demonstration and practice session was arranged for each participant, which was followed by the actual EEG data recording of 20 minutes. During the data recording the participants were asked to avoid, as far as possible, movement of any kind except for responding to the target stimuli. This experiment was conducted once with each participant. Moreover, only a single participant’s data was recorded in a day. The data recording was done between 7:00 AM - 10:00 AM as per the availability of the participants.
Vigilance Task: We used the popular Mackworth Clock Test for the vigilance detection experiment. In this task, the task of the participant is to monitor a red pointer moving in front of a black background in steps like the seconds’ hand of a clock and responds when the clock randomly skips a beat. In a session, whenever the clock skips a beat, the user has to promptly respond by pressing the "space bar" key.
Participants comprising of research scholars and post-graduate students available on campus at IIT Kharagpur were randomly selected for this study. The participants’ were having normal intelligence with no history of mental. Each participant had normal or corrected-to-normal vision and was right handed. Besides, the participants were not sleep deprived; they had no deviations from their usual circadian cycle, and they took no medicine or alcohol. They were asked to refrain from having tea or coffee three hours before the experiment. Informed consent from all participants was taken before conducting the experiment. Appropriate certificate of approval was also obtained from the Institutional Ethical committee at IIT Kharagpur.
Subject | Gender | Age | Sleeping hours | Medicine Intake | Tea/ Coffee Consumption |
---|---|---|---|---|---|
S1 | M | 28 | ~8 hours | No | No |
S2 | F | 29 | ~6 hours | No | No |
S3 | F | 31 | 8 hours | No | No |
S4 | M | 26 | 6 hours | No | No |
S5 | M | 32 | ~7 hours | No | No |
S6 | M | 27 | 10 hours | No | No |
S7 | M | 28 | 7 hours | No | No |
S8 | M | 30 | 5 hours | No | No |
S9 | M | 32 | 8 hours | No | No |
S10 | M | 24 | 6 hours | No | No |
Our main objective is to build Adaptive E-learning using BCI. The E-learning system should adapt itself based on user’s cognitive load, which will be measured from his or her brain signal. So we collect our cognitive load data from different subjects during their reading of different readability level texts. Images given below are used texts for one trial of data collection. Level 1 text has high readability index, so data collected during reading of this text is considered as low level cognitive load data. Same way, data collected during reading of level 2 and level 3 texts are considered as medium level and high level cognitive load data respectively.
The experiments were conducted in controlled environment in an electrically isolated BCI laboratory at IIT Kharagpur. We used R.M.S. EEG device and its 64 channel brain cap for this data collection. To minimize any of the artifacts generated, the subjects were asked to avoid any unnecessary physical movements and their hands were placed in a fixed position. Subjects were requested to refrain from excessive blinking of eye lid.
Eight healthy male and three healthy female volunteers engaged in graduate study participated in the experiment with age ranging between 24 to 29 years. All subjects had normal or corrected to normal eye-sight. All subjects were given adequate practice and idea of the experiments. Ethical Clearance was taken for collecting EEG data from them. Room temperature was controlled to ensure adequate comfort during the experiments.
Participants | Gender | Age | Medically fit? | Time of data collection |
---|---|---|---|---|
S1 | M | 26 | Yes | FN |
S2 | M | 27 | Yes | AN |
S3 | M | 29 | Yes | FN |
S4 | M | 24 | Yes | FN |
S5 | M | 26 | Yes | AN |
S7 | M | 29 | Yes | FN |
S8 | M | 31 | Yes | FN |
S9 | F | 27 | Yes | FN |
S10 | F | 29 | Yes | FN |
S11 | F | 28 | Yes | AN |
Related to different directions of BCI-HCI reserach, a large amount of data has been collected in our BCI-HCI Lab, IIT Kharagpur. Further, the implementation of some of our work is published now for the benefits of budding reserachers in the area. Please click for Data and Code.
Dr. Debasis Samanta
Associate Professor
Department of Computer Science & Engineering
IIT Kharagpur
Dr. Pabitra Mitra
Associate Professor
Department of Computer Science & Engineering
IIT Kharagpur
Dr. Monalisa Sarma
Assistant Professor
Subir Chowdhury School of Quality and Reliability
IIT Kharagpur
S. R. Sreeja
Department of Computer Science & Engineering, IIT Kharagpur
Topic: Text Entry System Development using Motor Imagery EEG Signals
Supervisors: Dr. Debasis Samanta, Dr. Pabitra Mitra
Shabnam Samima
Subir Chowdhury School of Quality and Reliability, IIT Kharagpur
Topic: Human Performance Prediction using Brain Computing Interfaces
Supervisor: Dr. Monalisa Sarma
Anushri Saha
Department of Computer Science & Engineering, IIT Kharagpur
Topic: Adaptive E-learning System using Brain Computing Interface
Supervisors: Dr. Debasis Samanta
Prachi Agarwal
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.
Lt. Col. Baljeet Singh Cheema
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.
Joytirmoy Rabha
Department: Computer Science & Engineering
Year: Second
Area of Research: BCI/HCI, Machine Learning (ML), Deep Learning.
Arnab Bir
Department: Mathematics
Year: Fourth
Area of Research: BCI/HCI, Machine Learning.
Vikash
Department: Computer Science & Engineering
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.
Sanjit
Department of Computer Science & Engineering, IIT Kharagpur
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.
Nagarjuna
Department of Computer Science & Engineering, IIT Kharagpur
Year: Fourth
Areas of research: Machine Learning, Artificial Intelligence.