Meta, the company that is banned in the Russian Federation, introduced the AI-model Brain2Qwerty v2, which can recreate the text a user types on the keyboard based on the analysis of the electrical activity of the brain recorded using magnetoencephalography (MEG). The toolkit for training and running the model, a framework for processing magnetoencephalography data, and a library for training models using brain electrical activity data are all available. The dataset used to train the model in the first experiment can be downloaded here (data for the second experiment will be released later, after acceptance into a scientific journal). The library code is open under the MIT license, and the data is distributed under the CC BY-NC 4.0 license.
The accuracy of the Brain2Qwerty v2 model, when analyzing raw magnetoencephalography results, averaged 61% in recreating the typing of individual words on the keyboard. The best result achieved by one of the experiment participants was 78%. In comparison, the first version of the model, trained on less data, had an average efficiency of 40% and 48% for the best result. Other non-invasive text recovery methods based on brain activity analysis have an estimated effectiveness of around 8%.
The Brain2Qwerty v2 model was trained on brain activity data recorded from 22 thousand sentences typed by 9 participants in the experiment. The data was collected using the Megin magnetoencephalography system (Elekta Neuromag) with 102 magnetometers and 204 gradiometers, scanning at 1000 scans per second.
During the experiment, participants wore headphones and were asked to type a specific sentence they heard while their brain activity was recorded. They were instructed to gaze at a rotating square on the screen while typing.
