A Story on How an Economist Is Helping Create ‘Smart’ Artificial Limbs

The Higher School of Economics is very serious about students getting involved in undergraduate research. For Russian Science Day, celebrated on February 8, the HSE News Service spoke with three young HSE researchers about their work and learned how the students set goals for themselves, achieve results, and move forward with what some might deem frighteningly ambitious research ideas. One of them was Elizaveta Okorokova, second-year student in the ‘Cognitive Sciences and Technologies: From Neuron to Cognition’ master’s programme.

The School of Psychology is home to the Centre for Cognition & Decision Making, which conducts research at the forefront of neuroscience with the help of modern technologies such as transcranial magnetic stimulators, myograms, and multi-channel EEG. Taking part is this groundbreaking research are psychologists, neuroscientists, economics, linguistics, physicists, and engineers from countries such as Russia, Italy, Turkey, England, the U.S., and Spain. One such research group includes Elizaveta Okorokova, whose work might lead to the creation of smart prosthetic limbs.

Elizaveta’s story is a shining example of how open and interdisciplinary contemporary science truly is. Though she is currently a second-year student in the Cognitive Sciences and Technologies: From Neuron to Cognition’ master’s programme, Elizaveta actually completed her bachelor’s in economics from the HSE International College of Economics and Finance (ICEF).

‘As a student in ICEF, I worked with Alexei Belyanin in the Laboratory for Experimental and Behavioural Economics, where we studied people’s behaviour in economic situations,’ Elizaveta says. ‘There came a point when I realised that you could look at this problem from a different light and from a biological point of view. What processes occur in a person’s head when making a decision? I became fascinated by this question, and Alexei recommended that I read about neuroscience. He introduced me to Vasily Klyucharev and Anna Shestakova, who work in the field in the School of Psychology. They told me that the programme was accepting new students, which really interested me, so I decided to delve further into the subject,’ Elizaveta comments.

And that is exactly what she did – after just a year, Elizaveta’s researched was published in the international journal Frontiers in Neuroscience.

The idea for the study partially came from HSE Professor Alexey Ossadtchi and Mikhail Lebedev from Duke University, who are both Elizaveta’s academic supervisors. The original idea concerned handwriting recognition based on a myogram, i.e., signals that are non-invasively read from the surface of human muscles, in this case wrists.

But how can an economist understand these signals? This is possible because the same data analysis methods used in the field of economics and finance are somewhat similar to those used in neuroscience. ‘Since I had prior experience in finance and exchange processing signals, I was able to quickly switch over to processing temporary series of a neurophysiological nature,’ Elizaveta explains.

It is worth noting that very few researchers previously studied this area; aside from Duke University specialists, almost nobody had focused on handwriting recognition. Mikhail Lebedev was able to build a linear model that recognised the coordinates of a pen using a myogram. This model was improved upon at HSE, with dynamic properties being added to the system, allowing for the volume of necessary data to be decreased while the ergonomics of restored trajectories was increased.

What was done exactly? A myogram has signals that can be read from a person’s hand muscles, and there are also coordinates that reflect changes in where a person is holding the pen on the paper while writing. The objective was to think of a model that would best link the myogram’s signals to these coordinates.

Many tech companies are busy developing gadgets that use similar principles, but this mostly concerns intermittent recognition, whereby only separate movements can be modelled. ‘What we’ve done recognises not individual changes, but continuous alterations in coordinates on a piece of paper. There are not two, three, or five positions here, but an unlimited amount; a pen can be anywhere at any point in time,’ Elizaveta comments.

‘For example, when you write the number 7, while you’re moving your pen in a straight line, our algorithm is paying special attention to a dynamic model that “loves” smooth movements,’ Alexey Ossadtchi explains. ‘But when you reach the corner [of the 7], the myogram signal that dictates sharp changes in direction becomes more important for the algorithm. Naturally, everything happens in an automatic and adaptive fashion, and we are not teaching the algorithm to write each figure separately; instead we are using a general dynamic model and a “predictions merger” with the help of a Kalman filter.’

Elizaveta Okorokova believes that the model she and her colleagues have developed, while better than what existed before, still does not perfectly determine a pen’s exact coordinates. But this is still the first and most important step towards creating ‘smart’ artificial limbs that allow disabled people and amputees to regain fine motor skills and improve more microscopic movements as well. This would not only allow them to write, but they would also be able to paint, sculpt, and play musical instruments.

More and more researchers are joining in on the project that Okorokova and Ossadtchi are working on. ‘George Sapozhnikov, a graduate of the Moscow Aviation Institute, came to us the first year of my master’s programme,’ Elizaveta remembers. ‘He is writing a programme that allows for muscle signals to be read, and multiple movements recognised, in real time. So if we were to put electrodes on you, this programme would be able to recognise the movements you’re making,’ she says.

In the future, these developments and the improved recognition algorithm will be tested on an experimental prosthetic limb currently being created by a Skolkovo resident that is among the many companies that work with HSE. Then clinical testing will follow, and the technology will be tried on real patients.

All stories