Milestone-Proposal:First control of movement of a physical object using signals emanating from a human brain, 1988


To see comments, or add a comment to this discussion, click here.

Docket #:2018-12

This proposal has been submitted for review.


To the proposer’s knowledge, is this achievement subject to litigation? No

Is the achievement you are proposing more than 25 years old? Yes

Is the achievement you are proposing within IEEE’s designated fields as defined by IEEE Bylaw I-104.11, namely: Engineering, Computer Sciences and Information Technology, Physical Sciences, Biological and Medical Sciences, Mathematics, Technical Communications, Education, Management, and Law and Policy. Yes

Did the achievement provide a meaningful benefit for humanity? Yes

Was it of at least regional importance? Yes

Has an IEEE Organizational Unit agreed to pay for the milestone plaque(s)? Yes

Has an IEEE Organizational Unit agreed to arrange the dedication ceremony? Yes

Has the IEEE Section in which the milestone is located agreed to take responsibility for the plaque after it is dedicated? Yes

Has the owner of the site agreed to have it designated as an IEEE Milestone? Yes


Year or range of years in which the achievement occurred:

1988

Title of the proposed milestone:

First control of a physical object (a robot) using signals emanating from a human brain - 1988

Plaque citation summarizing the achievement and its significance:

In 1988 in this building, for the first time in the world, a physical object (a robot) was controlled using signals emanating from a human brain. After that event, the research and applications of controlling physical objects such as home appliances, wheelchairs, prostheses, drones, and other robots has spread worldwide, for benefit of the society.

In what IEEE section(s) does it reside?

Region 8 Republic of Macedonia Section.Section Chair: Pero Latkoskiemail: pero@feit.ukim.edu.mk

IEEE Organizational Unit(s) which have agreed to sponsor the Milestone:

IEEE Organizational Unit(s) paying for milestone plaque(s):

Unit: Region 8, Republic of Macedonia Section
Senior Officer Name: Senior officer name masked to public

IEEE Organizational Unit(s) arranging the dedication ceremony:

Unit: Region 8, Republic of Macedonia Section
Senior Officer Name: Senior officer name masked to public

IEEE section(s) monitoring the plaque(s):

IEEE Section: Region 8, Republic of Macedonia Section
IEEE Section Chair name: Section chair name masked to public

Milestone proposer(s):

Proposer name: Proposer's name masked to public
Proposer email: Proposer's email masked to public

Please note: your email address and contact information will be masked on the website for privacy reasons. Only IEEE History Center Staff will be able to view the email address.

Street address(es) and GPS coordinates of the intended milestone plaque site(s):

18 Rugjer Boskovic street, 1000 Skopje, Macedonia Faculty of Electrical Engineering and Information Technologies GPS Coordinates: 42°00'17.6"N 21°24'30.0"E

Describe briefly the intended site(s) of the milestone plaque(s). The intended site(s) must have a direct connection with the achievement (e.g. where developed, invented, tested, demonstrated, installed, or operated, etc.). A museum where a device or example of the technology is displayed, or the university where the inventor studied, are not, in themselves, sufficient connection for a milestone plaque.

Please give the address(es) of the plaque site(s) (GPS coordinates if you have them). Also please give the details of the mounting, i.e. on the outside of the building, in the ground floor entrance hall, on a plinth on the grounds, etc. If visitors to the plaque site will need to go through security, or make an appointment, please give the contact information visitors will need. The building where the plaque will be placed has direct connection to the Milestone because in that building the Milestone was achieved. It is the Annex building of the main building of the Faculty of Electrical Engineering and Information Technology (FEEIT) in Skopje. The milestone was achieved in the Laboratory of Intelligent Machines and Bioinformation Systems which was in that building. There are no other historical markers on the site.

Are the original buildings extant?

Yes

Details of the plaque mounting:

t will be mounted on the wall of the hall at the entrance of the Annex building of the Faculty of Electrical Engineering and Information Technology (FEEIT), in which the lab resided at the time the Milestone was achieved.

How is the site protected/secured, and in what ways is it accessible to the public?

The place is inside a University building, so it is secured by the regular rules of entrance to the Annex of the FEEIT building. Usually students are passing by, as well as faculty members. The visitors of the plaque site do not need to go through security during the normal business hours.

Who is the present owner of the site(s)?

University “Sts Cyril and Methodius” in Skopje, MacedoniaFaculty of Electrical Engineering and Information Technologies in Skopje, Macedonia

What is the historical significance of the work (its technological, scientific, or social importance)?

Historical significance

Before 1988, human EEG (electroencephalogram) signals and robotics were distant and mutually exclusive area of science and applications. Moreover, before 1988, controlling movement of a physical object using signals emanating from a human brain was in the realm of the science fiction. There was no idea of using engineering methods to address neither the possibility of psychokinesis nor the relation of EEG and robotics.

In 1988 it was shown for the first time that EEG research and robotics research are related by possibility of new communication channel between humans and robots. It was also shown that a psychokinesis can be achieved by engineering methods. As part of that, for the first time in the technological and scientific history, it was shown how movement of a physical object with a mass can be controlled using signals emanating from a human brain. It was shown how thoughts and absence of them can produce states of the brain which generate different EEG patters which can be used for control of a movement of a physical objects with a mass.

Those achievements, 1) establishing connection between EEG and robotics, as well as 2) solving the long lasting problem of engineering solution of psychokinesis, are most significant regarding scientific and technological importance of this achievement. Wider , social importance is that 3) another science fiction challenge became reality (examples of others are traveling to the moon, traveling below the ocean, etc).

Importance to the evolution of electrical and computer engineering and science

This achievement contributed to the evolution of the way robots communicate with humans. A new communication channel was opened, a direct brain communication with robots.

It also expanded the EEG research and application with a new challenge of controlling physical objects, like opening doors, driving drones, driving wheelchairs, using EEG only. Medical applications with paraplegic patients are also part of this evolution branch.

This achievement also contributed to the evolution of control circuits. It opened a road toward EEG-emulated control circuits. In the 1988 report a picture was shown of an EEG-emulated Schmidt trigger. This control switch recognized change in amplitude and frequency of an EEG signal and triggered corresponding robot control signal. Later in evolution, in 2015, it was shown how other control circuits such as flip-flop and demultiplexer can be EEG-emulated (reference from 2015 in the list of references below).

With that, a new evolutionary branch was opened in the science and engineering evolution tree. It is the evolution branch of EEG controlled physical devices. There is also a term in use for this evolution branch, and it is brain-machine interface. It requires a brain-to-computer interface and a computer-to-machine interface. It is not an universally accepted term, and we use the more descriptive term of control of movement of a physical objects using signals generated by a human brain. In this year, 2018, it is a 30th anniversary of opening this evolution branch. Current status is that almost every biomedical research and/or robotics laboratory now carries out such experiments. Wheelchairs and prostheses are being built and tested showing how paraplegic patients control them using brain signals. The game industry controls robots and drones using brain signals (e.g. Mindwave with Orbit helicopter, Emotiv with other devices). Pentagon is also experimenting with control of drones using brain signals. It is expected that this evolutionary path will be further followed.

Importance to regional/national/international development

It is a worldwide pioneering milestone achievement. The significance of the idea and achievement is, among other things, in how much it was ahead of time. The second achievement of control of movement of a physical object with signals generated by an animal brain was reported in 1999, eleven years after the 1988 achievement. The report was given by Nicolelis' group in USA (cited below). The 1988 and 1999 control of a physical object using signals from a brain, were the only ones achieved in the 20th century. Moreover the 1988 achievement was the only one using a human brain. In the 21st century there is an explosion of development and applications of moving physical objects using signals emanating from a human brain. Applications are medical, gaming, educational, military and others So it has significance in international development of a new technology for communication with robots and more generally in controlling movement of physical objects with a mass.

Benefits to humanity

The benefits to humanity can be seen in new research road as well as new applications for humanity. There are demonstrated applications in paraplegic subjects controlling physical objects. There are applications in entertainment industry with various drones controlled by the brain signals. It is also used in education labs, showing how students can control objects using own EEG signals.


The ways the achievement was a significant advance rather than an incremental improvement of existing technology.

This achievement was a significant advance rather than an incremental improvement of existing technology. The control of a physical object with a mass, using EEG generated intentionally by modulation of the state of the brain, was an unique achievement of solving the psychokinesis problem and opening a road to EEG control of wheelchairs, prostheses, home appliances, drones, and other physical objects. It was a pioneering achievement and opened a new road for science and technology.

What obstacles (technical, political, geographic) needed to be overcome?

Because it was a pioneering result in science and engineering, the most important obstacles were from conceptual and technical nature.

The most important obstacle was the challenge itself, of how to build a system that will demonstrate a psychokinesis control, control of a physical object using signals emanating from the human brain, willingly, by command of a subject. The subject should be able willingly to generate particular signal that will be emanated from the brain. The idea came to use EEG signals. The EEG is a representative of the signals emanating from a human brain. Actually from the brain to the electrodes, which are placed on the scalp outside the brain, the signals travel through various media, but not through wires. It is a wireless contact between the brain and the electrodes which, like antennas, collect signals emanated from the brain

The second part of the challenge was the physical object to be moved. The idea came to be a robot which is energy-autonomous. It would use onboard batteries to move the mass of the robot. EEG signals would be just control signals.

Basically, once the pioneering idea came as a conceptual solution, the other obstacles were of technological and engineering nature.


The second obstacle was the technological infrastructure. It was needed a robot moving space, which should be connected to a brain signal processing unit (biopotential amplifies, AD/DA converter, and a computer). That way EEG research will be related to robotic research. As solution, it was built a robot moving facility with two floors. One floor was robot arena, of size about a ping-pong table, where robots would move. The other, upper floor, was a unit containing a computer-to-robot interface, with power amplifiers for robot motors. That way the wires attached to robots came from above , not to pose an obstacle to the movement of the mobile robots in the arena. So, to overcome the infrastructure obstacle, a unique robot control setup was built, shown in attached Figures.

The third obstacle was to chose the way of encoding mental commands in the EEG signals. It was chosen the well known feature of EEG frequency band between 8 and 13 Hz, (also known as alpha or mu rhythm) to increase amplitudes if brain goes to a relaxation (idle) state. To achieve relaxation, the subject was instructed to close the eyes and relax, while EEG is measured from occipital (visual processing) area. So, when eyes are closed and visual brain relaxed, the EEG amplitudes as well as oscillation periods are higher, which is a command to the robot to move. When eyes are open to see how far the robot is from a predefined station, the visual brain is engaged, the EEG amplitudes and rhythm periods are lower, and the robot stops.

The fourth technical problem was a method of calibration (machine learning) the EEG parameters, which are different for each human subject engaged in controlling the robot. It was chosen a new method of learning two probability density distributions, rather than one, which was previously used in various other control problems. The two distributions were: the differences of both EEG amplitudes and time differences, between the extrema of the EEG signal amplitudes. This double-difference method proved successful in recognizing the events when EEG signal changes both its amplitude and the frequency, which is the case when EEG beta rhythm changes to EEG alpha rhythm and vice versa.

The fifth problem was choosing a demonstration robot and demonstration scenario. It was needed to find a robot that will have meaningful and interesting demonstration of a EEG-based control. A solution was found in the state-of-the-art toy robot from Elehobby, named Movit Line Tracer II which was able to follow a line drawn on the floor. It was a state of the art robot and it was purchased at the Akihabara market in Tokyo, Japan. It was controlled by a start/stop mechanical switch. It the lab, the switch was replaced by a EEG-emulated control switch. The robot was provided with a new outfit, to look like a shuttle robot in a Flexible Manufacturing System (FMS). The demonstration scenario was designed in which a robot moves along a closed circular (actually rectangular) line and was stopped/restarted at particular places ("FMS stations") by EEG encoded commands. That demonstration proved that psychokinesis (movement of a physical object using signals willingly generated from a human brain) was achieved by engineering methods. That way was shown how electrical and computer engineering solved another challenge of the science fiction.

What features set this work apart from similar achievements?

Firstly, it is its pioneering nature. Before 1988 there was no scientific work in relation between human brain signals and mechanical robots. This work also made a transition from science fiction to engineering. Many people taught of possibility of moving physical object with signals emanating from a human brain, but until 1988 nobody came to an idea to record EEG signals and build interface needed to control a movement of a physical object.

Second feature was the time period until the second such achievement was obtained. The 1988 achievement was reported at a IEEE conference in USA and at a conference in Croatia. In 1988 there was no World Wide Web, so it took 11 years till the next similar achievement was reported, in 1999 (Chapin J., Moxon K., Markowitz R., Nicolelis M., Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neuroscience 2, pp. 664–670, 1999). It was about a control of a physical object (a robotic arm) using signals recorded inside the brain of an animal, a rat. The first, 1988 achievement, recorded EEG non-invasively, with surface electrodes placed on a human scalp, while the second achievement in 1999, recorded invasively, with implanted electrodes inside an animal brain.

Another feature is time in history, the relation to the turn of the century. The mentioned 1988 and 1999 reports were the only ones describing control of physical objects with signals generated by a brain (both human and animal). The here described 1988 achievement was the only achievement using a human brain. So in the 20th century there were only one achievement of using human brain signals to control movement of a physical object. After the turn a century, in the 21st century, there are hundreds, possibly thousands of reports showing research and applications of this technology.

References to establish the dates, location, and importance of the achievement: Minimum of five (5), but as many as needed to support the milestone, such as patents, contemporary newspaper articles, journal articles, or citations to pages in scholarly books. You must supply the texts or excerpts themselves, not just the references. At least one of the references must be from a scholarly book or journal article.

To this IEEE Milestone application are attached 7 reference papers which establish dates, locations, and importance of achievement: Comments and excerpts are also given with the papers listed here.

1. Bozinovski, S., Sestakov, M., Bozinovska, L.(1988) Using EEG alpha rhythm to control a mobile robot. In: G. Harris, C. Walker (eds.) Proceedings of 10th Annual Conference of the IEEE Engineering in Medicine and Biology Society, track 17: Biorobotics, New Orleans, LA, vol. 10, pp. 1515–1516

Comment: Edited book from the IEEE conference where the achievement was reported in 1988

2. Bozinovski S. (1990) Mobile robot trajectory control: From fixed rails to direct bioelectric control. In O. Kaynak (Ed.) Proc IEEE International Workshop on Intelligent Motion Control, Istanbul, vol 2: 463-467

Comment: Edited book from the IEEE workshop where the algorithm of learning and pattern recognition was described, which was used in 1988 robot control using EEG signals.

3. Bozinovski, S. (2013) Controlling robots using EEG signals, since 1988, In S. Markovski, M. Gusev (eds.) ICT Innovations 2012, Springer Verlag, p. 1-11, 2013

Comment: Edited book chapter from conference where, as plenary keynote paper, more detailed report of the achievement was presented.

4. Bozinovski S., Bozinovski A. (2015) Mental states, EEG manifestations, and mentally emulated digital circuits for brain-robot interaction, IEEE Transactions on Autonomous Mental Development 7(1): 39-51

Comment: Journal paper mentioning the 1988 achievement and describing emulation of digital circuits using EEG signals. In addition to the EEG emulated switch used in 1988, EEG emulation of a flip-flop and a demultiplexer is also described in this paper.

5. Lebedev M. (2016) Augmentation of sensorimotor functions with neural prostheses. Opera Medica and Physiologica. Vol. 2 (3): 211-227

Comment: Journal paper, a review on neural prostheses, from a member of the team that in 1999 achieved second control of a robot using signals from a brain.

Excerpt ( page 213): "In 1988, the first report was published where human subjects controlled a robot with their EEG (Bozinovski, Sestakov et al. 1988). In that study, subjects issued binary commands by closing and opening their eyes. This maneuver started and stopped an alpha recorded with EEG sensors placed over the occipital cortex."

6. Bozinovski S. (2017) Signal processing robotics using signals generated by a human head: From pioneering works to EEG-based emulation of digital circuits, In A. Rodić and T. Borangiu (eds.), Advances in Robot Design and Intelligent Control, Advances in Intelligent Systems and Computing 540, Springer Verlag, p. 449-462

Comment: Edited book chapter from conference where, as plenary keynote paper, description was given on evolution of signal processing robotics, including speech signal for control of a mobile robots, and understanding human EEG messages.

Excerpt (page 454): "1988: Step in robot evolution: Robots gain capability of recognizing human EEG messages"

7. Lebedev M., Nicolelis M. (2017) Brain-machine interfaces: from basic science to neuroprostheses and neurorehabilitation, Physiological Review 97:737-867

Comment: Journal paper, a review on the brain-machine interface from the leader and a member the team which in 1999 made the second achievement of control of a robot using signals from a brain.

Excerpt (page 779): "It is worth noting that the first publication on a human controlling a robot with EEG activity dates back to 1988 (85)" where reference (85) is referencing the 1988 paper. .

8. Stevo Bozinovski & Liljana Bozinovska (2019), Brain–Computer Interface in Europe: the thirtieth anniversary, Journal for Control, Measurement, Electronics, Computing and Communications, Volume 60, 2019 - Issue 1, https://www.tandfonline.com/doi/full/10.1080/00051144.2019.1570644

Supporting materials (supported formats: GIF, JPEG, PNG, PDF, DOC): All supporting materials must be in English, or if not in English, accompanied by an English translation. You must supply the texts or excerpts themselves, not just the references. For documents that are copyright-encumbered, or which you do not have rights to post, email the documents themselves to ieee-history@ieee.org. Please see the Milestone Program Guidelines for more information.

This application contains 3 supporting materials:

1. The supporting 7 papers listed above, in PDF format. They are:

2. The supporting 6 photographs are shown with explanations. They are attached in pdf format in a file named Media:Milestone1988ETFsupportingPhotographs.pdf

3. The supporting letters from IEEE Section supporting this application with plaque payment and maintenance, and from the owner of the site with agreement to place a plaque to be designated as a IEEE Milestone. The letters are sent to the IEEE History Center separately from this application.

Please email a jpeg or PDF a letter in English, or with English translation, from the site owner(s) giving permission to place IEEE milestone plaque on the property, and a letter (or forwarded email) from the appropriate Section Chair supporting the Milestone application to ieee-history@ieee.org with the subject line "Attention: Milestone Administrator." Note that there are multiple texts of the letter depending on whether an IEEE organizational unit other than the section will be paying for the plaque(s).