View source for Milestone-Proposal:First control of movement of a physical object using signals emanating from a human brain, 1988

{{Proposal
|docketid=2018-12
|litigation=No
|more than 25 years=Yes
|within fields of interest=Yes
|benefit to humanity=Yes
|regional importance=Yes
|ou is paying=Yes
|ou is arranging dedication=Yes
|section is taking responsibility for plaque=Yes
|a11=Yes
|a3=1988
|a1=First control of a physical object (a robot) using signals emanating from a human brain - 1988
|plaque citation=In 1988 in this building, for the first time in the world, a physical object (a robot) was controlled using signals emanating from a human brain. After that event, the research and applications of controlling physical objects such as home appliances, wheelchairs, prostheses, drones, and other robots has spread worldwide, for benefit of the society.
|a2b=Region 8 Republic of Macedonia Section.

Section Chair: Pero Latkoski
email: pero@feit.ukim.edu.mk
|IEEE units paying={{IEEE Organizational Unit Paying
|Unit=Region 8, Republic of Macedonia Section
|Senior officer name=Section Chair: Pero Latkoski
|Senior officer email=pero@feit.ukim.edu.mk
}}
|IEEE units arranging={{IEEE Organizational Unit Arranging
|Unit=Region 8, Republic of Macedonia Section
|Senior officer name=Section Chair: Pero Latkoski
|Senior officer email=pero@feit.ukim.edu.mk
}}
|IEEE sections monitoring={{IEEE Section Monitoring
|Section=Region 8, Republic of Macedonia Section
|Section chair name=Section Chair: Pero Latkoski
|Section chair email=pero@feit.ukim.edu.mk
}}
|Milestone proposers={{Milestone proposer
|Proposer name=Stevo Bozinovski
|Proposer email=sbozinovski@scsu.edu
}}
|a2a=18 Rugjer Boskovic street, 1000 Skopje, Macedonia 
Faculty of Electrical Engineering and Information Technologies 
GPS Coordinates: 42°00'17.6"N 21°24'30.0"E
|a7=The building where the plaque will be placed has direct connection to the Milestone because in that building the Milestone was achieved. It is the Annex building of the main building of the Faculty of Electrical Engineering and Information Technology (FEEIT) in Skopje. The milestone was achieved in the Laboratory of Intelligent Machines and Bioinformation Systems which was in that building. There are no other historical markers on the site.
|a8=Yes
|mounting details=t will be mounted on the wall of the hall at the entrance of the Annex building of the Faculty of Electrical Engineering and Information Technology (FEEIT), in which the lab resided at the time the Milestone was achieved.
|a9=The place is inside a University building, so it is secured by the regular rules of entrance to the Annex  of the FEEIT building. Usually students are passing by, as well as faculty members. The visitors of the plaque site do not need to go through security during the normal business hours.
|a10=University “Sts Cyril and Methodius” in Skopje, Macedonia
Faculty of Electrical Engineering and Information Technologies in Skopje, Macedonia
|a4=Historical significance 

Before 1988, human EEG (electroencephalogram) signals and robotics were distant and mutually exclusive area of science and applications. Moreover, before 1988, controlling movement of a physical object using signals emanating from a human brain was in the realm of the science fiction. There was no idea of using engineering methods to address neither the possibility of psychokinesis nor the  relation of EEG and robotics.  

In 1988 it was shown for the first time that EEG research and robotics research are related by possibility of new communication channel between humans and robots. It was also shown that a psychokinesis can be achieved by engineering methods. As part of that, for the first time in the technological and scientific history, it was shown how movement of a physical object with a mass can be controlled using signals emanating from a human brain. It was shown how thoughts and absence of them can produce states of the brain which generate different EEG patters which can be used for control of a movement of a physical objects with a mass. 

Those achievements, 1) establishing connection between EEG and robotics, as well as 2) solving the long lasting problem of engineering solution of psychokinesis, are most significant regarding scientific and technological importance of this achievement. Wider , social importance is that 3) another science fiction challenge became reality (examples of others are traveling to the moon, traveling below the ocean, etc). 

Importance to the evolution of electrical and computer engineering and science

This achievement contributed to the evolution of the way robots communicate with humans. A new communication channel was opened, a direct brain communication with robots. 

It also expanded the EEG research and application with a new challenge of controlling physical objects, like opening doors, driving drones, driving wheelchairs, using EEG only. Medical applications with paraplegic patients are also part of this evolution branch. 

This achievement also contributed to the evolution of control circuits. It opened a road toward EEG-emulated control circuits. In the 1988 report a picture was shown of an EEG-emulated Schmidt trigger. This control switch recognized change in amplitude and frequency of an EEG signal and triggered corresponding robot control signal. Later in evolution, in 2015, it was shown how other control circuits such as flip-flop and demultiplexer can be EEG-emulated (reference from 2015 in the list of references below).  

With that, a new evolutionary branch was opened in the science and engineering evolution tree. It is the evolution branch of EEG controlled physical devices. There is also a term in use for this evolution branch, and it is brain-machine interface. It requires a brain-to-computer interface and a computer-to-machine interface. It is not an universally accepted term, and we use the more descriptive term of control of movement of a physical objects using signals generated by a human brain. In this year, 2018, it is a 30th anniversary of opening this evolution branch. Current status is that almost every biomedical research and/or robotics laboratory now carries out such experiments. Wheelchairs and prostheses are being built and tested showing how paraplegic patients control them using brain signals. The game industry controls robots and drones using brain signals (e.g. Mindwave with Orbit helicopter, Emotiv with other devices). Pentagon is also experimenting with control of drones using brain signals. It is expected that this evolutionary path will be further followed.  

Importance to regional/national/international development

It is a worldwide pioneering milestone achievement. 
The significance of the idea and achievement is, among other things, in how much it was ahead of time. The second achievement of control of movement of a physical object with signals generated by an animal brain was reported in 1999, eleven years after the 1988 achievement. The report was given by Nicolelis' group in USA (cited below). The 1988 and 1999 control of a physical object using signals from a brain, were the only ones achieved in the 20th century. Moreover the 1988 achievement was the only one using a human brain. In the 21st century there is an explosion of development and applications of moving physical objects using signals emanating from a human brain. Applications are medical, gaming, educational, military and others So it has significance in international development of a new technology for communication with robots and more generally in controlling movement of physical objects with a mass.

Benefits to humanity

The  benefits to humanity can be seen in new research road as well as new applications for humanity. There are demonstrated applications in paraplegic subjects controlling physical objects. There are applications in entertainment industry with various drones controlled by the brain signals. It is also used in education labs, showing how students can control objects using own EEG signals.  


The ways the achievement was a significant advance rather than an incremental improvement of existing technology.

This achievement was a significant advance rather than an incremental improvement of existing technology. The control of a physical object with a mass, using EEG generated intentionally by modulation of the state of the brain,  was an unique achievement of solving the psychokinesis problem and opening a road to EEG control of wheelchairs, prostheses, home appliances, drones, and other physical objects. It was a pioneering achievement and opened a new road for science and technology.
|a6=Because it was a pioneering result in science and engineering, the most important obstacles were from conceptual and technical nature. 

The most important obstacle was the challenge itself, of how to build a system that will demonstrate a psychokinesis control, control of a physical object using signals emanating from the human brain, willingly, by command of a subject. The subject should be able willingly to generate  particular signal that will be emanated from the brain. The idea came to use EEG signals. The EEG is a representative of the signals emanating from a human brain. Actually from the brain to the electrodes, which are placed on the scalp outside the brain,  the signals travel through various media, but not through wires. It is a wireless contact between the brain and the electrodes which, like antennas,  collect signals emanated from the brain

The second part of the challenge was the physical object to be moved. The idea came to be a robot which is energy-autonomous. It would use onboard batteries to move the mass of the robot. EEG signals would be just control signals. 

Basically, once the pioneering idea came as a conceptual solution, the other obstacles were of technological and engineering nature. 


The second obstacle was the technological infrastructure.  It was needed a robot moving space, which should be connected to a brain signal processing unit (biopotential amplifies, AD/DA converter, and a computer). That way EEG research will be related to robotic research. As solution, it was built a robot moving facility with two floors. One floor was robot arena, of size about a ping-pong table, where robots would move. The other, upper floor, was a unit containing a computer-to-robot interface, with power amplifiers for robot motors. That way the wires attached to robots came from above , not to pose an obstacle to the movement of the mobile robots in the arena. So, to overcome the infrastructure obstacle, a unique robot control setup was built, shown in attached Figures. 

The third obstacle was to chose the way of encoding mental commands in the EEG signals. It was chosen the well known feature of EEG frequency band between 8 and 13 Hz, (also known as alpha or mu rhythm) to increase amplitudes if brain goes to a relaxation (idle) state. To achieve relaxation, the subject was instructed to close the eyes and relax, while EEG is measured from occipital (visual processing) area. So, when eyes are closed and visual brain relaxed, the EEG amplitudes as well as oscillation periods are higher, which is a command to the robot to move. When eyes are open to see how far the robot is from a predefined station, the visual brain is engaged, the EEG amplitudes and rhythm periods are lower, and the robot stops. 

The fourth technical problem was a method of calibration (machine learning) the EEG parameters, which are different for each human subject engaged in controlling  the robot. It was chosen a new method of learning two probability density distributions, rather than one, which was previously used in various other control problems. The two distributions were: the differences of both EEG amplitudes and time differences, between the extrema of the EEG signal amplitudes. This double-difference method proved successful in recognizing the events when EEG signal changes both its amplitude and the frequency, which is the case when EEG beta rhythm changes to EEG alpha rhythm and vice versa. 

 The fifth problem was choosing a demonstration robot and demonstration scenario. It was needed to find a robot that will have meaningful and interesting demonstration of a EEG-based control. A solution was found in the state-of-the-art toy robot from Elehobby, named Movit Line Tracer II which was able to follow a line drawn on the floor. It was a state of the art robot and it was purchased at the Akihabara market in Tokyo, Japan. It was controlled by a start/stop mechanical switch. It the lab, the switch was replaced by a EEG-emulated control switch. The robot was provided with a new outfit, to look like a shuttle robot in a Flexible Manufacturing System (FMS). The demonstration scenario was designed in which a robot moves along a closed circular (actually rectangular) line and was stopped/restarted at particular places ("FMS stations") by EEG encoded commands. That demonstration proved that psychokinesis (movement of a physical object using signals willingly generated from a human brain) was achieved by engineering methods. That way was shown how electrical and computer engineering solved another challenge of the science fiction.
|a5=Firstly, it is its pioneering nature. Before 1988 there was no scientific work in relation between human brain signals and mechanical robots. This work also made a transition from science fiction to engineering. Many people taught of possibility of moving physical object with signals emanating from a human brain, but until 1988 nobody came to an idea to record EEG signals and build interface needed to control a movement of a physical object. 

Second feature was the time period until the second such achievement was obtained. The 1988 achievement was reported at a IEEE conference in USA and at a conference in Croatia. In 1988 there was no World Wide Web, so it took 11 years till the next similar achievement was reported, in 1999 (Chapin J., Moxon K., Markowitz R., Nicolelis M., Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neuroscience 2, pp. 664–670, 1999). It was about a control of a physical object (a robotic arm) using signals recorded inside the brain of an animal, a rat. The first, 1988  achievement, recorded EEG non-invasively, with surface electrodes placed on a human scalp, while the second achievement in 1999, recorded invasively, with implanted electrodes inside an animal brain. 

Another feature is time in history, the relation to the turn of the century. The mentioned 1988 and 1999 reports were the only ones describing control of physical objects with signals generated by a brain (both human and animal). The here described 1988 achievement was the only achievement using a human brain. So in the 20th century there were only one achievement of using human brain signals to control movement of a physical object. After the turn a century, in the 21st century, there are hundreds, possibly thousands of reports showing research and applications of this technology.
|references=To this IEEE Milestone application are attached 7 reference papers which establish dates, locations,  and importance of achievement: Comments and excerpts are also given with the papers listed here.  

1. Bozinovski, S., Sestakov, M., Bozinovska, L.(1988) Using EEG alpha rhythm to control a mobile robot. In: G. Harris, C. Walker (eds.) Proceedings of 10th Annual Conference of the IEEE Engineering in Medicine and Biology Society, track 17: Biorobotics, New Orleans, LA, vol. 10, pp. 1515–1516 

	Comment: Edited book from the  IEEE conference where the achievement was reported 	in 1988

2. Bozinovski S. (1990) Mobile robot trajectory control: From fixed rails to direct bioelectric control. In O. Kaynak (Ed.) Proc IEEE International Workshop on Intelligent Motion Control, Istanbul, vol 2: 463-467

	Comment: Edited book from the IEEE workshop where the algorithm of learning and pattern recognition was described, which was used in 1988 robot control using EEG signals.

3. Bozinovski, S. (2013) Controlling robots using EEG signals, since 1988, In S. Markovski, M. Gusev (eds.) ICT Innovations 2012, Springer Verlag, p. 1-11, 2013  

	Comment: Edited book chapter from conference where, as plenary keynote paper, more 	detailed report of the achievement was presented. 

4. Bozinovski S., Bozinovski A. (2015) Mental states, EEG manifestations, and mentally emulated digital circuits for brain-robot interaction, IEEE Transactions on Autonomous Mental Development 7(1): 39-51

	Comment: Journal paper mentioning the 1988 achievement and describing emulation of 	digital circuits using EEG signals. In addition to the EEG emulated switch used in 1988, EEG emulation of a flip-flop and a demultiplexer is also described in this paper. 

5. Lebedev M. (2016) Augmentation of sensorimotor functions with neural prostheses. Opera Medica and Physiologica. Vol. 2 (3): 211-227

	Comment: Journal paper, a review on neural prostheses, from a member of the team that  in 1999 achieved second control of a robot using signals from a brain.

	Excerpt ( page 213): "In 1988, the first report was published where human subjects controlled a robot with their EEG (Bozinovski, Sestakov et al. 1988). In that study, 	subjects issued binary commands by closing and opening their eyes. This maneuver 	started and stopped an alpha recorded with EEG sensors placed over the occipital cortex."

6. Bozinovski S. (2017) Signal processing robotics using signals generated by a human head: From pioneering works to EEG-based emulation of digital circuits, In A. Rodić and T. Borangiu (eds.), Advances in Robot Design and Intelligent Control, Advances in Intelligent Systems and Computing 540, Springer Verlag, p. 449-462

	Comment: Edited book chapter from conference where, as plenary keynote paper, description was given on evolution of signal processing robotics, including speech signal for control of a mobile robots, and understanding  human EEG messages. 

	Excerpt (page 454): "1988: Step in robot evolution: Robots gain capability of 	recognizing human EEG messages"

7. Lebedev M., Nicolelis M. (2017) Brain-machine interfaces: from basic science to neuroprostheses and neurorehabilitation, Physiological  Review  97:737-867  

	Comment: Journal paper, a review on the brain-machine interface from the leader and a 	member the team which in 1999 made the second achievement of control of a robot using signals from a brain.

	Excerpt (page 779): "It is worth noting that the first publication on a human controlling a 	robot with EEG activity dates back to 1988 (85)" where reference (85) is referencing 	the 1988 paper. .
|supporting materials=This application contains 3 supporting materials: 

1. The supporting 7 papers listed above, in PDF format. They are:

*[[Media:1988 EEGrobotControl NewOrleans.pdf]] ‎
*[[Media:1990 EEGalgorithm Istanbul.pdf]]‎
*[[Media:2012 moreDetails SpringerBookChapter.pdf]] ‎
*[[Media:2015 EEGemulatedCircuits JournalPaper.pdf]]‎
*[[Media:2016 page213ReviewExplicitCitationHighlited JournalPaper.pdf]] ‎
*[[Media:2017 page454RobotCommunicateusingEEG SpringerBookChapterhighlighted.pdf]] ‎
*[[Media:2017 page779ReviewCitationHighlighted JournalPaper.pdf]] ‎

2. The supporting 6 photographs are shown with explanations. They are attached in pdf  format in a file named [[Media:Milestone1988ETFsupportingPhotographs.pdf]] ‎

3. The supporting letters from IEEE Section supporting this application with plaque payment and maintenance, and from the owner of the site with agreement to place a plaque to be designated as a IEEE Milestone.  The letters are sent to the IEEE History Center separately from this application.
|submitted=Yes
}}