Milestones:MPEG integrated circuits
Title
MPEG Multimedia Integrated Circuits, 1984-1993
Citation
Beginning in 1984, Thomson Semiconducteurs (now STMicroelectronics) developed multimedia integrated circuits, which accelerated Moving Picture Experts Group (MPEG) standards. By 1993, MPEG-2 integrated decoders -- including innovative discrete cosine transform (developed jointly with ENST, now Telecom ParisTech), bitstream decompression, on-the-fly motion compensation, and display unit -- were announced in one silicon die: the STi3500. Subsequent MPEG-2 worldwide adoption made compressed full-motion video and audio inexpensive and available for everyday use.
Street address(es) and GPS coordinates of the Milestone Plaque Sites
45.203333, 5.695833, (45.203333, 5.695833)
Details of the physical location of the plaque
Plaques will be installed in a public place at the main entrance as shown in figure 1. It will be in front of company security personnel who are monitoring entrance 24 hours in a day, every day in a year. Very close as indicated there is the public road. Plaque will be placed at 12, rue Jules Horowitz F-38000 Grenoble, France. GPS coordinates are: 45°12’12” N ; 05°41’45” E (45.203333, 5.695833). Therefore, interested visitors will not need to be escorted by the company security personnel to watch the plaque or to take a photo with it. The milestone plaque will be installed right in front of the main entrance access. It is where employees, visitors and customers must pass through to get into the Grenoble offices on daily basis. It is monitored continuously 24h/7d by security human resources and surveillance camera. It is a public place, right outside the site restricted perimeter.
How the intended plaque site is protected/secured
It is protected by company security human resources and with surveillance camera, 24h/7d. It is publicly and easily accessible from 12, rue Jules Horowitz F-38000 Grenoble, France. The site can be reached also from Lion Airport, close to Grenoble as shown in figure 2.
Historical significance of the work
The importance of the proposed milestone will be described as follows: section 1 introduces the object; section 2 lists the technical innovations; section 3 discusses the benefits of the innovations and section 4 their achievements; section 5 provides the historical background about when the innovations were introduced; Section 6 reports the public recognition of the integrated circuits by specialized journals. Finally, section 7 is for the supporting letters provided by world renown experts.
1) Object of the milestone proposal
Starting from 1984 new digital multimedia technologies were conceived and implemented by SGS-THOMSON on different CMOS silicon technologies. They were dedicated processors very optimized in area and power consumption. They were hand-designed and produced in high volume, with high yields and reliability to achieve lowest cost. Next, they were incrementally super-integrated in one silicon die to perform multimedia video decoding anticipating specifications as established by MPEG in the 1992 onward. Several innovations were introduced to reduce the complexity due to the super integration. They anticipated the acceleration of the MPEG1, 2 and subsequent standards. Since no digital multimedia content services for moving pictures were existing at that time, they set the hardware to introduce them. They facilitated widespread adoption by the end users and eased switch-off from analog decoders to digital ones. A special mention is for the STi3500 chip which was the world first MPEG2 decoder. The chip die is shown in the figure 3.
2) Technical innovations introduced by the milestone
The circuits introduced several innovations. They were:
a) Block based pixel memory management to achieve closely the peak computational power.
b) Compressed MPEG decoding on the fly to reduce video storage in RAM.
c) Frame buffer lossy compression to co-decompress on the fly MPEG2 video
d) Unified memory controller allowed to plug and play additional MPEG and other hardware clients.
3) Benefits of the innovative solutions subject of the milestone
The benefits of integrated circuits which embodied innovations described in section 2, were:
a) Transistor count minimization to ease super integration in one silicon die.
b) Hand crafted hardware design methodology to produce very tiny circuits.
c) Energy consumption to less than one watt across chip generations.
d) Reliability to produce chips for widespread adoption
e) Resilience to faults.
4) Technical achievements of the proposed milestone
The achievements of the processors of the milestone were on the following key indicators:
a) Performances (highest degree of acceleration of MPEG functionalities).
b) Manufacturability (reliability, repeatability, high volumes, robustness, high yields)
c) Super integration of RAM.
d) Silicon node scaling from 2µm to 28nm FDSOI (fully depleted silicon on insulator).
e) Flexibility (VLIW processors, CPUs, 3D programmable graphic)
f) Security (data and memory protection encryption).
g) Efficiency (mW/sqmm/$, reduced memory bandwidth, and reduced calculations bit-depth)
h) Tiny dissipation (avoided costly and bulky heat-sinks and complex liquid cooling)
l) Costs (cheap solutions for end users, eased TV set manufacturers to achieve large scale savings)
m) Ecosystem (key enabler of broad range digital video applications and digital multimedia vertical services, facilitated the end user to enter the new digital entertainment era)
5) Historical background and contributions
THOMSON Semiconductor (headquarter in Grenoble, France) was merged with SGS (headquarter in Agrate Brianza Italy) in 1987; the merge was named SGS-THOMSON; then the company was renamed in 1998 as STMicroelectronics (today name).
In the late 1980s ISO/IEC experts felt the need to standardize motion picture decoding. Therefore, MPEG committee had first meeting in 1988. MPEG1 standard was finalized in 1992 and MPEG2 was finalized in 1994.
All MPEG integrated circuits, object of this proposed milestone, are developed fully in-house SGS-THOMSON Grenoble and intellectual properties are owned by STMicroelectronics.
Next points a) to d) details that the technology subject of this proposed milestone, the MPEG circuits :
a) In 1984, THOMSON Semiconductors started to develop own Discrete Cosine Transform (DCT). This effort was patented twice, therefore the company decided to launch an own chip design and production project in 1986.
b) In 1986 ENST (now Telecon ParisTech) had in parallel a DCT experiments based on a less hardware efficient algorithm than THOMSON designed.
c) In 1987, THOMSON Semiconductor and ENST, even if not essential to DCT chip, decided to cooperate on redesigning a part of the whole DCT chip. The operative part with a serial architecture was conceptualized by ENST and designed by THOMSON in hardware.
d) In 1988 SGS-THOMSON had following products ready for production
i. STV3200 the multi format DCT, not MPEG compliant
ii. STV3208 8x8 DCT, MPEG compliant. Note that DCT into modern chips was just 0.05% of total chip area
Next points aim to explain how MPEG chips were developed.
e) First MPEG1/H261 standard compliant . First solution was composed by two integrated circuits: the picture motion compensation, huffman decoding, dequantization, memory control (into STi3240) and the 8x8 DCT (STV3208). Second solution was the super integration of the two circuits into a single chip named STi3400 .
f) World first STi3500 MPEG2 decoder circuit. It integrated STV3208 with MPEG2 compliant motion compensation. RCA, later acquired by Thomson Consumer Electronics, was creating the world first Digital Satellite TV broadcast service but missed the video decoding function. STi3500 was delivered to RCA (ahead of the competition) and enable decoding of the DirectTV digital multimedia broadcast. Next STI3520A was the world first single chip for MPEG 2 video decoding in only 16 Mbit RAM super integrated with the world first MPEG MP3 Audio decoder (STi4500).
g) First HDTV decoder circuit with RAM reduction algorithms integrated: the STi7000 in 1998. Next versions STi7100, STi7200 integrated 3D shader-based graphics engine and powerful micro controllers. STi7108 had dual CPU host processors with embedded memory. It was the first circuit in the world to integrate 3D OpenGL-ES 2.0 and OpenVG vector graphics.
h) STi5500 was the first (after signal demodulation circuit) complete decoder for set-top boxes and DVD players. Capable to demux MPEG2 transport stream. Memory compression algorithms reduced its RAM needs.
i) World first STi5505, that integrated all the back-end functions of a DVD player. More powerful than the STi5500 and capable to support MPEG2 program stream demultiplexing inside. STi5500 won from a field of nearly 300 nominations, the prestigious European IT Prize awarded by the European Commission in conjunction with Euro-CASE, the European Council of Applied Science and Engineering. Its further evolutions were STi5600 and STi5610.
l) Further generations. Many other integrated circuits were designed over 30 years, as shown in figure 7. Complex operative systems such as Linux and Windows were incrementally supported. These processors also pioneered TV Cable transmission with Scientific Atlanta (which was acquired by Cisco later). They accelerated the introduction of CD-ROM, DVDs, Set Top Box and digital TVs home systems. The analog to digital entertainment switch was therefore supported by mass production circuits and doable in everyone home.
Since the functionalities they integrated, these chips were deployed also into smartphones since early 2000s. The Nomadik chip family (STn8800/10/15 and subsequent) was derived to accelerate multimedia into mobile phones. The pioneering 8x8 MPEG DCT chip, the STV3208, represented less than 0.1% of the die area of the SoC as shown in figure 8.
ST manufactured more than 500 million (cumulative number) of these integrated systems worldwide as shown in Figure 9.
A rich digital consumer ecosystem has been created thanks to those circuits. It was composed by many operators, equipment manufacturers, software developers and conditional access companies as shown in the next figure 10.
7) Public recognition of the integrated circuits Specialize magazines reported the leadership of the MPEG circuits.
a) EETimes article “In the first shift or wave (referred to in the set-top box market), which began about 1994, the boxes were simple channel-hopping devices for satellite and cable TV providers, Hatch explained. (STMicroelectronics's MPEG-2 video decoders were used then in the boxes manufactured by Thomson's RCA subsidiary.)”.
b) EDN article on 1996 “Dataquest has confirmed SGS-THOMSON Microelectronics (Lincoln, MA) as the world's leading supplier of MPEG decoder ICs in 1995.”
c) STMicroelectronics NV History quoting “ STMicroelectronics is the world's leading manufacturer of .... MPEG-2 decoder chips, used to provide video decompression for DVD players and digital television set-top boxes.” and “Another area in which SGS-Thomson became an early player was its development of MPEG decompression chips. The company's dedication to MPEG technology was risky, as a number of other digital video decompression schemes were competing for what promised to be a huge market in the future. SGS Thomson gained an early lead in the MPEG market. By 1993, the company had debuted its "multimedia" chip, capable of decompressing digital video files for display on a television set. This chip helped the company take a major position in the new set-top box market, starting with supplying the chip for the Hughes digital satellite television set-top box.”
8) Supporting letters by world renown and independent experts in the field Please refer to part 7 of bibliography
Features that set this work apart from similar achievements
They were: a) accuracy of results provided by the chips, b) their maturity, and d) the advancements they featured.
a) Accuracy of results ST chip without visible mismatch errors versus anchor’s references as established by MPEG core experiments
b) Chip maturity: SGS-Thomson chips were released earlier and the more mature vs Toshiba, Fujitsu, Telettra, ANT, C-Cube. c) Chip Advancements. They significantly provided low silicon area footprint by exploiting advanced CMOS silicon lithography such as 0.8, 0.5, 0.35um up to 28nm FDSOI.
Significant references
1) Technical articles, conference papers & books
[1.1] A. Artieri, S. Kritter, F. Jutand and N. Demassieux, "A one chip VLSI for real time two-dimensional discrete cosine transform," 1988., IEEE International Symposium on Circuits and Systems, 1988, pp. 701-704 vol.1, doi: 10.1109/ISCAS.1988.15022.
[1.2] STV3200 DISCRETE COSINE TRANSFORM (DCT) Datasheet
[1.3] A. Artieri and O. Colavin, "A chip set core for image compression," in IEEE Transactions on Consumer Electronics, vol. 36, no. 3, pp. 395-402, Aug. 1990, doi: 10.1109/30.103150.
[1.4] STi3400 MPEG/H.261 VIDEO DECODER DATASHEET
[1.5] STi3500 MPEG2 CCIR 601H.261 VIDEO DECODER DATASHEET
[1.6] Image and Video Compression standards: algorithm and architectures, Bhaskaran, Vasudev, Konstantinides, Konstantinos, ISBN 978-1-4615-6199-6, 1995
[1.7] K. Konstantinides and V. Bhaskaran, "Monolithic architectures for image processing and compression," in IEEE Computer Graphics and Applications, vol. 12, no. 6, pp. 75-86, Nov. 1992, doi: 10.1109/38.163627.
[1.8] STi70000 press release
[1.9] STi71000 datasheet
[1.10] STi7200 datasheet
[1.11] STi7108 datasheet
[1.12] STi7108 processor with 3D graphics made public.
[1.13] A chip set for MPEG-2 video encoding; Proceedings of the IEEE 1995 Custom Integrated Circuits Conference, 1995
[1.14] STi3240 MPEG1/H261 datasheet
[1.15] STi3208 DCT chip datasheet
[1.16] STi3520A MPEG2 chip datasheet
[1.17] STi5600 MPEG2 chip datasheet
[1.18] STi5610 MPEG2 chip datasheet
[1.19] F. Jutand, N. Demassieux, G. Concordel, J. Guichard and E. Cassimatis, "A single chip video rate 16×16 discrete cosine transform," ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986, pp. 805-808, doi: 10.1109/ICASSP.1986.1169147.
[1.20] N. Demassieux, G. Concordel, J. Durandeau and F. Jutand, "An optimized VLSI architecture for a multiformat discrete cosine transform," ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1987, pp. 547-550, doi: 10.1109/ICASSP.1987.1169851.
[1.21] Tomasko, M., Buchhauser, D., Bushroe, M. et al. The Descent Imager/Spectral Radiometer (DISR) Experiment on the Huygens Entry Probe of Titan. Space Science Reviews 104, 469–551 (2002). https://doi.org/10.1023/A:1023632422098
[1.22] Report of the meeting of the DCT chip manufacturers in Martlesham UK 24-25 May 1988, CCITT SGXV Document #349, Working Party XV/1 21 June 1988, Specialists Group on Coding for Visual Telephony
[1.23] An Integrated MPEG-1 and MPEG-2 Decoder, IEEE International Conference on Consumer Electronics, 21-23 June 1994 https://ieeexplore.ieee.org/document/582280
[1.24] Progress on Inverse DCT Accuracy Specifications, C.C.I.T.T. Study group XV WP XV/1 Specialist Group on Picture Coding
[1.25] Matrix multiplier transform analysis and design of a general purpose transformer; Signal Processing, Volume 28, Issue 2, August 1992, Pages 153-162 https://doi.org/10.1016/0165-1684(92)90032-R
2) Online information and citations
[2.1] https://en.wikipedia.org/wiki/Motorola_68020
[2.2] https://techmonitor.ai/technology/sgs_thomson_debuts_one_chip_hdtv_system
[2.3] Leonardo Chiariglione awards https://leonardo.chiariglione.org/public-life/awards/
[2.4] see pdf and https://www.eetimes.com/apis-help-stmicroelectronics-grow-set-top-box-business/#
[2.5] see pdf and https://www.edn.com/sgs-thomson-microelectronics-named-number-one-mpeg-decoder-supplier-by-dataquest/
[2.6] see pdf and http://www.fundinguniverse.com/company-histories/stmicroelectronics-nv-history/
[2.10] http://ciclops.org/ir_index/208/In-Orbit?js=1
[2.11] https://en.wikipedia.org/wiki/Allan_Alcorn
[2.12] https://www.amazon.com/dp/B096G6TSF9/
[2.13] Computer History Museum, Oral History of Allan Alcorn Recorded: April 26, 2008 and May 23, 2008 Mountain View, California https://archive.computerhistory.org/resources/access/text/2012/09/102658257-05-01-acc.pdf
3) STMicroelectronics documents
[3.1] De la Transformée en Cosinus Discrète aux Processeurs Applicatifs Multimédia, Alain Artiéri, Septembre 2007, STMicroelectronics presentation
4) Patents
[4.1] USRE34734 "Integrated digital signal processing circuit for performing cosine transformation”
[4.2] US4872134A "Signal processing integrated circuit for row and column addition of matrices of digital value”
[4.3] US5600837A "Multitask processor architecture having a plurality of instruction pointers”
[4.4] Integrated signal processing circuit for online and column summation of digital value matrices https://patents.google.com/patent/FR2608802B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old
[4.5] Transposition memory for data processing circuit https://patents.google.com/patent/FR2617621B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old
[4.6] Method and apparatus for addressing a memory area of an MPEG decoder
[4.7] publication number US6081298A MPEG decoder with reduced memory capacity
[4.8] publication number 0 298 002 of 29.06.1988 Memoire de transposition pour circuit de traitement de donnes
[4.9] publication number FR2649226A1 BREWING CIRCUIT OF DATA
[4.10] publication number FR2650462B1 Device for converting a line scanning into a scanning in vertical saw teeth by bands; https://patents.google.com/patent/FR2650462B1/en?q=H04N3%2f30&inventor=alain+artieri&sort=old
[4.11] publication number EP0368731B1 Process and circuit for image representation signal filtration
[4.12] Circuit pour effectuer une transformation linéaire sur un signal numérique Patent issuer and number eu EP 0241352 A1 https://patents.google.com/patent/EP0241352A1
[4.13] Additionneur binaire comportant un opérande fixé, et multiplieur binaire parallèle-série comprenant un tel additionneur Patent issuer and number eu EP 0262032 B1 https://patents.google.com/patent/EP0262032B1
[4.14] Integrated digital signal processing circuit for performing cosine transform
[4.15] Signal processing integrated circuit for row and column addition of matrices of digital values
5) Honors
[5.1] Two European IT Prizes Awarded to STMicroelectronics
6) MPEG mentions
[6.1] Chiariglione and the birth of MPEG https://www.chiariglione.org/public-life/media/leonardo_ieee/prof.htm
7) Letters of support
[7.1] By Alain ARTIERI, the creator of MPEG2 chips in ST.
[7.2] Endorsement by Nicolas Demassieux, Senior Vice President of Research, Orange, assistant professor at ENST (now Telecom ParisTech) mid 80s.
[7.3] Endorsement by Leonardo Chiariglione, PhD Eng, the father of MPEG
[7.4] Endorsement by Professor Emeritus Fabio Rocca, Politecnico di Milano, pioneer of motion estimation in 1969
[7.5] Endorsement by Hisafumi Yamada, former Sony US TV CTO
[7.6] Endorsement by Sylvain Kritter, Product Director VIZYON
[7.7] Endorsement by Professor Emeritus Virginio Cantoni, University of Pavia. Also Fellow of the IAPR since 1994 and Life Fellow of the IEEE (he was Fellow since 1997). In 2006 the President of the Italian Republic conferred to Professor Virginio Cantoni the title of ‘Commander of the Order of Merit of the Italian Republic’.
[7.8] Endorsement by Professor Emeritus Mariagiovanna Sami, Politecnico di Miano. Past Full Professor, Digital Systems. IEEE Life Member. Minerva prize for woman scientists, the Seymour Cray prize for contributions in the area of parallel processing, the Herbert A. Simon Gold Medal, assigned by the Society for Design and Process Science. Cavaliere della Repubblica Italiana (knight of the Italian republic). Member of the Italian National Science Academy (“Dei Quaranta”).
[7.9] Endorsement by Professor Andrea Basso who directly contributed to MPEG standards since 90s while in EPFL and in AT&T Bell labs and AT&T Research Laboratories, Middletown, NJ, USA
[7.10] Endorsement by Allan Alcorn, Apple Fellow in the late 80s in the Advanced Technology Group at Apple Computer, Cupertino USA
[7.11] Endorsement by Jeff Cooper, former Senior R&D Engineer at Thomson Multimedia - RCA in the 90s responsible for DirectTV MPEG2 set top box design based on STi3500.
[7.12] Endorsement by Shigenobu Minami, former R&D team leader at Toshiba for MPEG4 standardization and LSI planning in the 90s.
Supporting materials
{{{support}}}
