Milestone-Proposal:MPEG integrated circuits: Difference between revisions

From IEEE Milestones Wiki
No edit summary
No edit summary
 
(129 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Proposal
{{Proposal
|docketid=2021-10
|litigation=No
|litigation=No
|more than 25 years=Yes
|more than 25 years=Yes
Line 9: Line 10:
|section is taking responsibility for plaque=Yes
|section is taking responsibility for plaque=Yes
|a11=Yes
|a11=Yes
|a3=1986
|a3=1984-1993
|a1=Multimedia Integrated Circuits for MPEG, 1986
|a1=MPEG Multimedia Integrated Circuits, 1984-1993
|plaque citation=SGS-THOMSON (now STMicroelectronics) pioneered the family of multimedia integrated circuits, accelerating Moving Picture Experts Group (MPEG) standards. Discrete Cosine Transform designed with ENST (now Telecom ParisTech) was the first. Innovative memory reduction techniques, highly optimized dedicated hardware accelerators, super-integration of heterogeneous processor units achieved significant performances and cost reduction with minimized energy consumption. These integrated circuits were the key enabler of digital multimedia services to entertain end-users in everyday life.
|plaque citation=Beginning in 1984, Thomson Semiconducteurs (now STMicroelectronics) developed multimedia integrated circuits, which accelerated Moving Picture Experts Group (MPEG) standards. By 1993, MPEG-2 integrated decoders -- including innovative discrete cosine transform (developed jointly with ENST, now Telecom ParisTech), bitstream decompression, on-the-fly motion compensation, and display unit -- were announced in one silicon die: the STi3500. Subsequent MPEG-2 worldwide adoption made compressed full-motion video and audio inexpensive and available for everyday use.
|a2b=France Section
|a2b=France Section
|IEEE units paying={{IEEE Organizational Unit Paying
|IEEE units paying={{IEEE Organizational Unit Paying
Line 36: Line 37:
}}
}}
|a2a=(45.203333, 5.695833)
|a2a=(45.203333, 5.695833)
|a7=The Grenoble site is of premier importance for STMicroelectronics since was owned by THOMSON SEMICONDUCTEURS and then merged with SGS in 1987 to form SGS-Thomson (now STMicroelectronics). Nowadays is the biggest R&D site of STMicroelectronics, hosting many product divisions from the three Product Groups of the company, including silicon and software design, test, and advanced packaging developments. The site were MPEG chips were conceived and designed
|a7=The Grenoble site is of premier importance for STMicroelectronics. It was owned by THOMSON SEMICONDUCTEURS, then merged with SGS in 1987 to form SGS-Thomson. Nowadays is the biggest R&D site of the company, hosting many product divisions from the three Product Groups of the company, including silicon and software design, test, and advanced packaging developments. In this site MPEG chips were conceived and designed
|a8=Yes, even if part of them were restructured they still exists @ STMicroelectronics, 12, rue Jules Horowitz F-38000 Grenoble, France
|a8=Yes, even if part of them were restructured they still exists @ STMicroelectronics, 12, rue Jules Horowitz F-38000 Grenoble, France
|mounting details=Plaques will be installed in a public place at the main entrance as shown below. It will be close to ST security personnel who are is monitoring entrance 24 hours in a day, every day in a year. Very close as indicated there are other public roads.  Plaque will be placed at 12, rue Jules Horowitz F-38000 Grenoble, France.  GPS coordinates are: 45°12’12” N ; 05°41’45” E (45.203333, 5.695833). Therefore, visitors will not need to be escorted by ST guards to look the plaque or to take a snapshot of it.  The milestone plaque will be installed right in front of STMicroelectronics, main entrance access. It is where employees, visitors and customers must pass to get into the ST Grenoble offices on daily basis.  It is monitored continuously 24h/7d by ST security human resources and surveillance camera. It is a public place, right outside ST site restricted perimeter.
|mounting details=Plaques will be installed in a public place at the main entrance as shown in figure 1. It will be in front of company security personnel who are monitoring entrance 24 hours in a day, every day in a year. Very close as indicated there is the public road.  Plaque will be placed at 12, rue Jules Horowitz F-38000 Grenoble, France.  GPS coordinates are: 45°12’12” N ; 05°41’45” E (45.203333, 5.695833). Therefore, interested visitors will not need to be escorted by the company security personnel to watch the plaque or to take a photo with it.  The milestone plaque will be installed right in front of the main entrance access. It is where employees, visitors and customers must pass through to get into the Grenoble offices on daily basis.  It is monitored continuously 24h/7d by security human resources and surveillance camera. It is a public place, right outside the site restricted perimeter.
|a9=It is protected by ST security human resources and with surveillance camera, 24h/7d.  It is publicly and easily accessible from 12, rue Jules Horowitz F-38000 Grenoble, France.  The site can be reached also from Lion Airport, close to Grenoble as shown in figure 2.
|a9=It is protected by company security human resources and with surveillance camera, 24h/7d.  It is publicly and easily accessible from 12, rue Jules Horowitz F-38000 Grenoble, France.  The site can be reached also from Lion Airport, close to Grenoble as shown in figure 2.
|a10=STMicroelectronics (Grenoble 2) SAS
|a10=STMicroelectronics (Grenoble 2) SAS
|a4=The technology objective of STMicroelectronics was to conceive, ahead of MPEG standard, and introduce innovative hardware solutions able to achieve several order of magnitude speed acceleration, compared to unfeasible full software implementation of MPEG decoding and encoding functionalities, by introducing for the first time  memory compressions and reductions, dedicated hardware implementation in order to save silicon area per technology node, super integrate new functionalities in a less than a watt power envelope.  
|a4=[[image:Figure1 Placque installation.jpg|thumb|Figure 1 Candidate place for plaque installation]]
The DCT circuit, the first in the family, was so reliably designed and produced that was subsequently used in the Cassini-Huygens probe for transmitting compressed images of Saturn during 20 years.
Specific objectives, that motivated STMicroelectronics (short name ST) to pioneer Mpeg family of System on a Chips and associated CMOS silicon technology development to implement it and starting since 1987 and onward, were:
1) To create reliable and low complex CMOS building (integrated circuits) blocks at production maturity to dramatically accelerate real time MPEG video decoding and encoding functionalities at increasing image resolutions and frame rates.


2) To super-integrate such building blocks in more complex integrated circuits and SoCs to accelerate MPEG 1, 2, 4, H264, HEVC and subsequent standardized specifications, at the same time they were under development. In other words before the standards were officially frozen and went public.
[[image:Figure2b.jpg|thumb|Figure 2 How to get the plaque from Lyon airport]]


3) To achieve lowest on chip memory footprint and bit-depth of computing data paths to dramatically save silicon area given the existing (µm) CMOS technology constraints for mass production and reliable manufacturability with marginal chip defectivity.
The importance of the proposed milestone will be described as follows: section 1 introduces the object; section 2 lists the technical innovations; section 3 discusses the benefits of the innovations and section 4 their achievements; section 5 provides the historical background about when the innovations were introduced; Section 6 reports the public recognition of the integrated circuits by specialized journals. Finally, section 7 is for the supporting letters provided by world renown experts.


4) To achieve minimal power consumption, to and below 1W across different silicon technologies generations and starting from 1.2um silicon gate manufacturing processes. Successive lower dimension lithography, were needed considering an increasing need to super integrate functionalities, heterogeneous hardware accelerators, microprocessors, graphic engines, crypto engines, a rich set of peripherals and interfaces etc.
1) Object of the milestone proposal


5) To remove any need of costly and bulky heat-sinks as well as complex cooling mechanisms to offer an easier viable integration into thinner video equipment manufactured systems.
Starting from 1984 new digital multimedia technologies were conceived and implemented by SGS-THOMSON on different CMOS silicon technologies. They were dedicated processors very optimized in area and power consumption. They were hand-designed and produced in high volume, with high yields and reliability to achieve lowest cost. Next, they were incrementally super-integrated in one silicon die to perform multimedia video decoding anticipating specifications as established by MPEG in the 1992 onward. Several innovations were introduced to reduce the complexity due to the super integration. They anticipated the acceleration of the MPEG1, 2 and subsequent standards. Since no digital multimedia content services for moving pictures were existing at that time, they set the hardware to introduce them. They facilitated widespread adoption by the end users and eased switch-off from analog decoders to digital ones. A special mention is for the STi3500 chip which was the world first MPEG2 decoder. The chip die is shown in the figure 3.


6) To dramatically reduce as low as possible video communication bandwidth between the chip and the external DRAM memories to minimize memory requirements of those chips and associated costly power consumption.
[[image:Figure 3.png|thumb|Figure 3 Silicon die of the STi3500, the first MPEG2 video decoder]]
 
2) Technical innovations introduced by the milestone


7) To control those chips with very simple and low computing power 8bits micro controllers and incrementally as requirements will grew with 32 bits: this to support flexibility that requires further software computing needs to offer un precedented user experience. Including the use of advanced graphics to ease content accessibility and fruition.
The circuits introduced several innovations. They were:


8) To enable viable, widely adopted, broad range of diversified digital multimedia applications and services, implying affordable software development and deployment to a vast range of end users at worldwide level with different needs.
a) Block based pixel memory management to achieve closely the peak computational power.


In the following there is the summary of achievements by STi family in the present milestone proposal. To meet the above-mentioned challenging objectives and long-term technology vision, ST, since 1986, has developed following key technologies and chips. A few but very relevant technologies and chips will be described, because the family is too huge to report completely in this document.
b) Compressed MPEG decoding on the fly to reduce video storage in RAM.
1) Seminal work on ST CMOS technologies, started at ENST [1.24] [1.25] [7.0] with Thomson Semiconductors (now STMicroelectronics), produced the architecture of an economically reasonable chip implementing various sizes DCT 16x16, 16x8, 8x8, 8x4, 4x4. The main features of the chip under development with THOMSON SEMICONDUCTEURS in 2 µm first and then 1.25 µm CMOS technology were 70,400 Transistors, 25 mm2, Internal clock 13.5 MHz, Direct/Reverse DCT, 8 bits pixels, 16 bits internal accuracy. The chip was sampled by THOMSON SEMICONDUCTEUR in October 1987.
 
2) STV3200 [1.1] [1.2] was a dedicated integrated circuit to accelerate the discrete cosine transform (DCT). The two-dimensional forward DCT (FDCT) and inverse DCT (IDCT) was implemented for various image block sizes and the pixel rate was up to 15.0 MHz The circuit architecture was fully bidirectional with a 9-bit magnitude pixel data bus and a 12-bit magnitude coefficient data bus programmed as input or output depending on the selection of Forward DCT or Inverse DCT function. DCT was so complex that to be implemented on a CPU required about 1 billion additions per second (1 giga Hertz operations per second) to process the moving pictures in real time. CPUs and CMOS silicon process were unable to achieve such a very high frequency, therefore implying the computing power of fifty Motorola 68020 [2.1], one of the faster micro-processors in the 1980s. The integrated circuit went officially in production at the end of 1987 and was a world premiere product. Since 1988, this component was successfully sold by SGS-Thomson. It implemented key patents [4.1][4.2] for rearranging, permuting or selecting data according to predetermined rules, for changing the order of data flow, e.g. matrix transposition, LIFO buffers; Overflow or underflow handling that were key for DCT and IDCT implementations.
c) Frame buffer lossy compression to co-decompress on the fly MPEG2 video
Innovative memory footprint reduction and distribution techniques were used since the hardware design did not use the traditional architecture (such as a unique multiplier that would read/write in a large memory), but a bit-serial pipelined architecture, where memory was distributed across the chip, to feed the 16 bit/serial dedicated multipliers that the chip had.
 
The STV3200 chip was used in the DISR (descent imager) [1.26] [2.9] [7.0] of Cassini-Huygens probe that was launched in 1997 and has transmitted magnificent images of Saturn until 2017. All the wonderful images that are available in this site [2.10] have been processed through the chip!
d) Unified memory controller allowed to plug and play additional MPEG and other hardware clients.
Design of the STV3200 happened during the December 1986 to August 1987 period (product sampling around October 1987) as mentioned at the end of [1.25]. As said commercialization of the STV3200 started in 1988 while on November 1988 there was the publication of the H261 standard (1st standard using DCT for real time video coding) and on May 1988 there was the 1st meeting of the MPEG standard (that was published in 1993).
 
3) Benefits of the innovative solutions subject of the milestone
 
The benefits of integrated circuits which embodied innovations described in section 2, were:
 
a) Transistor count minimization to ease super integration in one silicon die.
 
b) Hand crafted hardware design methodology to produce very tiny circuits.
 
c) Energy consumption to less than one watt across chip generations.
 
d) Reliability to produce chips for widespread adoption
 
e) Resilience to faults.
 
4) Technical achievements of the proposed milestone
 
The achievements of the processors of the milestone were on the following key indicators:
 
a) Performances (highest degree of acceleration of MPEG functionalities).
 
b) Manufacturability (reliability, repeatability, high volumes, robustness, high yields)
 
c) Super integration of RAM.
 
d) Silicon node scaling from 2µm to 28nm FDSOI (fully depleted silicon on insulator).
 
e) Flexibility (VLIW processors,  CPUs, 3D programmable graphic)
 
f) Security (data and memory protection encryption).
 
g) Efficiency (mW/sqmm/$, reduced memory bandwidth, and reduced calculations bit-depth)
 
h) Tiny dissipation (avoided costly and bulky heat-sinks and complex liquid cooling)
 
l) Costs (cheap solutions for end users, eased TV set manufacturers to achieve large scale savings)
 
m) Ecosystem (key enabler of broad range digital video applications and digital multimedia vertical services, facilitated the end user to enter the new digital entertainment era)
 
5) Historical background and contributions
 
THOMSON Semiconductor (headquarter in Grenoble, France) was merged with SGS (headquarter in Agrate Brianza Italy) in 1987; the merge was named SGS-THOMSON; then the company was renamed in 1998 as STMicroelectronics (today name).
 
In the late 1980s ISO/IEC experts felt the need to standardize motion picture decoding. Therefore, MPEG committee had first meeting in 1988. MPEG1 standard was finalized in 1992 and MPEG2 was finalized in 1994.
 
All MPEG integrated circuits, object of this proposed milestone, are developed fully in-house SGS-THOMSON Grenoble and intellectual properties are owned by STMicroelectronics.
 
Next points a) to d) details that the technology subject of this proposed milestone, the MPEG circuits :
 
 
a) In 1984, THOMSON Semiconductors started to develop own Discrete Cosine Transform (DCT). This effort was patented twice, therefore the company decided to launch an own chip design and production project in 1986.
 
b) In 1986 ENST (now Telecon ParisTech) had in parallel a DCT experiments based on a less hardware efficient algorithm than THOMSON designed.
 
c) In 1987, THOMSON Semiconductor and ENST, even if not essential to DCT chip, decided to cooperate on redesigning a part of the whole DCT chip. The operative part with a serial architecture was conceptualized by ENST and designed by THOMSON in hardware.  
 
d) In 1988 SGS-THOMSON had following products ready for production
 
i. STV3200 the multi format DCT, not MPEG compliant
 
[[image:Figure3 dct chip.jpg|thumb|Figure 4 DCT chip package]]
   
   
Figure 3 DCT chip
[[image:Stv3200litography.png|thumb|Figure 5 DCT chip die, 1987, 40sqmm, CMOS 1.2µm, 115,000 transistors]]
3) STi3220 [1.3] [1.4] [1.5] was the second member of the family, a real time motion estimation integrated chip dedicated to motion estimation at video rates. The chip was optimized to compute the displacement vector of 8x4n or 16x4n pixel blocks in a search window, between temporally distant pictures, defined by a maximum horizontal and vertical displacement of +7/-8 pixels corresponding to 256 different vectors. The chip computed 256 distortion for each block according to the MAE (mean absolute error) criteria. The motion estimation required about 4 billion of additions per second (4 giga Hertz operations per second). If it would be implemented on CPU it would require the equivalent computing power of several hundred of Motorola 68020. Therefore, it was unfeasible in software. Its development started in 1988, went in production and be sold by SGS-Thomson since 1989. This component was another world premiere product in conjunction with STV3200. Furthermore, the Motion estimation processor was a key building block to accelerate more complex MPEG-2 Video Encoders [1.17]
 
4) Super integration of those fundamental functions with others fundamental building blocks (e.g. scalar quantization and de-quantization, entropy coding and decoding, motion compensation, unified memory management), was the next vital step required to manufacture reliable system solution for mass production while before they were consisting of few chips. In this respect [1.20] presents the deployment of STV3200 and STi3220 in a complete video codec hardware architecture for high pixel rates addressing standard (SD) and high definition (HD) picture formats.
ii. STV3208 8x8  DCT, MPEG compliant. Note that DCT into modern chips was just 0.05% of total chip area
5) STi3240 [1.18] was the 1st MPEG1/H261 chip with STV3208 (DCT processor) [1.19] supporting video decoding schemes up to 10 Mbit/s of the ISO/MPEG future (at that time under active definition) video standard and CCITT/H261 recommendation at video rate (352x288 pixels at 30fps). It required between 512 KBytes up to 4Mbytes of external DRAM with a 8/16 bits microprocessor interface. External DRAM was a scarce and costly resource, dramatically impacting the overall power consumption and bill of material. Therefore, the use of memory reduction techniques to support high chip calculations throughput and minimized data bandwidth between the chip and the memory were conceived for the first time and implemented.
 
Next points aim to explain how MPEG chips were developed.
 
e) First MPEG1/H261 standard compliant . First solution was composed by two integrated circuits: the picture motion compensation, huffman decoding, dequantization, memory control (into STi3240) and the 8x8 DCT (STV3208). Second solution was the super integration of the two circuits into a single chip named STi3400 .
 
[[image:figure6mpeg1.png|thumb|Figure 6 The discrete MPEG1 decoder system with the STV3208 and STi3240]]
 
f) World first STi3500 MPEG2 decoder circuit. It integrated STV3208 with MPEG2 compliant motion compensation. RCA, later acquired by Thomson Consumer Electronics, was creating the world first Digital Satellite TV broadcast service but missed the video decoding function. STi3500 was delivered to RCA (ahead of the competition) and enable decoding of the DirectTV digital multimedia broadcast. Next STI3520A was the world first single chip for MPEG 2 video decoding in only 16 Mbit RAM super integrated with the world first MPEG MP3 Audio decoder (STi4500).
 
g) First HDTV decoder circuit with RAM reduction algorithms integrated: the STi7000 in 1998. Next versions STi7100, STi7200 integrated 3D shader-based graphics engine and powerful micro controllers. STi7108 had dual CPU host processors with embedded memory. It  was the first circuit in the world to integrate 3D OpenGL-ES 2.0 and OpenVG vector graphics.
 
h) STi5500 was the first (after signal demodulation circuit) complete decoder for set-top boxes and DVD players. Capable to demux MPEG2 transport stream. Memory compression algorithms reduced its RAM needs.  
 
i) World first STi5505, that integrated all the back-end functions of a DVD player. More powerful than the STi5500 and capable to support MPEG2 program stream demultiplexing inside. STi5500 won from a field of nearly 300 nominations, the prestigious European IT Prize awarded by the European Commission in conjunction with Euro-CASE, the European Council of Applied Science and Engineering. Its further evolutions were STi5600 and STi5610.
 
l) Further generations. Many other integrated circuits were designed over 30 years, as shown in figure 7. Complex operative systems such as Linux and Windows were incrementally supported. These processors also pioneered TV Cable transmission with Scientific Atlanta (which was acquired by Cisco later). They accelerated the introduction of CD-ROM, DVDs, Set Top Box and digital TVs home systems. The analog to digital entertainment switch was therefore supported by mass production circuits and doable in everyone home.
 
[[image:Figure5 chiproadmap.png|thumb|Figure 7 The MPEG decoders generation by generation]
 
Since the functionalities they integrated, these chips were deployed also into smartphones since early 2000s. The Nomadik chip family (STn8800/10/15 and subsequent) was derived to accelerate multimedia into mobile phones. The pioneering 8x8 MPEG DCT chip, the STV3208, represented less than 0.1% of the die area of the SoC as shown in figure 8.
   
   
Figure 4 decoder system, STV3208 was DCT co processor
[[image:Figure8_nom.jpg|thumb|Figure 8 Super integration of STV3208 with other multimedia functions]]
6) Then the super integration of way started with active development of System of Chip. Earliest members of the STi digital multimedia chip family for mpeg moving picture processing were: STi3400, STi3500, STi3520A. Other early examples were STi7000, STi5000 (and the very many others that followed). They were the first able to offer a dramatic reduction in silicon and package size, power consumption, with higher level of transistors integration, functionalities, and lowest cost of equipment’s. Such equipment’s were made previously by many un-optimized discrete signal processors (DSP) and which were placed into many more boards. They far from being manufacturable viably and affordably for MPEG-1 and MPEG-2 standards deployment at end user side.  STi chips were very instrumental to accelerate the introduction to the end users of CD-ROM, DVDs, Set Top Box and digital TVs that became ubiquitously adopted throughout the entertainment world and in everyone home (and mobile phones) across 3 decades.
 
7) Inside those chips’, innovative and advanced reduction techniques of external memory bandwidth were conceived and implemented first. They were key to lower power consumption to and under 1W. In that respect several key methods are described in the following subsections:
ST manufactured more than 500 million (cumulative number) of these integrated systems worldwide as shown in Figure 9.  
a) block based pixel memory addressing and communication [4.3] allowed to minimize the cost of external memory page openings and closures, distributing the costly overhead per block size (composed by several pixels) instead of per single or per few pixels. In turn this allowed to avoid hardware pipeline stalls resulting in using peak performances and resulting in a full motion interactive frame rates, clear picture decoding, free from block artefacts
 
b) picture decoding on the fly [4.4] was a break-through technology to decompress and motion compensate at the same time while displaying MPEG B-pictures. This to avoid inefficient and further storage of them in the external frame buffer, thus resulting in a dramatic reduction of the external memory to only 16 Mbits for SD definition [1.20]
[[image:Figure7 cumulative shipments.png|thumb|Figure 9 Cumulative shipment of SGS-THOMSON MPEG devices]]
c) frame buffer compression [1.11] [4.5] was another key technology used to co-decompress on the fly I , P and B MPEG pictures by halving the associated external DRAM memory and the memory bandwidth without annoying picture artefacts. Indeed, in STi solution the quantization noise was cleverly masked by MPEG coding noise, resulting in not human perceivable loss of quality. This allowed to reach 8 Mbits DRAM footprint in SD enabling super integration of 1 bit transistor per cell DRAM on the same chip, in a more competitive way than integrating 6 transistor per cell SRAM. Since then STi did not require any additional external memory for the MPEG SD video decompression task. Another kind of memory reduction was applied to HDTV decoders with even further benefits. [1.23] was novel method to reduce the external memory needed by STi MPEG-2 HDTV decoder architecture. The total amount of memory was reduced from 96 to 32 Mbits preserving a good picture quality. Furthermore, it required low hardware complexity increases of less than 5%, in 0.35 pm technology, the total decoder silicon area with respect to the standard decoder.
 
8) The STi implementation of the ST comprehensive vision to create the digital Multimedia domain was made possible with those and many other chips.
A rich digital consumer ecosystem has been created thanks to those circuits. It was composed by many operators, equipment manufacturers, software developers and conditional access companies as shown in the next figure 10.
However, the pioneering development work began since 1986 [1.24] [1.26] among different and key French partners:
 
1) Thomson Semiconductor (merged later into SGS-THOMSON, now STMicroelectronics) launched innovative circuit architecture studies for integrated for the Discrete Cosine Transform since 1986
[[image:Figure8 ecosystem.png|thumb|Figure 10 Worlds wide ecosystem for digital multimedia services enabled by SGS-THOMSON integrated circuits]]
2) ENST Paris (now Telecon ParisTech) was very active and interested since 1986 in integrated circuit architectures for DCT [1.24] [1.25] single chip for video rates.
 
3) Rennes Electronics Laboratory, expert in the field of image compression, built image transmission equipment between studios and digital recording.
7) Public recognition of the integrated circuits
Under the finance support of the DAII (French State Agency) these three laboratories collaborated to form a team responsible for realizing the first DCT component (STV3200) and the Motion Estimation chip (STi3220).  The DCT and Motion estimation chips existed thanks to the strong collaboration between SGS-Thomson and the public research lab of ENST.
Specialize magazines reported the leadership of the MPEG circuits.
Thus, on 1988, SGS-THOMSON finalized these seminal developments in production (but with background developments since 1986 [1.24] [1.26] in Thomson Semiconductor) therefore creating the wave of key circuits and SoC for MPEG. By consequence it mastered at industrial level initially the 2 key components which made possible and economically viable the realization of more complex, reliable, ready for mass production, yet low cost and low power MPEG-1 and MPEG-2 encoders and decoder systems for the (mid 80s) for the not existing digital multimedia television markets.  
 
9) Apple Computer was looking for a technology partner in the field of image compression. Since aware of above-mentioned developments, contacted SGS-THOMSON in 1988. The 68020 microprocessors was used from March 1987 to March 1992 and only used twice, once in the Macintosh II as a high-end processor, and then again with the Macintosh LC, as a low-end processor. However, it was too slow and inadequate to support video decoding functions as for MPEG decoding. From this key technical limit, a collaboration with SGS-Thomson was started, the first in its kind, to study and develop highly integrated set of components to provide video functions for Apple's microcomputers. That was a very advanced concept for that time. Therefore SGS-THOMSON entered a partnership with Apple. Thus, based on SGS-THOMSON understanding of 68020 limits and the contribution of innovative chip solutions from SGS-THOMSON, the first architecture of an integrated system for image compression was defined to support the MPEG1 standard: the STi3240 [1.18], STi3400 [1.6].  STi3400 was a real time video, super integrated with DCT, decompression integrated processor supporting MPEG-1 and H.261 standards. The digital output was for PAL 50Hz and NTSC 60 Hz interlaced displays.
a) EETimes article “In the first shift or wave (referred to in the set-top box market), which began about 1994, the boxes were simple channel-hopping devices for satellite and cable TV providers, Hatch explained. (STMicroelectronics's MPEG-2 video decoders were used then in the boxes manufactured by Thomson's RCA subsidiary.)”.
10) Next leveraging his expertise, SGS-THOMSON started a background work with RCA, which ended in 1992 with the development of an MPEG2 video decoder, STi3500 [1.7] and subsequent STi3520 [1.8]. Those were other essential components at the heart of the revolutionary multimedia digital television services introduced firstly from the USA Digital Satellite DirectTV firm. SGS-THOMSON succeeded in delivering the first MPEG2 decoder in the world, much ahead of the Californian start-up C-CUBE. It must be noticed that C-Cube was funded on 1988 when SGS-THOMSON had already above-mentioned chips in production and when Thomson semiconductor started on 1986.  
 
11) STi3500 was a real time video decompression integrated circuit supporting MPEG-1 and MPEG-2 standards at video rates up to 720x480x60Hz or 720x576x50Hz requiring minimal support from an external 8bit microcontroller (ST8) used only to initialize the chip, reducing the complexity of software and associated code footprint. It required only 1W of power consumption. The STi3500 was a success since achieved a turnover greater than $ 100M in less than 3 years as a sign of market adoption. In that chip STV3200 (which was super integrated) represented only 10% of the total silicon area. STi3520 on top of STi3500, used only 16Mbits memory and integrated audio decoder compliant with MPEG layers I and II supporting sampling rates of 32, 44.1 and 48 KHz.  STI3520A [1.20] was a single chip for MPEG MP@ML video decoding in only 16 Mbit RAM integrated with STi4500 MPEG L1 and L2 Audio decoder and integrating on screen display generator. The chip was using 0.5µm CMOS SGS-THOMSON technology
b) EDN article on 1996 “Dataquest has confirmed SGS-THOMSON Microelectronics (Lincoln, MA) as the world's leading supplier of MPEG decoder ICs in 1995.
12) STi7000 [1.12] [2.2] was an integrated system for High-Definition Television (HDTV), which combined an MPEG-2 decoder with a display and format converter onto one chip in 1998. As already introduced in point 5) the most advanced trick was the frame buffer memory reduction through tiny on chip compression engine based on scalar adaptive quantization allowing full HDTV pictures to be decoded and displayed with only 64 Mbits of external memory instead of 128 Mbits. Designed for use in HDTV and other digital TV receivers, set top boxes and PCs, the STi7000 was developed in collaboration with Thomson Multimedia, a strategic partner with whom SGS-THOMSON shared a joint design center in Grenoble, France. The chip incorporated all the 18 video formats defined by the ATSC (Advanced Television Systems Committee) and Grand Alliance specifications. STi7000 supported video rates of up to 1920 x 1088 x 30Hz interlaced or 1280 x 720 x 60Hz progressive. Built in 0.35-micron SGS-Thomson HCMOS6 silicon technology, the STi7000 also included interfaces for a host microcontroller, local SDRAM, standard or high-definition video output and D1 digitized video input. Next versions STi7100 [1.13] STi7200 [1.14] also integrated graphics engine and power powerful micro controllers. STi7108 had dual ST40-300 CPU host processors linked to a 256K L2 cache giving up to 2000 DMIPS performance and a total of 4000 DMIPS. A 3D graphics engine [1.15] enabled advanced Internet content and high-performance gaming. It was the first set-top box IC [1.16] in the market to combine 3D OpenGL-ES 2.0 graphics, Ethernet, USB and e-SATA interfaces to connect Internet devices, DVR storage or external Flash or hard-disk (HDD) drives.  
 
13) The STi5500 was the first member of back-end decoders for set-top boxes and DVD players. Compliant with MPEG-2 to decode its transport stream and convert it into sounds and images that sent to loudspeakers, monitors, TV screens and similar human interfaces. The STi5500 was the first integrated circuit in the world to incorporate not only all the circuitry required to handle all the back-end functions but also a 32-bit microprocessor. The built-in microprocessor had sufficient processing power to handle system-level functions as well as the managing the MPEG decoding function, so it eliminated the need for a separate system controller in most applications. Its super integration was the crucial factor and achieved immediate customers adoption. Since then, ST has developed even more powerful members including the STi5505, which is the first silicon chip in the world to integrate all the back-end functions of a DVD player.  
c) STMicroelectronics NV History quoting “ STMicroelectronics is the world's leading manufacturer of .... MPEG-2 decoder chips, used to provide video decompression for DVD players and digital television set-top boxes.” and “Another area in which SGS-Thomson became an early player was its development of MPEG decompression chips. The company's dedication to MPEG technology was risky, as a number of other digital video decompression schemes were competing for what promised to be a huge market in the future. SGS Thomson gained an early lead in the MPEG market. By 1993, the company had debuted its "multimedia" chip, capable of decompressing digital video files for display on a television set. This chip helped the company take a major position in the new set-top box market, starting with supplying the chip for the Hughes digital satellite television set-top box.
As demonstration of the innovation it brought in the field of digital multimedia, STi5500 won [5.1] from a field of nearly 300 nominations, the prestigious European IT PrizesPrize awarded by the European Commission in conjunction with Euro-CASE, the European Council of Applied Science and Engineering, for the development of products that are expected to play fundamental roles in helping European industry to increase its share of world markets.
 
Summarizing SGS-THOMSON STi gained its leadership position in the field of digital consumer multimedia decoder-type products for digital satellite TV and DVD players. Early members such as STV3200, STi3220, STi3240, STi3400, STi3500, STi3520A, STi7000, STi7100, STi7108, STi5500 SoC through the 80s, 90s and 2000 years continued for the development of digital multimedia applications at heart of the worldwide digital multimedia convergence of the TV services for everyday users, the world of telecommunications and the world of the PC.  
8) Supporting letters by world renown and independent experts in the field
Many other generations were manufactured across the years as shown in the next figure 5 even supporting complex operative systems such as Linux.
Please refer to part 7 of bibliography
|a6=1) Technical obstacles
 
Several obstacles were addressed. Hereafter associated solutions will be discussed in the following order: a) Silicon; b) Algorithm; c) Hardware; d) Memory reduction, e) Power consumption; f) User experience.
 
a) Silicon technology
 
One key initial obstacle was represented by the low density 2 and 1.25 um CMOS silicon technology that limited circuity complexity and super-integration. SGS-THOMSON hardware designers overcome this barrier by hand crafting the design, making it very efficient and low transistor count. A careful decision was about which functions to accelerate with respect to the ones to execute in software. As result, STV3208 needed only 115,000 transistors for 8x8 MPEG DCT. Multiple technical dimensions were addressed such as:
 
i) data precision of internal calculations, as low as 8 bits. DCT coefficients were coded with only 12bits to reduce RAM. Data have been also processed at 16 bits. This avoided 32bits integer and floating point arithmetic.
 
ii) Designed circuits to be compliant with IEEE 1180-1990, the IEEE Standard Specifications for the Implementations of 8x8 Inverse Discrete Cosine Transform.
 
iii) Internal memory distribution. Transposition of read and write memory data access avoided costly RAM. DCT separability was exploited to save silicon area. Integrated RAM was as low as 4Kbits.
 
iv) Design flexibility. Its design was capable to support specifications as established by MPEG-1, 2, 4, H.264 and HEVC standards with a plug and play of new accelerators thanks to the unified memory controller architecture.
 
b) Hardware technology
 
The software computing bottleneck was solved by hardwiring DCT, variable length coding, motion compensation, de quantization processing functions. It removed the need for any time critical handshake between the external micro controller and the integrated circuits. Achieved performances order of magnitude higher than pure software execution.
 
d) Memory reduction technology
 
A major obstacle was the cost of the RAM . Super integration required unified memory management. Many accelerators super integrated were contending memory access simultaneously. Decoding PAL and NTSC resolutions was achieved into only 16 Mbits with several memory tricks. Novel memory reductions technique fixed the obstacle allowing MPEG SD decoding as low as 8Mbits and HTDV in 32Mbits. In figure 11 the picture of the first consumer high-definition MPEG video chip. The Motion compensation unit with memory reduction is the block on the bottom right corner, with a raw of memory at the bottom and two almost parallel rows on the top. These are mostly due to memory related to data management for decompression.
   
   
Figure 5
[[image:Figure12hdmpeg.GIF|thumb|Figure 11 STi7000 layout with memory reduction integrated]]
Interestingly many decades later, the pioneering work that conducted to STV3200 represented less than 0.1% of the surface of the SoC (figure 6).  
 
e) Power consumption limitation
 
The integrated chips were consuming as low as between 0.5 to 1 W (figure 12).
 
[[image:Figure10 power consumption.png|thumb|Figure 12 Power dissipation minimized by STi family under 1W]]
 
f) User experience
 
It required high quality images blended with real time 2D/3D graphics (figure 13). Integrated powerful image blitters, on screen display processors, 3D shader based, and 2D vector graphics engines provided the support for user interfaces, internet browsers, gaming, and 3D TV. These chips were the 1st in offering to the digital consumer market by implementing OpenGL-ES and OpenVG graphic standards as established by the Khronos group.
 
 
[[image:Figura9 graphics.png|thumb|Figure 13 3D graphic images rendered to enhance user experience]]
 
 
2) Organizational obstacles
 
Management was uncertain at the beginning to invest too much into the development of a full roadmap of multimedia circuits.
 
a) Management commitment
 
In late 1980s the digital multimedia satellite/cable/terrestrial/streaming video and audio services were not existing. Analog TV transmissions were dominating the market and in everyone home, while the PC was marginal for that domain. Therefore, the company management was reluctant to start investing in developing multimedia chips with an unpredictable return on investments. The internal investments were purely R&D generating only financial losses. However, the design win achieved with RCA/DirecTV with the STi3500 was a cornerstone, and convinced to invest more resources (figure 14). This achievement marked the end of the 10 years pioneering phase by the company.
 
[[image:Figure14 total.png|thumb|Figure 14 30 years of investment on R&D and production]]
|a5=They were: a) accuracy of results provided by the chips, b) their maturity, and d) the advancements they featured.
 
a) Accuracy of results
ST chip without visible mismatch errors versus anchor’s references as established by MPEG core experiments


b) Chip maturity: SGS-Thomson chips were released earlier and the more mature vs Toshiba, Fujitsu, Telettra, ANT, C-Cube.  
Figure 6 Super integration of DCT with other multimedia functions
Until STMicroelectronics has sold over 500,000,000 (cumulated) of those integrated systems worldwide. This figure undoubtedly demonstrates the pervasive of STi family through the world. Figure 7 shows cumulative shipments of ST products per year basis.
Figure 7
14) As proof of the high relevance of the proposed milestone a number of letter of supports were provided by key and well know experts in the field
a) A letter of endorsement [7.0] to ST milestone is provided by Nicolas Demassieux, Senior Vice President of Research, Orange [2.3], and assistant professor at ENST (now Telecom ParisTech) working on very optimized VLSI implementations for signal processing, included the DCT processor.
b) A letter of endorsement [7.1] to ST milestone is provided by Leonardo Chiariglione [2.3], the father of MPEG and the driving force behind MPEG standards for digitized video. Leonardo is unanimously considered a genius since created a new way of enjoying music, with Leonardo Chiariglione’ s MP3 standard.
As mentioned in the public article [6.1] with Leonardo Chiariglione, SGS­Thomson has been a big MPEG driver since it was the world's leading producer of MPEG-related ICs and supplied close to 70 percent of the world's MPEG-2 decoder chips. That was the result of the audiovisual vision that was shared by the leaders of SGS-Thomson, Thomson Multimedia SA, Paris, and France's government. Also, Chiariglione noted, Thomson's U.S. subsidiary RCA was in the need of MPEG chips for a huge order of 1st generation satellite TV decoders--a million set-top boxes for the Hughes-RCA DirecTV newborn system--and SGS-Thomson Microelectronics got the contract, as proof of its leading integrated circuits technology started since 1986.
c) A letter of endorsement [7.2] to ST milestone is provided by prof Fabio Rocca. Prof Rocca on 1969 was the pioneer on motion estimation and compensation technologies.
d) A letter of endorsement [7.3] to ST milestone is provided by Hisafumi Yamada, past Sony USA TV CTO that prove the high innovation STi HDTV chips provided to ATV USA television.
15) In the following years STi products on MPEG where the key enabling factor of a huge digital consumer ecosystem composed by many operators, equipment manufacturers, software developers and conditional access companies as shown in the next figure 6
Figure 8 Worlds wide level ecosystem for digital multimedia services created thanks to STi family
16) As written in the EETimes article [2.4] “In the first shift or wave (referred to in the set-top box market) , which began about 1994, the boxes were simple channel-hopping devices for satellite and cable TV providers, Hatch explained. (ST's MPEG-2 video decoders were used then in the boxes manufactured by Thomson's RCA subsidiary.)” which clearly witness how ST was ahead.
17) As written in the EDN article [2.5] on 1996 “Dataquest has confirmed SGS-THOMSON Microelectronics (Lincoln, MA) as the world's leading supplier of MPEG decoder ICs in 1995.” which witnessed ST leadership on MPEG decoders.
18) as written into a public article STMicroelectronics NV History [2.6] quoting “ ST is the world's leading manufacturer of analog ICs (integrated circuits) and MPEG-2 decoder chips, used to provide video decompression for DVD players and digital television set-top boxes.” and “Another area in which SGS Thomson became an early player was its development of MPEG decompression chips. The company's dedication to MPEG technology was risky, as a number of other digital video decompression schemes were competing for what promised to be a huge market in the future. SGS Thomson gained an early lead in the MPEG market, however, releasing its first Motion Estimation Processor in 1990. By 1993, the company had debuted its "multimedia" chip, capable of decompressing digital video files for display on a television set. This chip helped the company take a major position in the new set-top box market, starting with supplying the chip for the Hughes digital satellite television set-top box.” It helps to also state that ST was dominating the MPEG domain winning over the risks due to its early investments.
19) SGS-THOMSON was also not only pioneer of MPEG2 in satellite digital multimedia business with DirecTV but also in Cable business with Scientific Atlanta (which was acquired by Cisco later). Thanks to the collaboration the STi chips named 5600 and 5610 were the first chips to integrate graphics engine with multimedia processors [1.21], [1.22].
|a6=One key initial obstacle was represented by the initially used 1.2 um silicon technology that prevented to super integrate in a single chip too many hardwired and micro controller bocks. Therefore, hardware designers had to carefully decide what to accelerate with respect to a software implementation, for example on 68020 micro controllers used in Apple computers. In fact, STV3200 needed only 115,000 transistors because of a deep hardware optimized design involving multiple technical dimensions.
1) data precision of internal calculations, as minimized between 8, 12 to 16 bits
2) datapath branches to compensate low precision compute of some part of the low bit depth circuitry to be compliant with IEEE 1180-1990, the IEEE Standard Specifications for the Implementations of 8x8 Inverse Discrete Cosine Transform,
3) internal memory distribution, transposition [4.1][4.2] of read and write memory data access to avoid using costly additional SRAM blocks.
These dimensions were met in the chip despite the need to pioneer the support of various block formats such as 16x16, 8x8, 4x4, 16x8, 8x4, 8x16, 4x8 that will be used many years later by MPEG-1,2, 4, H.264 and HEVC video standards. DCT coefficients were minimized to use only 12bits reducing furthermore internal memory footprint [4.6]. Internal data have been processed at 16 bits to minimize loss of precision instead of using 32 bits integer arithmetic. Integrated memory was only 4Kbits and be transposed [4.1] [4.2] in real time by the internal controller avoiding unnecessary duplications. Also, DCT separability to 1D was exploited to save silicon resources [4.7], [4.8], [4.9].
Similar optimizations were applied to STi3220 motion estimator based on block matching between block of pixels; by limiting the search window to 256 positions the silicon complexity of the block matcher was reduced, shifting the random-access memory bottleneck into proper burst access to avoid the costly penalties of opening and closing memory pages (to access frame buffers) at pixel level with associated deadly loss of efficiency of the processor.
The software computing obstacle had to be addressed in the most efficient manner.
1) By hardwiring DCT and motion estimation/compensation processing functions, the most silicon area demanding function, hundreds of powerful micro controllers (such as Motorola 68020 or equivalent) were not anymore needed, while only an external 8bit controller using only 8Kbytes of ROM was capable to initiate decoding and searching operation once any picture was under decompression without being on the decoding critical path nor perform pixel level operations. That removed the need for any handshake between the external micro controller and the integrated circuits, avoiding hardware pipeline stalls that could result in picture freeze or annoying block artefacts affecting picture quality.
Another challenge, as the silicon technology advanced for STi3240, STi3400, STi3500, STi3520, STi7000, STi5500 and successive derivative chips, was the memory bottleneck that became even more urgent to address due to the unified addressing space used by different accelerators. DRAM was costly and severely limited in space and bandwidth which in turn prevented the super integration since required to store I, P, B pictures and MPEG compressed bitstream. Decoding PAL and NTSC resolutions into only 16 Mbits of memory, to keep low the costs, implied several tricks to be implemented such as MPEG-2 B pictures decoding on the fly [4.4] and frame buffer compression [4.3] [4.5] [1.11] applied to both standard and high-definition decoders across the STi family. Novel memory reductions technique fixed the obstacle allowing MPEG SD decoding in only 8Mbits and HTDV in 32Mbits one of the most impressive achievement at that time.
The energy consumption obstacle needed to be addressed to avoid power hungry systems. SoC optimizations allowed to keep the power consumption challenge between 0.5 and 1 W because higher power consumption would have reduced dramatically the reliability of the chips which were not using any heat sink other than a plastic package.  This allowed to increase speed and functionality up to full High-Definition pictures within a single chip, also addressing the engine orchestrating obstacle due the on-chip CPU, video, audio and graphics concurrent executions and later on, super integrating also the micro controller (8 to 32bits), which resulted into dramatically minimized costs and greatly improved performances, as compared with competition multi-chip implementations.
The STi family consisted of: STV3200 (DCT), STV3220 (Motion Estimator), STi3240, STi3400 (MPEG 1 video decoder), STi3500 (MPEG2 Video decoder), STi3520 (MPEG 2 audio and video decoder) and STi7000 (HD MPEG 2 video decoder and 3D graphics), STi5000 (integrating micro controllers) and many many others which we are not listing here for sake of brevity.
Using low pin count and plastic packages without heat sinks mitigated the manufacturing cost obstacle and increased reliability of those chips that also necessitated from a simple 8-bit micro controller to a more complex 32bits CPU as companion first and then integrated in the SoC.
In term of addressing the acceleration obstacle, by using such a deep hardware optimized design approach vs a full software one, the achieved performances were incomparable to state of art micro controllers such as 68020 which was unthinkable to be used due the excessive two digits number of them (to implement the decoding function) since 68020 was not optimized to process 60 or 120 million of pixels per second such the one required by high-definition MPEG 2 decoding. This need was further exacerbated later by the need to use on screen graphics to enhance used experience.
Another obstacle to address was user experience which required high quality images produced with real time graphics processing. This was addressed by super integrating incrementally powerful image blitters, on screen display processor, 3D and 2D vector graphics engines to support user interfaces, internet browsers, gaming and 3D TV. ST was the 1st in bringing 3D graphics to the digital consumer market by implementing OpenGL-ES and OpenVG standards from the Khronos group.
Figure 9 graphic images rendered by STi family to enhance user experience
Organizational obstacles
c) Chip Advancements. They significantly provided low silicon area footprint by exploiting advanced CMOS silicon lithography such as 0.8, 0.5, 0.35um up to 28nm FDSOI.
At the beginning in 1986 the digital multimedia applications and services were not existing because of the analog TV transmissions were dominating in everyone entertainment life, while the PC one was marginal for that domain. By consequence, the SGS-THOMSON management was reluctant to invest initially in a full generation of STi chips with an unpredictable return on investment a-head. The internal investments were purely R&D without any revenues, just losses in term of money discouraging further investments. Moreover, the seed cooperation between Rennes Electronics Laboratory, ENST Paris and Thomson Semiconductor under DAII sponsorship since 1986 first, then with Apple computer and RCA removed those barriers and was instrumental for SGS-Thomson to create a roadmap of chips, manufacture them and to boost the chip production motivating subsequent investments and multimedia applications and service flourishment.
|references=1) Technical articles, conference papers & books
The architecture of the DCT chip was invented at ENST, [4.10] [4.11] [7.0], and the DCT industrial chip that was designed within the ENST laboratory (Alain Artieri spent a year in ENST team) using a specific CAD tool that was also developed within ENST, that made possible to generate this Hardware-Optimized solution. At that time, the head of the IC research group at ENST was Professor Francis Jutand, and Nicolas Demassieux was assistant professor. Nicolas Demassieux was personally participating to the MPEG1 (and later MPEG2) standardization efforts, and this work was fueled by EC-funded projects (such as the VADIS project). Both of Jutand and Demassieux came to SGS-Thomson, to convince the company managers to adopt DCT architecture and CAD tool. At that time, the internal ST design team was proposing 12 chips solutions, using gate-arrays. It took nearly a year to convince the manager of ST design team that Jutand and Demassieux had a credible solution, and a contract was signed between ENST and ST to a) licensing the intellectual property of two patents to ST, and 2) set-up a joint-design team that would design the DCT chip. Alain Artieri came for a year in the ENST team, and we designed together (with help of a couple of younger students) the layout of chip, while all the verifications (DRC, logical simulation....) were carried out in ST at Grenoble. Both Alain Artieri and Nicolas Demassieux designed the chip as equals in the collaboration.
 
The ST team in Grenoble should be hailed for this adventure, and for being open-minded to change their "usual way of working" and accept this innovative idea of a joint academic/industry design team, to deliver this world's first STV3200 chip. This was a unique case (to our best knowledge) of an industrial chip being designed by a joint academic/industry team, through the collaboration between ENST and ST. Initial research work at ENST in 1986, described in [1.24]. [4.10] [4.11] patents granted to ENST in 1986 resulting from this research. Licensing and contractual agreement between ST and ENST during 1986 with final meeting to close the deal on Nov 12, 1986.
A key management directive was to seek the market interest on those chips and an initial customer which was Apple Computer with whom SGS-Thomson specified the first MPEG1 ST3240 chip using available STV3200 implementing DCT. Secondly RCA, acquired by Thomson Consumer Electronics, was massively investing on Digital Satellite broadcast but its end-to-end system was lacking affordable decoding technologies. SGS-Thomson management decision was to take the risk on the decoder development without any commitment from RCA, with potential loss of development costs. Thanks to the deep hard work of SGS-Thomson a key engineer like Alain Artieri, it was released to RCA, one and half year ahead of C-Cube therefore being the first decoder being adopted into DirectTV digital multimedia broadcast.
|a5=As reported into [1.9] many solutions for image and compression ICs were existing in 1992 however SGS-Thomson started since 1986. The ST innovations, which have been outlined in this IEEE Milestone proposal and which have created and provided significantly advancements in the field of digital multimedia, can be concisely summarized as follows:
1) The silicon development started on 1986 between Rennes Electronics Laboratory, ENST Paris and Thomson Semiconductor under DAII sponsorship a-head of anybody else
3) The 1st chip for mass production for forward and inverse DCT was STV3200 in 1988.
4) The 1st chip for mass production for motion estimator was STV3220 in 1989.
5) Their hardwired implementations replaced software solutions which needed hundreds of Motorola 68020 which were unable to achieve production and deployment into end user Set top boxes and digital TV.
6) They incrementally implemented innovative techniques to reduce memory bandwidth such as block based efficient burst memory read and write, picture decoding on the fly, frame buffer compression to minimize external RAM needs and enable super integration of 1 bit transistor memory RAM cells in an on per requirement fashion.
7) Gave the designer the freedom to use more advanced CMOS silicon lithography such as 0.5, 0.35um and successive in order to achieve super integration of more and more functionalities, to meet affordable mass production.
8) Integrate more heterogeneous functions such as micro controller (8 bits, 32bits), audio decoding and 3D OpenGL-ES graphics
9) System on Chip integration including faster processor cores (high instruction per cycles and frequency), caches (to handle code density and latency), memory controllers (for minimal latency communication with DDR), assisted by tool chains (with compilers adopted by a wide developer community)
10) Addressing HDTV decoding, Video encoding and content Transcoding in real time
11) Super integrating MPEG2 transport processing capabilities using STM32 bit microprocessor.
12) Low dissipation <1W (figure 10) and high throughput efficiency
Figure 10 Power dissipation minimized by STi family under 1W
13) A continued investment on R&D and production started on 1986 till 2016 for 30 years of break-through products that shaped the digital multimedia domain (figure 11). 
Figure 11
|references=1. TECHNICAL ARTICLES, CONFERENCE PAPERS & BOOKS
[1.1] A. Artieri, S. Kritter, F. Jutand and N. Demassieux, "A one chip VLSI for real time two-dimensional discrete cosine transform," 1988., IEEE International Symposium on Circuits and Systems, 1988, pp. 701-704 vol.1, doi: 10.1109/ISCAS.1988.15022.
[1.1] A. Artieri, S. Kritter, F. Jutand and N. Demassieux, "A one chip VLSI for real time two-dimensional discrete cosine transform," 1988., IEEE International Symposium on Circuits and Systems, 1988, pp. 701-704 vol.1, doi: 10.1109/ISCAS.1988.15022.
[1.2] STV3200 DISCRETE COSINE TRANSFORM (DCT) Datasheet
[1.2] STV3200 DISCRETE COSINE TRANSFORM (DCT) Datasheet
[1.3] A. Artieri and F. Jutand, "A versatile and powerful chip for real-time motion estimation," International Conference on Acoustics, Speech, and Signal Processing, 1989, pp. 2453-2456 vol.4, doi: 10.1109/ICASSP.1989.266964.
 
[1.4] STi3220 MOTION ESTIMATOR PROCESSOR Datasheet
[1.3] A. Artieri and O. Colavin, "A chip set core for image compression," in IEEE Transactions on Consumer Electronics, vol. 36, no. 3, pp. 395-402, Aug. 1990, doi: 10.1109/30.103150.  
[1.5] APPLICATION NOTE STi3220 MOTION ESTIMATION PROCESSOR CODEC
 
[1.6] STi3400 MPEG/H.261 VIDEO DECODER DATASHEET
[1.4] STi3400 MPEG/H.261 VIDEO DECODER DATASHEET
[1.7] STi3500 MPEG2 CCIR 601H.261 VIDEO DECODER DATASHEET
 
[1.8] Image and Video Compression standards: algorithm and architectures, Bhaskaran, Vasudev, Konstantinides, Konstantinos, ISBN 978-1-4615-6199-6, 1995
[1.5] STi3500 MPEG2 CCIR 601H.261 VIDEO DECODER DATASHEET
[1.9] K. Konstantinides and V. Bhaskaran, "Monolithic architectures for image processing and compression," in IEEE Computer Graphics and Applications, vol. 12, no. 6, pp. 75-86, Nov. 1992, doi: 10.1109/38.163627.
 
[1.10] A. Artieri and O. Colavin, "A chip set core for image compression," in IEEE Transactions on Consumer Electronics, vol. 36, no. 3, pp. 395-402, Aug. 1990, doi: 10.1109/30.103150.
[1.6] Image and Video Compression standards: algorithm and architectures, Bhaskaran, Vasudev, Konstantinides, Konstantinos, ISBN 978-1-4615-6199-6, 1995
[1.11] A. Chimienti, M. Lucenteforte, D. Pau and R. Sannino, "A novel co-decoding scheme to reduce memory in MPEG-2 MP@ML decoder," 1998 URSI International Symposium on Signals, Systems, and Electronics. Conference Proceedings (Cat. No.98EX167), 1998, pp. 272-277, doi: 10.1109/ISSSE.1998.738080.
 
[1.12] STi70000 press release
[1.7] K. Konstantinides and V. Bhaskaran, "Monolithic architectures for image processing and compression," in IEEE Computer Graphics and Applications, vol. 12, no. 6, pp. 75-86, Nov. 1992, doi: 10.1109/38.163627.
[1.13] STi71000 datasheet
 
[1.14] STi7200 datasheet
[1.8] STi70000 press release
[1.15] STi7108 datasheet
 
[1.16] STi7108 processor with 3D graphics made public.
[1.9] STi71000 datasheet
[1.17] A chip set for MPEG-2 video encoding; Proceedings of the IEEE 1995 Custom Integrated Circuits Conference, 1995  
 
[1.18] STi3240 MPEG1/H261 datasheet
[1.10] STi7200 datasheet
[1.19] STi3208 DCT chip datasheet
 
[1.20] STi3520A MPEG2 chip datasheet
[1.11] STi7108 datasheet
[1.21] STi5600 MPEG2 chip datasheet
 
[1.22] STi5610 MPEG2 chip datasheet
[1.12] STi7108 processor with 3D graphics made public.
[1.23] R. Bruni, A. Chimienti, M. Lucenteforte, D. Pau and R. Sannino, "A novel adaptive vector quantization method for memory reduction in MPEG-2 HDTV decoders," in IEEE Transactions on Consumer Electronics, vol. 44, no. 3, pp. 537-544, Aug. 1998, doi: 10.1109/30.713161.
 
[1.24] F. Jutand, N. Demassieux, G. Concordel, J. Guichard and E. Cassimatis, "A single chip video rate 16×16 discrete cosine transform," ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986, pp. 805-808, doi: 10.1109/ICASSP.1986.1169147.
[1.13] A chip set for MPEG-2 video encoding; Proceedings of the IEEE 1995 Custom Integrated Circuits Conference, 1995
[1.25] N. Demassieux, G. Concordel, J. Durandeau and F. Jutand, "An optimized VLSI architecture for a multiformat discrete cosine transform," ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1987, pp. 547-550, doi: 10.1109/ICASSP.1987.1169851.
 
[1.26] Tomasko, M., Buchhauser, D., Bushroe, M. et al. The Descent Imager/Spectral Radiometer (DISR) Experiment on the Huygens Entry Probe of Titan. Space Science Reviews 104, 469–551 (2002). https://doi.org/10.1023/A:1023632422098
[1.14] STi3240 MPEG1/H261 datasheet
 
2. ONLINE INFORMATION AND CITATIONS
[1.15] STi3208 DCT chip datasheet
[2.1] https://en.wikipedia.org/wiki/Motorola_68020  
 
[2.2] https://techmonitor.ai/technology/sgs_thomson_debuts_one_chip_hdtv_system  
[1.16] STi3520A MPEG2 chip datasheet
[2.3] Leonardo Chiariglione awards https://leonardo.chiariglione.org/public-life/awards/  
 
[2.4] see pdf and https://www.eetimes.com/apis-help-stmicroelectronics-grow-set-top-box-business/#  
[1.17] STi5600 MPEG2 chip datasheet
[2.5] see pdf and https://www.edn.com/sgs-thomson-microelectronics-named-number-one-mpeg-decoder-supplier-by-dataquest/  
 
[2.6] see pdf and http://www.fundinguniverse.com/company-histories/stmicroelectronics-nv-history/  
[1.18] STi5610 MPEG2 chip datasheet
[2.7] https://books.google.it/books?id=C-bgBwAAQBAJ&pg=PA234&lpg=PA234&dq=stv3200+sgs-thomson&source=bl&ots=HrOQA0LEvS&sig=ACfU3U0vpZ7K5HJzJ1qJWjjlk4yfzPW82w&hl=it&sa=X&ved=2ahUKEwj2lZ2p35HwAhWBzKQKHVrmBBMQ6AEwEnoECBUQAw#v=onepage&q=stv3200%20sgs-thomson&f=false  
 
[1.19] F. Jutand, N. Demassieux, G. Concordel, J. Guichard and E. Cassimatis, "A single chip video rate 16×16 discrete cosine transform," ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986, pp. 805-808, doi: 10.1109/ICASSP.1986.1169147.
 
[1.20] N. Demassieux, G. Concordel, J. Durandeau and F. Jutand, "An optimized VLSI architecture for a multiformat discrete cosine transform," ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1987, pp. 547-550, doi: 10.1109/ICASSP.1987.1169851.
 
[1.21] Tomasko, M., Buchhauser, D., Bushroe, M. et al. The Descent Imager/Spectral Radiometer (DISR) Experiment on the Huygens Entry Probe of Titan. Space Science Reviews 104, 469–551 (2002). https://doi.org/10.1023/A:1023632422098
 
[1.22] Report of the meeting of the DCT chip manufacturers in Martlesham UK 24-25 May 1988, CCITT SGXV Document #349, Working Party XV/1 21 June 1988, Specialists Group on Coding for Visual Telephony
 
[1.23] An Integrated MPEG-1 and MPEG-2 Decoder, IEEE International Conference on Consumer Electronics, 21-23 June 1994 https://ieeexplore.ieee.org/document/582280
 
[1.24] Progress on Inverse DCT Accuracy Specifications, C.C.I.T.T. Study group XV WP XV/1 Specialist Group on Picture Coding
 
[1.25] Matrix multiplier transform analysis and design of a general purpose transformer; Signal Processing, Volume 28, Issue 2, August 1992, Pages 153-162 https://doi.org/10.1016/0165-1684(92)90032-R    
 
2) Online information and citations
 
[2.1] https://en.wikipedia.org/wiki/Motorola_68020
 
[2.2] https://techmonitor.ai/technology/sgs_thomson_debuts_one_chip_hdtv_system
 
[2.3] Leonardo Chiariglione awards https://leonardo.chiariglione.org/public-life/awards/
 
[2.4] see pdf and https://www.eetimes.com/apis-help-stmicroelectronics-grow-set-top-box-business/#
 
[2.5] see pdf and https://www.edn.com/sgs-thomson-microelectronics-named-number-one-mpeg-decoder-supplier-by-dataquest/
 
[2.6] see pdf and http://www.fundinguniverse.com/company-histories/stmicroelectronics-nv-history/
 
[2.7] https://books.google.it/books?id=C-bgBwAAQBAJ&pg=PA234&lpg=PA234&dq=stv3200+sgs-thomson&source=bl&ots=HrOQA0LEvS&sig=ACfU3U0vpZ7K5HJzJ1qJWjjlk4yfzPW82w&hl=it&sa=X&ved=2ahUKEwj2lZ2p35HwAhWBzKQKHVrmBBMQ6AEwEnoECBUQAw#v=onepage&q=stv3200%20sgs-thomson&f=false
 
[2.8] http://www.quretec.com/u/vilo/edu/2002-03/Tekstialgoritmid_I/Loengud/Loeng7_Compression/www.faqs.org/faqs/compression-faq/part1/preamble.html
[2.8] http://www.quretec.com/u/vilo/edu/2002-03/Tekstialgoritmid_I/Loengud/Loeng7_Compression/www.faqs.org/faqs/compression-faq/part1/preamble.html
[2.9] https://books.google.fr/books?id=UsH8CAAAQBAJ&lpg=PA501&ots=OJr_PN0LKo&dq=stv3200%20DCT&hl=fr&pg=PA501#v=onepage&q=stv3200%20DCT&f=false  
 
[2.9] https://books.google.fr/books?id=UsH8CAAAQBAJ&lpg=PA501&ots=OJr_PN0LKo&dq=stv3200%20DCT&hl=fr&pg=PA501#v=onepage&q=stv3200%20DCT&f=false
 
[2.10] http://ciclops.org/ir_index/208/In-Orbit?js=1  
[2.10] http://ciclops.org/ir_index/208/In-Orbit?js=1  


[2.11] https://en.wikipedia.org/wiki/Allan_Alcorn
[2.12] https://www.amazon.com/dp/B096G6TSF9/
[2.13] Computer History Museum, Oral History of Allan Alcorn Recorded: April 26, 2008 and May 23, 2008 Mountain View, California https://archive.computerhistory.org/resources/access/text/2012/09/102658257-05-01-acc.pdf
3) STMicroelectronics documents


3. STMicroelectronics documents
[3.1] De la Transformée en Cosinus Discrète aux Processeurs Applicatifs Multimédia, Alain Artiéri, Septembre 2007, STMicroelectronics presentation
[3.1] De la Transformée en Cosinus Discrète aux Processeurs Applicatifs Multimédia, Alain Artiéri, Septembre 2007, STMicroelectronics presentation


4. PATENTS
  4) Patents
[4.1] Integrated signal processing circuit for online and column summation of digital value matrices https://patents.google.com/patent/FR2608802B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old  
 
[4.2] Transposition memory for data processing circuit https://patents.google.com/patent/FR2617621B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old  
[4.1] USRE34734 "Integrated digital signal processing circuit for performing cosine transformation”
[4.3] Method and apparatus for addressing a memory area of an MPEG decoder4.6
 
[4.4] publication number US6081298A MPEG decoder with reduced memory capacity
[4.2] US4872134A "Signal processing integrated circuit for row and column addition of matrices of digital value”
[4.5] ITVA960016D0 Metodo di ricompressione e decompressione adpcm di un flusso di dati digitali costituente un segnale video digitale e stimatore https://patents.google.com/patent/ITVA960016D0/it?inventor=danilo+pau&oq=danilo+pau&sort=old
 
[4.6] publication number 0 298 002 of 29.06.1988 Memoire de transposition pour circuit de traitement de donnes
[4.3] US5600837A "Multitask processor architecture having a plurality of instruction pointers”
[4.7] publication number FR2649226A1 BREWING CIRCUIT OF DATA
 
[4.8] publication number FR2650462B1 Device for converting a line scanning into a scanning in vertical saw teeth by bands; https://patents.google.com/patent/FR2650462B1/en?q=H04N3%2f30&inventor=alain+artieri&sort=old  
[4.4] Integrated signal processing circuit for online and column summation of digital value matrices https://patents.google.com/patent/FR2608802B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old
[4.9] publication number EP0368731B1 Process and circuit for image representation signal filtration
 
[4.10] Circuit pour effectuer une transformation linéaire sur un signal numérique Patent issuer and number eu EP 0241352 A1 https://patents.google.com/patent/EP0241352A1
[4.5] Transposition memory for data processing circuit https://patents.google.com/patent/FR2617621B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old
[4.11] Additionneur binaire comportant un opérande fixé, et multiplieur binaire parallèle-série comprenant un tel additionneur Patent issuer and number eu EP 0262032 B1 https://patents.google.com/patent/EP0262032B1  
 
5. Honors
[4.6] Method and apparatus for addressing a memory area of an MPEG decoder
 
[4.7] publication number US6081298A MPEG decoder with reduced memory capacity
 
[4.8] publication number 0 298 002 of 29.06.1988 Memoire de transposition pour circuit de traitement de donnes
 
[4.9] publication number FR2649226A1 BREWING CIRCUIT OF DATA
 
[4.10] publication number FR2650462B1 Device for converting a line scanning into a scanning in vertical saw teeth by bands; https://patents.google.com/patent/FR2650462B1/en?q=H04N3%2f30&inventor=alain+artieri&sort=old
 
[4.11] publication number EP0368731B1 Process and circuit for image representation signal filtration
 
[4.12] Circuit pour effectuer une transformation linéaire sur un signal numérique Patent issuer and number eu EP 0241352 A1 https://patents.google.com/patent/EP0241352A1
 
[4.13] Additionneur binaire comportant un opérande fixé, et multiplieur binaire parallèle-série comprenant un tel additionneur Patent issuer and number eu EP 0262032 B1 https://patents.google.com/patent/EP0262032B1
 
[4.14] Integrated digital signal processing circuit for performing cosine transform
 
[4.15] Signal processing integrated circuit for row and column addition of matrices of digital values
 
5) Honors
 
[5.1] Two European IT Prizes Awarded to STMicroelectronics
[5.1] Two European IT Prizes Awarded to STMicroelectronics
6) MPEG mentions
[6.1] Chiariglione and the birth of MPEG https://www.chiariglione.org/public-life/media/leonardo_ieee/prof.htm
7) Letters of support
[7.1] By Alain ARTIERI, the creator of MPEG2 chips in ST.
[7.2] Endorsement by Nicolas Demassieux, Senior Vice President of Research, Orange, assistant professor at ENST (now Telecom ParisTech) mid 80s.
[7.3] Endorsement by Leonardo Chiariglione, PhD Eng, the father of MPEG
[7.4] Endorsement by Professor Emeritus Fabio Rocca, Politecnico di Milano, pioneer of motion estimation in 1969
[7.5] Endorsement by Hisafumi Yamada, former Sony US TV CTO
[7.6] Endorsement by Sylvain Kritter, Product Director VIZYON
[7.7] Endorsement by Professor Emeritus Virginio Cantoni, University of Pavia. Also Fellow of the IAPR since 1994 and Life Fellow of the IEEE (he was Fellow since 1997). In 2006 the President of the Italian Republic conferred to Professor Virginio Cantoni the title of ‘Commander of the Order of Merit of the Italian Republic’.
[7.8] Endorsement by Professor Emeritus Mariagiovanna Sami, Politecnico di Miano. Past Full Professor, Digital Systems. IEEE Life Member. Minerva prize for woman scientists, the Seymour Cray prize for contributions in the area of parallel processing, the Herbert A. Simon Gold Medal, assigned by the Society for Design and Process Science. Cavaliere della Repubblica Italiana (knight of the Italian republic). Member of the Italian National Science Academy (“Dei Quaranta”).
[7.9] Endorsement by Professor Andrea Basso who directly contributed to MPEG standards since 90s while in EPFL and in AT&T Bell labs and AT&T Research Laboratories, Middletown, NJ, USA
[7.10] Endorsement by Allan Alcorn, Apple Fellow in the late 80s in the Advanced Technology Group at Apple Computer, Cupertino USA


6. MPEG mentions
[7.11] Endorsement by Jeff Cooper, former Senior R&D Engineer at Thomson Multimedia - RCA in the 90s responsible for DirectTV MPEG2 set top box design based on STi3500.
[6.1] Chiariglione and the birth of MPEG https://www.chiariglione.org/public-life/media/leonardo_ieee/prof.htm


7. Letters of support
[7.12] Endorsement by Shigenobu Minami, former R&D team leader at Toshiba for MPEG4 standardization and LSI planning in the 90s.
[7.0] Endorsement by Leonardo Chiariglione, Nicolas Demassieux, Senior Vice President of
Research, Orange, assistant professor at ENST (now Telecom ParisTech) mid 80s.
[7.1] Endorsement by Leonardo Chiariglione, the father of MPEG
[7.2] Endorsement by Prof Rocca, pioneer of motion estimation in 1969
[7.3] Endorsement by Hisafumi Yamada, former Sony US TV CTO
|submitted=Yes
|submitted=Yes
}}
}}

Latest revision as of 13:13, 15 June 2023


To see comments, or add a comment to this discussion, click here.

Docket #:2021-10

This Proposal has been approved, and is now a Milestone


To the proposer’s knowledge, is this achievement subject to litigation? No

Is the achievement you are proposing more than 25 years old? Yes

Is the achievement you are proposing within IEEE’s designated fields as defined by IEEE Bylaw I-104.11, namely: Engineering, Computer Sciences and Information Technology, Physical Sciences, Biological and Medical Sciences, Mathematics, Technical Communications, Education, Management, and Law and Policy. Yes

Did the achievement provide a meaningful benefit for humanity? Yes

Was it of at least regional importance? Yes

Has an IEEE Organizational Unit agreed to pay for the milestone plaque(s)? Yes

Has an IEEE Organizational Unit agreed to arrange the dedication ceremony? Yes

Has the IEEE Section in which the milestone is located agreed to take responsibility for the plaque after it is dedicated? Yes

Has the owner of the site agreed to have it designated as an IEEE Milestone? Yes


Year or range of years in which the achievement occurred:

1984-1993

Title of the proposed milestone:

MPEG Multimedia Integrated Circuits, 1984-1993

Plaque citation summarizing the achievement and its significance:

Beginning in 1984, Thomson Semiconducteurs (now STMicroelectronics) developed multimedia integrated circuits, which accelerated Moving Picture Experts Group (MPEG) standards. By 1993, MPEG-2 integrated decoders -- including innovative discrete cosine transform (developed jointly with ENST, now Telecom ParisTech), bitstream decompression, on-the-fly motion compensation, and display unit -- were announced in one silicon die: the STi3500. Subsequent MPEG-2 worldwide adoption made compressed full-motion video and audio inexpensive and available for everyday use.

200-250 word abstract describing the significance of the technical achievement being proposed, the person(s) involved, historical context, humanitarian and social impact, as well as any possible controversies the advocate might need to review.


IEEE technical societies and technical councils within whose fields of interest the Milestone proposal resides.


In what IEEE section(s) does it reside?

France Section

IEEE Organizational Unit(s) which have agreed to sponsor the Milestone:

IEEE Organizational Unit(s) paying for milestone plaque(s):

Unit: France
Senior Officer Name: Claire LAJOIE-MAZENC

IEEE Organizational Unit(s) arranging the dedication ceremony:

Unit: France with STMicroelectronics
Senior Officer Name: Claire LAJOIE-MAZENC

IEEE section(s) monitoring the plaque(s):

IEEE Section: France
IEEE Section Chair name: Claire LAJOIE-MAZENC

Milestone proposer(s):

Proposer name: Danilo Pau
Proposer email: Proposer's email masked to public

Proposer name: Jean-Michel Moutin
Proposer email: Proposer's email masked to public

Please note: your email address and contact information will be masked on the website for privacy reasons. Only IEEE History Center Staff will be able to view the email address.

Street address(es) and GPS coordinates in decimal form of the intended milestone plaque site(s):

(45.203333, 5.695833)

Describe briefly the intended site(s) of the milestone plaque(s). The intended site(s) must have a direct connection with the achievement (e.g. where developed, invented, tested, demonstrated, installed, or operated, etc.). A museum where a device or example of the technology is displayed, or the university where the inventor studied, are not, in themselves, sufficient connection for a milestone plaque.

Please give the address(es) of the plaque site(s) (GPS coordinates if you have them). Also please give the details of the mounting, i.e. on the outside of the building, in the ground floor entrance hall, on a plinth on the grounds, etc. If visitors to the plaque site will need to go through security, or make an appointment, please give the contact information visitors will need. The Grenoble site is of premier importance for STMicroelectronics. It was owned by THOMSON SEMICONDUCTEURS, then merged with SGS in 1987 to form SGS-Thomson. Nowadays is the biggest R&D site of the company, hosting many product divisions from the three Product Groups of the company, including silicon and software design, test, and advanced packaging developments. In this site MPEG chips were conceived and designed

Are the original buildings extant?

Yes, even if part of them were restructured they still exists @ STMicroelectronics, 12, rue Jules Horowitz F-38000 Grenoble, France

Details of the plaque mounting:

Plaques will be installed in a public place at the main entrance as shown in figure 1. It will be in front of company security personnel who are monitoring entrance 24 hours in a day, every day in a year. Very close as indicated there is the public road. Plaque will be placed at 12, rue Jules Horowitz F-38000 Grenoble, France. GPS coordinates are: 45°12’12” N ; 05°41’45” E (45.203333, 5.695833). Therefore, interested visitors will not need to be escorted by the company security personnel to watch the plaque or to take a photo with it. The milestone plaque will be installed right in front of the main entrance access. It is where employees, visitors and customers must pass through to get into the Grenoble offices on daily basis. It is monitored continuously 24h/7d by security human resources and surveillance camera. It is a public place, right outside the site restricted perimeter.

How is the site protected/secured, and in what ways is it accessible to the public?

It is protected by company security human resources and with surveillance camera, 24h/7d. It is publicly and easily accessible from 12, rue Jules Horowitz F-38000 Grenoble, France. The site can be reached also from Lion Airport, close to Grenoble as shown in figure 2.

Who is the present owner of the site(s)?

STMicroelectronics (Grenoble 2) SAS

What is the historical significance of the work (its technological, scientific, or social importance)? If personal names are included in citation, include justification here. (see section 6 of Milestone Guidelines)

Figure 1 Candidate place for plaque installation
Figure 2 How to get the plaque from Lyon airport

The importance of the proposed milestone will be described as follows: section 1 introduces the object; section 2 lists the technical innovations; section 3 discusses the benefits of the innovations and section 4 their achievements; section 5 provides the historical background about when the innovations were introduced; Section 6 reports the public recognition of the integrated circuits by specialized journals. Finally, section 7 is for the supporting letters provided by world renown experts.

1) Object of the milestone proposal

Starting from 1984 new digital multimedia technologies were conceived and implemented by SGS-THOMSON on different CMOS silicon technologies. They were dedicated processors very optimized in area and power consumption. They were hand-designed and produced in high volume, with high yields and reliability to achieve lowest cost. Next, they were incrementally super-integrated in one silicon die to perform multimedia video decoding anticipating specifications as established by MPEG in the 1992 onward. Several innovations were introduced to reduce the complexity due to the super integration. They anticipated the acceleration of the MPEG1, 2 and subsequent standards. Since no digital multimedia content services for moving pictures were existing at that time, they set the hardware to introduce them. They facilitated widespread adoption by the end users and eased switch-off from analog decoders to digital ones. A special mention is for the STi3500 chip which was the world first MPEG2 decoder. The chip die is shown in the figure 3.

Figure 3 Silicon die of the STi3500, the first MPEG2 video decoder

2) Technical innovations introduced by the milestone

The circuits introduced several innovations. They were:

a) Block based pixel memory management to achieve closely the peak computational power.

b) Compressed MPEG decoding on the fly to reduce video storage in RAM.

c) Frame buffer lossy compression to co-decompress on the fly MPEG2 video

d) Unified memory controller allowed to plug and play additional MPEG and other hardware clients.

3) Benefits of the innovative solutions subject of the milestone

The benefits of integrated circuits which embodied innovations described in section 2, were:

a) Transistor count minimization to ease super integration in one silicon die.

b) Hand crafted hardware design methodology to produce very tiny circuits.

c) Energy consumption to less than one watt across chip generations.

d) Reliability to produce chips for widespread adoption

e) Resilience to faults.

4) Technical achievements of the proposed milestone

The achievements of the processors of the milestone were on the following key indicators:

a) Performances (highest degree of acceleration of MPEG functionalities).

b) Manufacturability (reliability, repeatability, high volumes, robustness, high yields)

c) Super integration of RAM.

d) Silicon node scaling from 2µm to 28nm FDSOI (fully depleted silicon on insulator).

e) Flexibility (VLIW processors, CPUs, 3D programmable graphic)

f) Security (data and memory protection encryption).

g) Efficiency (mW/sqmm/$, reduced memory bandwidth, and reduced calculations bit-depth)

h) Tiny dissipation (avoided costly and bulky heat-sinks and complex liquid cooling)

l) Costs (cheap solutions for end users, eased TV set manufacturers to achieve large scale savings)

m) Ecosystem (key enabler of broad range digital video applications and digital multimedia vertical services, facilitated the end user to enter the new digital entertainment era)

5) Historical background and contributions

THOMSON Semiconductor (headquarter in Grenoble, France) was merged with SGS (headquarter in Agrate Brianza Italy) in 1987; the merge was named SGS-THOMSON; then the company was renamed in 1998 as STMicroelectronics (today name).

In the late 1980s ISO/IEC experts felt the need to standardize motion picture decoding. Therefore, MPEG committee had first meeting in 1988. MPEG1 standard was finalized in 1992 and MPEG2 was finalized in 1994.

All MPEG integrated circuits, object of this proposed milestone, are developed fully in-house SGS-THOMSON Grenoble and intellectual properties are owned by STMicroelectronics.

Next points a) to d) details that the technology subject of this proposed milestone, the MPEG circuits :


a) In 1984, THOMSON Semiconductors started to develop own Discrete Cosine Transform (DCT). This effort was patented twice, therefore the company decided to launch an own chip design and production project in 1986.

b) In 1986 ENST (now Telecon ParisTech) had in parallel a DCT experiments based on a less hardware efficient algorithm than THOMSON designed.

c) In 1987, THOMSON Semiconductor and ENST, even if not essential to DCT chip, decided to cooperate on redesigning a part of the whole DCT chip. The operative part with a serial architecture was conceptualized by ENST and designed by THOMSON in hardware.

d) In 1988 SGS-THOMSON had following products ready for production

i. STV3200 the multi format DCT, not MPEG compliant

Figure 4 DCT chip package
Figure 5 DCT chip die, 1987, 40sqmm, CMOS 1.2µm, 115,000 transistors

ii. STV3208 8x8 DCT, MPEG compliant. Note that DCT into modern chips was just 0.05% of total chip area

Next points aim to explain how MPEG chips were developed.

e) First MPEG1/H261 standard compliant . First solution was composed by two integrated circuits: the picture motion compensation, huffman decoding, dequantization, memory control (into STi3240) and the 8x8 DCT (STV3208). Second solution was the super integration of the two circuits into a single chip named STi3400 .

Figure 6 The discrete MPEG1 decoder system with the STV3208 and STi3240

f) World first STi3500 MPEG2 decoder circuit. It integrated STV3208 with MPEG2 compliant motion compensation. RCA, later acquired by Thomson Consumer Electronics, was creating the world first Digital Satellite TV broadcast service but missed the video decoding function. STi3500 was delivered to RCA (ahead of the competition) and enable decoding of the DirectTV digital multimedia broadcast. Next STI3520A was the world first single chip for MPEG 2 video decoding in only 16 Mbit RAM super integrated with the world first MPEG MP3 Audio decoder (STi4500).

g) First HDTV decoder circuit with RAM reduction algorithms integrated: the STi7000 in 1998. Next versions STi7100, STi7200 integrated 3D shader-based graphics engine and powerful micro controllers. STi7108 had dual CPU host processors with embedded memory. It was the first circuit in the world to integrate 3D OpenGL-ES 2.0 and OpenVG vector graphics.

h) STi5500 was the first (after signal demodulation circuit) complete decoder for set-top boxes and DVD players. Capable to demux MPEG2 transport stream. Memory compression algorithms reduced its RAM needs.

i) World first STi5505, that integrated all the back-end functions of a DVD player. More powerful than the STi5500 and capable to support MPEG2 program stream demultiplexing inside. STi5500 won from a field of nearly 300 nominations, the prestigious European IT Prize awarded by the European Commission in conjunction with Euro-CASE, the European Council of Applied Science and Engineering. Its further evolutions were STi5600 and STi5610.

l) Further generations. Many other integrated circuits were designed over 30 years, as shown in figure 7. Complex operative systems such as Linux and Windows were incrementally supported. These processors also pioneered TV Cable transmission with Scientific Atlanta (which was acquired by Cisco later). They accelerated the introduction of CD-ROM, DVDs, Set Top Box and digital TVs home systems. The analog to digital entertainment switch was therefore supported by mass production circuits and doable in everyone home.

Figure 7 The MPEG decoders generation by generation

Since the functionalities they integrated, these chips were deployed also into smartphones since early 2000s. The Nomadik chip family (STn8800/10/15 and subsequent) was derived to accelerate multimedia into mobile phones. The pioneering 8x8 MPEG DCT chip, the STV3208, represented less than 0.1% of the die area of the SoC as shown in figure 8.

Figure 8 Super integration of STV3208 with other multimedia functions

ST manufactured more than 500 million (cumulative number) of these integrated systems worldwide as shown in Figure 9.

Figure 9 Cumulative shipment of SGS-THOMSON MPEG devices

A rich digital consumer ecosystem has been created thanks to those circuits. It was composed by many operators, equipment manufacturers, software developers and conditional access companies as shown in the next figure 10.

Figure 10 Worlds wide ecosystem for digital multimedia services enabled by SGS-THOMSON integrated circuits

7) Public recognition of the integrated circuits Specialize magazines reported the leadership of the MPEG circuits.

a) EETimes article “In the first shift or wave (referred to in the set-top box market), which began about 1994, the boxes were simple channel-hopping devices for satellite and cable TV providers, Hatch explained. (STMicroelectronics's MPEG-2 video decoders were used then in the boxes manufactured by Thomson's RCA subsidiary.)”.

b) EDN article on 1996 “Dataquest has confirmed SGS-THOMSON Microelectronics (Lincoln, MA) as the world's leading supplier of MPEG decoder ICs in 1995.”

c) STMicroelectronics NV History quoting “ STMicroelectronics is the world's leading manufacturer of .... MPEG-2 decoder chips, used to provide video decompression for DVD players and digital television set-top boxes.” and “Another area in which SGS-Thomson became an early player was its development of MPEG decompression chips. The company's dedication to MPEG technology was risky, as a number of other digital video decompression schemes were competing for what promised to be a huge market in the future. SGS Thomson gained an early lead in the MPEG market. By 1993, the company had debuted its "multimedia" chip, capable of decompressing digital video files for display on a television set. This chip helped the company take a major position in the new set-top box market, starting with supplying the chip for the Hughes digital satellite television set-top box.”

8) Supporting letters by world renown and independent experts in the field Please refer to part 7 of bibliography

What obstacles (technical, political, geographic) needed to be overcome?

1) Technical obstacles

Several obstacles were addressed. Hereafter associated solutions will be discussed in the following order: a) Silicon; b) Algorithm; c) Hardware; d) Memory reduction, e) Power consumption; f) User experience.

a) Silicon technology

One key initial obstacle was represented by the low density 2 and 1.25 um CMOS silicon technology that limited circuity complexity and super-integration. SGS-THOMSON hardware designers overcome this barrier by hand crafting the design, making it very efficient and low transistor count. A careful decision was about which functions to accelerate with respect to the ones to execute in software. As result, STV3208 needed only 115,000 transistors for 8x8 MPEG DCT. Multiple technical dimensions were addressed such as:

i) data precision of internal calculations, as low as 8 bits. DCT coefficients were coded with only 12bits to reduce RAM. Data have been also processed at 16 bits. This avoided 32bits integer and floating point arithmetic.

ii) Designed circuits to be compliant with IEEE 1180-1990, the IEEE Standard Specifications for the Implementations of 8x8 Inverse Discrete Cosine Transform.

iii) Internal memory distribution. Transposition of read and write memory data access avoided costly RAM. DCT separability was exploited to save silicon area. Integrated RAM was as low as 4Kbits.

iv) Design flexibility. Its design was capable to support specifications as established by MPEG-1, 2, 4, H.264 and HEVC standards with a plug and play of new accelerators thanks to the unified memory controller architecture.

b) Hardware technology

The software computing bottleneck was solved by hardwiring DCT, variable length coding, motion compensation, de quantization processing functions. It removed the need for any time critical handshake between the external micro controller and the integrated circuits. Achieved performances order of magnitude higher than pure software execution.

d) Memory reduction technology

A major obstacle was the cost of the RAM . Super integration required unified memory management. Many accelerators super integrated were contending memory access simultaneously. Decoding PAL and NTSC resolutions was achieved into only 16 Mbits with several memory tricks. Novel memory reductions technique fixed the obstacle allowing MPEG SD decoding as low as 8Mbits and HTDV in 32Mbits. In figure 11 the picture of the first consumer high-definition MPEG video chip. The Motion compensation unit with memory reduction is the block on the bottom right corner, with a raw of memory at the bottom and two almost parallel rows on the top. These are mostly due to memory related to data management for decompression.

Figure 11 STi7000 layout with memory reduction integrated

e) Power consumption limitation

The integrated chips were consuming as low as between 0.5 to 1 W (figure 12).

Figure 12 Power dissipation minimized by STi family under 1W

f) User experience

It required high quality images blended with real time 2D/3D graphics (figure 13). Integrated powerful image blitters, on screen display processors, 3D shader based, and 2D vector graphics engines provided the support for user interfaces, internet browsers, gaming, and 3D TV. These chips were the 1st in offering to the digital consumer market by implementing OpenGL-ES and OpenVG graphic standards as established by the Khronos group.


Figure 13 3D graphic images rendered to enhance user experience


2) Organizational obstacles

Management was uncertain at the beginning to invest too much into the development of a full roadmap of multimedia circuits.

a) Management commitment

In late 1980s the digital multimedia satellite/cable/terrestrial/streaming video and audio services were not existing. Analog TV transmissions were dominating the market and in everyone home, while the PC was marginal for that domain. Therefore, the company management was reluctant to start investing in developing multimedia chips with an unpredictable return on investments. The internal investments were purely R&D generating only financial losses. However, the design win achieved with RCA/DirecTV with the STi3500 was a cornerstone, and convinced to invest more resources (figure 14). This achievement marked the end of the 10 years pioneering phase by the company.

Figure 14 30 years of investment on R&D and production

What features set this work apart from similar achievements?

They were: a) accuracy of results provided by the chips, b) their maturity, and d) the advancements they featured.

a) Accuracy of results ST chip without visible mismatch errors versus anchor’s references as established by MPEG core experiments

b) Chip maturity: SGS-Thomson chips were released earlier and the more mature vs Toshiba, Fujitsu, Telettra, ANT, C-Cube.   c) Chip Advancements. They significantly provided low silicon area footprint by exploiting advanced CMOS silicon lithography such as 0.8, 0.5, 0.35um up to 28nm FDSOI.

Supporting texts and citations to establish the dates, location, and importance of the achievement: Minimum of five (5), but as many as needed to support the milestone, such as patents, contemporary newspaper articles, journal articles, or chapters in scholarly books. 'Scholarly' is defined as peer-reviewed, with references, and published. You must supply the texts or excerpts themselves, not just the references. At least one of the references must be from a scholarly book or journal article. All supporting materials must be in English, or accompanied by an English translation.

1) Technical articles, conference papers & books

[1.1] A. Artieri, S. Kritter, F. Jutand and N. Demassieux, "A one chip VLSI for real time two-dimensional discrete cosine transform," 1988., IEEE International Symposium on Circuits and Systems, 1988, pp. 701-704 vol.1, doi: 10.1109/ISCAS.1988.15022.

[1.2] STV3200 DISCRETE COSINE TRANSFORM (DCT) Datasheet

[1.3] A. Artieri and O. Colavin, "A chip set core for image compression," in IEEE Transactions on Consumer Electronics, vol. 36, no. 3, pp. 395-402, Aug. 1990, doi: 10.1109/30.103150.

[1.4] STi3400 MPEG/H.261 VIDEO DECODER DATASHEET

[1.5] STi3500 MPEG2 CCIR 601H.261 VIDEO DECODER DATASHEET

[1.6] Image and Video Compression standards: algorithm and architectures, Bhaskaran, Vasudev, Konstantinides, Konstantinos, ISBN 978-1-4615-6199-6, 1995

[1.7] K. Konstantinides and V. Bhaskaran, "Monolithic architectures for image processing and compression," in IEEE Computer Graphics and Applications, vol. 12, no. 6, pp. 75-86, Nov. 1992, doi: 10.1109/38.163627.

[1.8] STi70000 press release

[1.9] STi71000 datasheet

[1.10] STi7200 datasheet

[1.11] STi7108 datasheet

[1.12] STi7108 processor with 3D graphics made public.

[1.13] A chip set for MPEG-2 video encoding; Proceedings of the IEEE 1995 Custom Integrated Circuits Conference, 1995

[1.14] STi3240 MPEG1/H261 datasheet

[1.15] STi3208 DCT chip datasheet

[1.16] STi3520A MPEG2 chip datasheet

[1.17] STi5600 MPEG2 chip datasheet

[1.18] STi5610 MPEG2 chip datasheet

[1.19] F. Jutand, N. Demassieux, G. Concordel, J. Guichard and E. Cassimatis, "A single chip video rate 16×16 discrete cosine transform," ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1986, pp. 805-808, doi: 10.1109/ICASSP.1986.1169147.

[1.20] N. Demassieux, G. Concordel, J. Durandeau and F. Jutand, "An optimized VLSI architecture for a multiformat discrete cosine transform," ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing, 1987, pp. 547-550, doi: 10.1109/ICASSP.1987.1169851.

[1.21] Tomasko, M., Buchhauser, D., Bushroe, M. et al. The Descent Imager/Spectral Radiometer (DISR) Experiment on the Huygens Entry Probe of Titan. Space Science Reviews 104, 469–551 (2002). https://doi.org/10.1023/A:1023632422098

[1.22] Report of the meeting of the DCT chip manufacturers in Martlesham UK 24-25 May 1988, CCITT SGXV Document #349, Working Party XV/1 21 June 1988, Specialists Group on Coding for Visual Telephony

[1.23] An Integrated MPEG-1 and MPEG-2 Decoder, IEEE International Conference on Consumer Electronics, 21-23 June 1994 https://ieeexplore.ieee.org/document/582280

[1.24] Progress on Inverse DCT Accuracy Specifications, C.C.I.T.T. Study group XV WP XV/1 Specialist Group on Picture Coding

[1.25] Matrix multiplier transform analysis and design of a general purpose transformer; Signal Processing, Volume 28, Issue 2, August 1992, Pages 153-162 https://doi.org/10.1016/0165-1684(92)90032-R

2) Online information and citations

[2.1] https://en.wikipedia.org/wiki/Motorola_68020

[2.2] https://techmonitor.ai/technology/sgs_thomson_debuts_one_chip_hdtv_system

[2.3] Leonardo Chiariglione awards https://leonardo.chiariglione.org/public-life/awards/

[2.4] see pdf and https://www.eetimes.com/apis-help-stmicroelectronics-grow-set-top-box-business/#

[2.5] see pdf and https://www.edn.com/sgs-thomson-microelectronics-named-number-one-mpeg-decoder-supplier-by-dataquest/

[2.6] see pdf and http://www.fundinguniverse.com/company-histories/stmicroelectronics-nv-history/

[2.7] https://books.google.it/books?id=C-bgBwAAQBAJ&pg=PA234&lpg=PA234&dq=stv3200+sgs-thomson&source=bl&ots=HrOQA0LEvS&sig=ACfU3U0vpZ7K5HJzJ1qJWjjlk4yfzPW82w&hl=it&sa=X&ved=2ahUKEwj2lZ2p35HwAhWBzKQKHVrmBBMQ6AEwEnoECBUQAw#v=onepage&q=stv3200%20sgs-thomson&f=false

[2.8] http://www.quretec.com/u/vilo/edu/2002-03/Tekstialgoritmid_I/Loengud/Loeng7_Compression/www.faqs.org/faqs/compression-faq/part1/preamble.html

[2.9] https://books.google.fr/books?id=UsH8CAAAQBAJ&lpg=PA501&ots=OJr_PN0LKo&dq=stv3200%20DCT&hl=fr&pg=PA501#v=onepage&q=stv3200%20DCT&f=false

[2.10] http://ciclops.org/ir_index/208/In-Orbit?js=1

[2.11] https://en.wikipedia.org/wiki/Allan_Alcorn

[2.12] https://www.amazon.com/dp/B096G6TSF9/

[2.13] Computer History Museum, Oral History of Allan Alcorn Recorded: April 26, 2008 and May 23, 2008 Mountain View, California https://archive.computerhistory.org/resources/access/text/2012/09/102658257-05-01-acc.pdf

3) STMicroelectronics documents

[3.1] De la Transformée en Cosinus Discrète aux Processeurs Applicatifs Multimédia, Alain Artiéri, Septembre 2007, STMicroelectronics presentation

  4) Patents

[4.1] USRE34734 "Integrated digital signal processing circuit for performing cosine transformation”

[4.2] US4872134A "Signal processing integrated circuit for row and column addition of matrices of digital value”

[4.3] US5600837A "Multitask processor architecture having a plurality of instruction pointers”

[4.4] Integrated signal processing circuit for online and column summation of digital value matrices https://patents.google.com/patent/FR2608802B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old

[4.5] Transposition memory for data processing circuit https://patents.google.com/patent/FR2617621B1/en?inventor=alain+artieri&oq=alain+artieri&sort=old

[4.6] Method and apparatus for addressing a memory area of an MPEG decoder

[4.7] publication number US6081298A MPEG decoder with reduced memory capacity

[4.8] publication number 0 298 002 of 29.06.1988 Memoire de transposition pour circuit de traitement de donnes

[4.9] publication number FR2649226A1 BREWING CIRCUIT OF DATA

[4.10] publication number FR2650462B1 Device for converting a line scanning into a scanning in vertical saw teeth by bands; https://patents.google.com/patent/FR2650462B1/en?q=H04N3%2f30&inventor=alain+artieri&sort=old

[4.11] publication number EP0368731B1 Process and circuit for image representation signal filtration

[4.12] Circuit pour effectuer une transformation linéaire sur un signal numérique Patent issuer and number eu EP 0241352 A1 https://patents.google.com/patent/EP0241352A1

[4.13] Additionneur binaire comportant un opérande fixé, et multiplieur binaire parallèle-série comprenant un tel additionneur Patent issuer and number eu EP 0262032 B1 https://patents.google.com/patent/EP0262032B1

[4.14] Integrated digital signal processing circuit for performing cosine transform

[4.15] Signal processing integrated circuit for row and column addition of matrices of digital values

5) Honors

[5.1] Two European IT Prizes Awarded to STMicroelectronics  

6) MPEG mentions

[6.1] Chiariglione and the birth of MPEG https://www.chiariglione.org/public-life/media/leonardo_ieee/prof.htm

7) Letters of support

[7.1] By Alain ARTIERI, the creator of MPEG2 chips in ST.

[7.2] Endorsement by Nicolas Demassieux, Senior Vice President of Research, Orange, assistant professor at ENST (now Telecom ParisTech) mid 80s.

[7.3] Endorsement by Leonardo Chiariglione, PhD Eng, the father of MPEG

[7.4] Endorsement by Professor Emeritus Fabio Rocca, Politecnico di Milano, pioneer of motion estimation in 1969

[7.5] Endorsement by Hisafumi Yamada, former Sony US TV CTO

[7.6] Endorsement by Sylvain Kritter, Product Director VIZYON

[7.7] Endorsement by Professor Emeritus Virginio Cantoni, University of Pavia. Also Fellow of the IAPR since 1994 and Life Fellow of the IEEE (he was Fellow since 1997). In 2006 the President of the Italian Republic conferred to Professor Virginio Cantoni the title of ‘Commander of the Order of Merit of the Italian Republic’.

[7.8] Endorsement by Professor Emeritus Mariagiovanna Sami, Politecnico di Miano. Past Full Professor, Digital Systems. IEEE Life Member. Minerva prize for woman scientists, the Seymour Cray prize for contributions in the area of parallel processing, the Herbert A. Simon Gold Medal, assigned by the Society for Design and Process Science. Cavaliere della Repubblica Italiana (knight of the Italian republic). Member of the Italian National Science Academy (“Dei Quaranta”).

[7.9] Endorsement by Professor Andrea Basso who directly contributed to MPEG standards since 90s while in EPFL and in AT&T Bell labs and AT&T Research Laboratories, Middletown, NJ, USA

[7.10] Endorsement by Allan Alcorn, Apple Fellow in the late 80s in the Advanced Technology Group at Apple Computer, Cupertino USA

[7.11] Endorsement by Jeff Cooper, former Senior R&D Engineer at Thomson Multimedia - RCA in the 90s responsible for DirectTV MPEG2 set top box design based on STi3500.

[7.12] Endorsement by Shigenobu Minami, former R&D team leader at Toshiba for MPEG4 standardization and LSI planning in the 90s.

Supporting materials (supported formats: GIF, JPEG, PNG, PDF, DOC): All supporting materials must be in English, or if not in English, accompanied by an English translation. You must supply the texts or excerpts themselves, not just the references. For documents that are copyright-encumbered, or which you do not have rights to post, email the documents themselves to ieee-history@ieee.org. Please see the Milestone Program Guidelines for more information.


Please email a jpeg or PDF a letter in English, or with English translation, from the site owner(s) giving permission to place IEEE milestone plaque on the property, and a letter (or forwarded email) from the appropriate Section Chair supporting the Milestone application to ieee-history@ieee.org with the subject line "Attention: Milestone Administrator." Note that there are multiple texts of the letter depending on whether an IEEE organizational unit other than the section will be paying for the plaque(s).

Please recommend reviewers by emailing their names and email addresses to ieee-history@ieee.org. Please include the docket number and brief title of your proposal in the subject line of all emails.