by Merv Adrian | October 17, 2016 | Comments Off on Multiprocessing Nostalgia
My esteemed Colleague Henry Cook offers this trip down memory lane (pun intended)
An office email exchange on the history of multi-processors prompted a quick recap, and it was suggested I blog it as others might find it interesting.
Once upon a time computers had a single CPU and a single register, they’d get faster year by year, but it didn’t take long before applications outgrew what those CPUs could deliver.
Initially we’d do multiprocessing programmed into the application to have multiple single CPU systems to share work between them, passing status and data across network links. This rapidly evolved into symmetric multiprocessing (SMP) systems. Servers sharing a set of disks, memory and a high speed bus with the operating system dispatching tasks across a collection of identical CPUs.
Multiprocessor systems didn’t need to be symmetric, and you can also have asymmetric multiprocessing (AMP) where work can be spread across processors of different types. Usually this means offloading some of the processing to a specialist processor from a general purpose processor. Using a GPU to complement a general purpose CPU would be an example today.
SMP processors had their limits, because they shared key resources, and with big workloads they’d conflict with each other for access to disk data and memory. They were also restricted in the amount of memory they could access, so – Non Uniform Memory Access (NUMA) was invented, bringing the idea that not all memory is created equal. Multiple processors accessed lots of memory, but some would be local each CPU, and accessed very quickly, some would be further away, and this would take longer to get to. Every CPU could get to every piece of memory, but not with a uniform access time (hence “Non-Uniform”). Putting the data to be processed by a CPU near that CPU helped optimize performance
But this still wouldn’t work for the very big data processing tasks for Data Warehouse systems, which were very large, because they enabled analysis across an organization and across time, by taking a copy of the data from every other system in the site, and holding several years of history.
Thus Shared Nothing / Massively Parallel Processing (SN/MPP) systems were invented, beginning with the Teradata Data Base Computer DBC/1012 (1012 = Tera for Terabyte, see the historic document from my bookshelf below). For the first time systems could be expanded linearly, to pretty much any practical size, simply by adding more servers.
The original Teradata had Access Module Processors (AMPs), 1 CPU x 1 core processing nodes based on i286, i386, i486 uniprocessor chips. With the move to multi CPU servers, then multicore CPUs, and the move from the Teradata OS to UNIX, AMPs because VAMPS (Virtual Access Module Processors, that looked to DBMS like lots of individual CPU’s.)
Some people referred to big SMP as MPP (without the shared nothing) with shared memory often enhanced by NUMA (Non Uniform Memory Access) that makes the memory in a cluster of servers look like one big shared memory. Sequent and Pyramid were machines of this type.
You could also mix these architectures, notably with Netezza’s architecture (now IBM) as Asymetric Massively Parallel Processing. (see: http://www.redbooks.ibm.com/redpapers/pdfs/redp4725.pdf)
This is where you have shared nothing parallel processing going on, but different types of processors are sharing the different types of work. With Netezza this was the difference between the Intel CPU’s doing the sophisticated parallel SQL optimization work, and the very large numbers of simple and inexpensive Field Programmable Gate Array (FPGA) processors attached, and blasting through the blocks of data streaming off the disks they were attached to, all the while doing low level SQL operations at hardware speed.
Now we have available, SMP within nodes, MPP between nodes, symmetric and asymmetric processors, and in memory processing, plus virtualization to combine and recombine elements of processing but it’s interesting to recall how we got here.
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.