Architecting Solutions for the Manycore Future

Architecting Solutions for the Manycore FutureTalbott CrowellThirdM

This talk will focus solution architects toward thinking about parallelism when designing applications and solutionsThreads vs. Tasks using TPL LINQ vs. PLINQ Object Oriented vs. Functional ProgrammingThis talk will also compare programming languages, how languages differ when dealing with manycore programming, and the different advantages to these languages. Abstractmanycore

Patrick Gelsinger, Intel VP February 2001, San Francisco, CA2001 IEEE International Solid-State Circuits Conference (ISSCC) If scaling continues at present pace, by 2005, high speed processors would have power density of nuclear reactor, by 2010, a rocket nozzle, and by 2015, surface of sun.Intel stock dropped 8% on the next day“Business as usual will not work in the future.”

The Power Wall: CPU Clock SpeedManycore->Multicore->Single core->From Katherine Yelick’s “Multicore: Fallout of a Hardware Revolution”

In 1966, Gordon Moore predicted exponential growth in number of transistors per chip based on the trend from 1959 to 1965Clock frequencies continued to increase exponentially until they hit the power wall in 2004 at around 3 to 4 GHz1971, Intel 4004 (first single-chip CPU) – 740 kHz1978, Intel 8086 (orgin of x86) – 4.77 MHz1985, Intel 80386DX – 16 MHz1993, Pentium P5 – 66 MHz1998, Pentium II – 450 MHz2001, Pentium II (Tualatin) – 1.4 GHz2004, Pentium 4F – 3.6 GHz2008, Core i7 (Extreme) – 3.3 GHzIntel is now doubling cores along with other improvements to continue to scaleEffect of the Power WallThis trend continues even todayThe Power WallEnter Manycore

Manycore, What is it?Manycore, Why should I care?Manycore, What do we do about it?FrameworksTask Parallel Library (Reactive Extensions and .NET 4)Languages, paradigms, and language extensionsF#, functional programming, LINQ, PLINQToolsVisual Studio 2010 Tools for ConcurrencyAgenda: Manycore Future

Single core: 1 processor on a chip die (1 socket)Many past consumer and server CPU’s (some current CPU’s for lightweight low power devices)Including CPU’s that support hyperthreading, but this is a grey areaMulticore: 2 to 8 core processors per chip/socketAMD Athlon 64 X2 (first dual-core desktop CPU released in 2005)Intel Core Duo, 2006 (32 bit, dual core, for laptops only)Core Solo was a dual core chip with one that doesn’t workIntel Core 2 (not multicore, instead a brand for 64 bit arch)Core 2 Solo (1 core)Core 2 Duo (2 cores)Core 2 Quad (4 cores)Manycore: more than 8 cores per chipCurrently prototypes and R&DManycore, What is it?

High-end Servers 2001-2004IBM Servers 2001 - IBM POWER4 PowerPC for AS/400 and RS/6000 “world's first non-embedded dual-core processor”Sun Servers 2004 - UltraSpark IV – “first multicore SPARC processor”Desktops/Laptops 2005-2006AMD Athlon 64 X2 (Manchester) May 2005 “first dual-core desktop CPU”Intel Core Duo, Jan 2006 Intel Pentium (Allendale) dual core Jan 2007Windows Servers 2006Intel Xeon (Paxville) dual core Dec 2005AMD Opteron (Denmark) dual core March 2006Intel Itanium 2 (Montecito) dual core July 2006Sony Playstation 3 – 20069 core Cell Processor (only 8 operational) - Cell architecture jointly developed by Sony, Toshiba, and IBMMulticore trends from servers to gaming consoles

Power Mac G5 - Mid 20032 x 1 core (single core) IBM PowerPC 970Mac Pro - Mid 20062 x 2 core (dual core) Intel Xeon (Woodcrest)Mac Pro - Early 20082 x 4 core (quad core) Intel Xeon (Harpertown)In 5 years number of cores doubled twice on Apple’s high end graphics workstationFrom 2 to 4 to 8Macintosh multicore trend

The chip is just designed for research efforts at the moment, according to an Intel spokesperson."There are no product plans for this chip. We will never sell it so there won't be a price for it," the Intel spokesperson noted in an e-mail. "We will give about a hundred or more to industry partners like Microsoft and academia to help us research software development and learn on a real piece of hardware, [of] which nothing of its kind exists today." http://redmondmag.com/articles/2009/12/04/intel-unveils-48-core-cloud-computer-chip.aspxMicrosoft said it had already put SCC into its development pipeline so it could exploit it in the future. http://news.bbc.co.uk/2/hi/technology/8392392.stm48 Core Single-chip Cloud Computer (SCC)

Why should I care?(about Manycore)

Hardware is changingProgramming needs to change to take advantage of new hardwareConcurrent ProgrammingParadigm Shift Designing applicationsDeveloping applicationsManycore, Why should I care?

“The computer industry is once again at a crossroads. Hardware concurrency, in the form of new manycore processors, together with growing software complexity, will require that the technology industry fundamentally rethink both the architecture of modern computers and the resulting software development paradigms.”Craig MundieChief Research and Strategy OfficerMicrosoft CorporationJune 2008First paragraph of the Forward of Joe Duffy’s preeminent tome “Concurrent Programming on Windows”Concurrent Programming

Excerpt from Mark Reinhold’s Blog post: November 24, 2009The free lunch is over. Multicore processors are not just coming—they’re here. Leveraging multiple cores requires writing scalable parallel programs, which is incredibly hard. Tools such as fork/join frameworks based on work-stealing algorithms make the task easier, but it still takes a fair bit of expertise and tuning. Bulk-data APIs such as parallel arrays allow computations to be expressed in terms of higher-level, SQL-like operations (e.g., ﬁlter, map, and reduce) which can be mapped automatically onto the fork-join paradigm. Working with parallel arrays in Java, unfortunately, requires lots of boilerplate code to solve even simple problems. Closures can eliminate that boilerplate. “It’s time to add them to Java.”http://blogs.sun.com/mr/entry/closures“There’s not a moment to lose!”

Herb Sutter 2005Programs are not doubling in speed every couple of years for free anymoreWe need to start writing code to take advantage of many coresCurrently painful and problematic to take advantage of many cores because of shared memory, locking, and other imperative programming techniques“The Free Lunch Is Over”

Is this just hype?Another Y2K scare?Fact:CPU’s are changingProgrammers will learn to exploit new architecturesWill you be one of them?Wait and see?You could just wait and let the tools catch up so you don’t have to think about it. Will that strategy work?Should you be concerned?

Just tools or frameworks will not solve the manycore problem aloneImperative programming by definition has limitations scaling in a parallel wayImperative programming (C, C++, VB, Java, C#)Requires locks and synchronization code to handle shared memory read/write transactions Not trivialDifficult to debugTools and frameworks may help, but will require different approach to the problem (a different paradigm) to really take advantage of the toolsThe Core Problem

Some frameworks are designed to be single threaded, such as ASP.NETBest practices for ASP.NET applications recommend avoiding spawning new threadsASP.NET and IIS handle the multithreading and multiprocessing to take advantage of the many processors (and now many cores) on Web Servers and Application ServersWill this best practice remain true?Even when server CPU’s have hundreds or thousands of cores?Will it affect all programmers?

What do we do about it?(How do we prepare for Manycore)

Identify where the dependencies areIdentify where you can parallelizeUnderstand the tools, techniques, and approaches for solving the piecesPut them together to understand overall performancePOC – Proof of ConceptTest, test, testPerformance goals up frontUnderstand Problem Domain

FrameworksTask Parallel Library (TPL)Reactive Extensions for .NET 3.5 (Rx)Used to be called Parallel Extensions or PFxBaked into .NET 4Programming paradigms, languages, and language extensionsFunctional programmingF#LINQ and PLINQToolsVisual Studio 2010 Tools for ConcurrencyManycore, What do we do about it?

Parallelism vs. ConcurrencyTask vs. Data ParallelismParallel Programming Concepts

Concurrency or Concurrent computingMany independent requestsWeb Server, works on multi-threaded single core CPUSeparate processes that may be executed in parallelMore general than parallelismParallelism or Parallel computingProcesses are executed in parallel simultaneouslyOnly possible with multiple processors or multiple coresYuan Lin: compares to black and white photography vs. color, one is not a superset of the otherhttp://www.touchdreams.net/blog/2008/12/21/more-on-concurrency-vs-parallelism/Parallelism vs. Concurrency

Task Parallelism (aka function parallelism and control parallelism)Distributing execution processes (threads/functions/tasks) across different parallel computing nodes (cores)http://msdn.microsoft.com/en-us/library/dd537609(VS.100).aspxData Parallelism (aka loop-level parallelism)Distributing dataacross different parallel computing nodes (cores)Executing same command over every element in a data structurehttp://msdn.microsoft.com/en-us/library/dd537608(VS.100).aspxTask vs. Data ParallelismSee MSDN for .NET 4, Parallel Programming, Data/Task Parallelism

Parallel Programming in the .NET Framework 4 Beta 2 - TPL

Reference System.ThreadingUse Visual Studio 2010 or .NET 4For Visual Studio 2008Download unsupported version for .NET 3.5 SP1 from Reactive Extensions for .NET (Rx)http://msdn.microsoft.com/en-us/devlabs/ee794896.aspxCreate a “Task”How to use the TPLFileStream fs = new FileStream(fileName, FileMode.CreateNew); var task = Task.Factory.FromAsync(fs.BeginWrite, fs.EndWrite, bytes, 0, bytes.Length, null);

Use Task classTask Parallelism with the TPL// Create a task and supply a user delegate // by using a lambda expression.vartaskA = new Task(() => Console.WriteLine("Hello from taskA."));// Start the task.taskA.Start();// Output a message from the calling thread.Console.WriteLine("Hello from the calling thread.");

Task<TResult>Getting return value from a TaskTask<double>[] taskArray = new Task<double>[]{ Task<double>.Factory.StartNew(() => DoComputation1()), // May be written more conveniently like this:Task.Factory.StartNew(() => DoComputation2()),Task.Factory.StartNew(() => DoComputation3())};double[] results = new double[taskArray.Length];for (inti = 0; i < taskArray.Length; i++) results[i] = taskArray[i].Result;

Task resembles new thread or ThreadPool work item, but higher level of abstractionTasks provide two primary benefits over Threads: More efficient and scalable use of system resourcesMore programmatic control than is possible with a thread or work itemTasks vs. Threads

Behind the scenes, tasks are queued to the ThreadPoolThreadPool now enhanced with algorithms (like hill-climbing) that determine and adjust to the number of threads that maximizes throughput. Tasks are relatively lightweightYou can create many of them to enable fine-grained parallelism. To complement this, widely-known work-stealing algorithms are employed to provide load-balancing..Tasks and the framework built around them provide a rich set of APIs that support waiting, cancellation, continuations, robust exception handling, detailed status, custom scheduling, and more.Tasks

Instead of:Use: Data Parallelism with the TPLfor (inti = 0; i < matARows; i++) { for (int j = 0; j < matBCols; j++) { ... }} Parallel.For(0, matARows, i => { for (int j = 0; j < matBCols; j++) { ... }}); // Parallel.For

Use Tasks not ThreadsUse Parallel.For in Data Parallelism scenariosOr…Use AsyncWorkflosw from F#, covered laterUse PLINQ, covered laterTPL Summary

1930’s: lambda calculus (roots)1956: IPL (Information Processing Language) “the first functional language”1958: LISP “a functional flavored language”1962: APL (A Programming Language)1973: ML (Meta Language)1983: SML (Standard ML)1987: Caml (Categorical Abstract Machine Language ) and Haskell1996: OCaml (Objective Caml)2005: F# introduced to public by Microsoft Research2010: F# is “productized” in the form of Visual Studio 2010Functional programming has been around a long time (over 50 years)

Most functional languages encourage programmers to avoid side effectsHaskell (a “pure” functional language) restricts side effects with a static type systemA side effectModifies some stateHas observable interaction with calling functions Has observable interaction with the outside worldExample: a function or method with no return valueFunctional programming is safe

Language Evolution (Simon Payton-Jones)C#, VB, Java, C are imperative programming languages. Very useful but can change the state of the world at anytime creating side effects.Nirvana! Useful and SafeF#Haskell is Very Safe, but not very useful. Used heavily in research and academia, but rarely in business.http://channel9.msdn.com/posts/Charles/Simon-Peyton-Jones-Towards-a-Programming-Language-Nirvana/

When a function changes the state of the programWrite to a file (that may be read later)Write to the screenChanging values of variables in memory (global variables or object state)Side Effect

Compare SQL to your favorite imperative programming languageIf you write a statement to store and query your data, you don’t need to specify how the system will need to store the data at a low levelExample: table partitioningLINQ is an example of bringing functional programming to C# and VB through language extensionsFunctional Programming

Use lots of processesAvoid side effectsAvoid sequential bottlenecksWrite “small messages, big computations” codeEfficient Multicore ProgrammingSource: Joe Armstrong’s “Programming Erlang, Software for a Concurrent World”Section 20.1 “How to Make Programs Run Efficiently on a Multicore CPU”

Functional language developed by Microsoft ResearchBy Don Syme and his team, who productized GenericsBased on OCaml (influenced by C# and Haskell)History2002: F# language design started2005 January: F# 1.0.1 releases to publicNot a product. Integration with VS2003Works in .NET 1.0 through .NET 2.0 beta, Mono2005 November: F# 1.1.5 with VS 2005 RTM support2009 October: VS2010 Beta 2, CTP for VS2008 & Non-Windows users2010: F# is “productized” and baked into VS 2010What is F#

Multi-ParadigmFunctional ProgrammingImperative ProgrammingObject Oriented ProgrammingLanguage Oriented ProgrammingF# is not just Functional

Parallel Computing and PDC09ToolsManaged LanguagesAxumVisual F#Visual Studio 2010ParallelDebugger WindowsNative LibrariesManaged LibrariesDryadLINQAsyncAgentsLibraryParallel Pattern LibraryProfiler ConcurrencyAnalysisParallel LINQRxTask ParallelLibraryData StructuresData StructuresMicrosoftResearchNative Concurrency RuntimeTask SchedulerRace DetectionManaged Concurrency RuntimeResource ManagerThreadPoolFuzzingOperating SystemThreadsUMS ThreadsHPC ServerWindows 7 / Server 2008 R2Research / IncubationVisual Studio 2010 / .NET 4Key:

Functional programming has been around a long timeNot newLong historyFunctional programming is safeA concern as we head toward manycore and cloud computingFunctional programming is on the riseWhy another language?

Parallel Programming in the .NET Framework 4 Beta 2 - PLINQ

“F# is, technically speaking, neutral with respect to concurrency - it allows the programmer to exploit the many different techniques for concurrency and distribution supported by the .NET platform” F# FAQ: http://bit.ly/FSharpFAQFunctional programming is a primary technique for minimizing/isolating mutable stateAsynchronous workflows make writing parallel programs in a “natural and compositional style”F# and Multi-Core Programming

Interactive ScriptingGood for prototypingSuccinct = Less codeType InferenceStrongly typed, strict (no dynamic typing)Automatic generalization (generics for free)Few type annotations1st class functions (currying, lazy evaluations)Pattern matchingKey Characteristics of F#

Concurrent Programming with F#

Luke Hoban at PDC 2009F# Program Managerhttp://microsoftpdc.com/Sessions/FT20

Demo – Imperative sumOfSquares

Difficult to turn existing sequential code into parallel codeMust modify large portions of code to use threads explicitlyUsing shared state and locks is difficultCareful to avoid race conditions and deadlocksTwo Problems Parallelizing Imperative Codehttp://www.manning.com/petricek/petricek_meapch1.pdf

Demo – Recursive sumOfSquares

Declarative programming style Easier to introduce parallelism into existing codeImmutability by defaultCan’t introduce race conditionsEasier to write lock-free codeFunctional Programming

Demo – Functional sumOfSquares

From Seq to PSeqMatthew Podwysocki’s Bloghttp://weblogs.asp.net/podwysocki/archive/2009/02/23/adding-parallel-extensions-to-f.aspxAdding Parallel Extensions to F# for VS2010 Beta 2Talbott Crowell’s Developer Bloghttp://talbottc.spaces.live.com/blog/cns!A6E0DA836D488CA6!396.entryParallel Extensions to F#

Demo – Parallel sumOfSquares

Asynchronous WorkflowsControl.MailboxProcessorTask Based Programming using TPLReactive Extensions“The Reactive Extensions can be used from any .NET language. In F#, .NET events are first-class values that implement the IObservable<out T> interface. In addition, F# provides a basic set of functions for composing observable collections and F# developers can leverage Rx to get a richer set of operators for composing events and other observable collections. ”S. Somasegar, Senior Vice President, Developer Division http://blogs.msdn.com/somasegar/archive/2009/11/18/reactive-extensions-for-net-rx.aspxF# Parallel Programming Options

ProblemResize a ton of imagesDemo of Image Processorlet files = Directory.GetFiles(@"C:\images\original")for file in files do use image = Image.FromFile(file) use smallImage = ResizeImage(image) let destFileName = DestFileName("s1", file)smallImage.Save(destFileName)

Asynchronous Workflowslet FetchAsync(file:string) =async { use stream = File.OpenRead(file) let! bytes = stream.AsyncRead(intstream.Length) use memstream = new MemoryStream(bytes.Length)memstream.Write(bytes, 0, bytes.Length) use image = Image.FromStream(memstream) use smallImage = ResizeImage(image) let destFileName = DestFileName("s2", file)smallImage.Save(destFileName) }let tasks = [for file in files -> FetchAsync(file)]let parallelTasks = Async.Parallel tasksAsync.RunSynchronouslyparallelTasks

Tomas PetricekUsing Asynchronous Workflowshttp://tomasp.net/blog/fsharp-webcast-async.aspx

LINQ declaratively specify what you want done not how you want it doneVersus:LINQvar source = Enumerable.Range(1, 10000);varevenNums = from num in source where Compute(num) > 0 select num;var source = Enumerable.Range(1, 10000);varevenNums = new List<int>();foreach (var num in source) if (Compute(num) > 0)evenNums.Add(num);

If I put a counter in Compute(num)?What will happen?var source = Enumerable.Range(1, 10000);varevenNums = from num in source where Compute(num) > 0 select num;private static int Compute(int num) {counter++; if (num % 2 == 0) return 1; return 0;}

LINQ declaratively specify what you want done not how you want it donePLINQDeclaratively specify “As Parallel”Under the hood, the framework will implement “the how” using TPL and threads.PLINQ = Parallel LINQvar source = Enumerable.Range(1, 10000);varevenNums = from num in source where Compute(num) > 0 select num;var source = Enumerable.Range(1, 10000);varevenNums = from num in source.AsParallel() where Compute(num) > 0 select num;

System.Linq.ParallelEnumerableAsParallel()The entry point for PLINQ. Specifies that the rest of the query should be parallelized, if it is possible.

Visual Studio 2010Tools for Concurrency

Steven Toub at PDC 2009Senior Program Manager on the Parallel Computing Platform http://microsoftpdc.com/Sessions/P09-09

Views enable you to see how your multi-threaded application interacts with ItselfHardwareOperating SystemOther processes on the host computerProvides graphical, tabular and textual dataShows the temporal relationships between the threads in your programthe system as a wholeConcurrency Visualizer in Visual Studio 2010

Performance bottlenecksCPU underutilizationThread contentionThread migrationSynchronization delaysAreas of overlapped I/Oand other info…Use Concurrency Visualizer to Locate

Concurrency VisualizerHigh level of Contentions during Async

CPU ViewThreads View (Parallel Performance)Cores ViewViews

CPU ViewAsync uses more of the CPU(s)/cores

Sync uses 1 CPU/coreThreads View

Full testClose up of SyncClose up of AsyncCore View

Tomas Petricek - F# Webcast (III.) - Using Asynchronous Workflowshttp://tomasp.net/blog/fsharp-webcast-async.aspxLuke Hoban - F# for Parallel and Asynchronous Programminghttp://microsoftpdc.com/Sessions/FT20More info on Asychrounous Workflows

The Landscape of Parallel Computing Research: A View from Berkeley 2.0 by David Pattersonhttp://science.officeisp.net/ManycoreComputingWorkshop07/Presentations/David%20Patterson.pdfParallel Dwarfshttp://paralleldwarfs.codeplex.com/More Research

“The architect as we know him today is a product of the Renaissance.” (1)“But the medieval architect was a master craftsman (usually a mason or a carpenter by trace), one who could build as well as design, or at least ‘one trained in that craft even if he had ceased to ply his axe and chisel’(2).” (1)“Not only is he hands on, like the agile architect, but we also learn from Arnold that the great Gothic cathedrals of Europe were built, not with BDUF, but with ENUF”(1). Dana Arnold, Reading Architectural History, 2002(2). D. Knoop & G. P. Jones, The Medieval Mason, 1933(3). Architects: Back to the future?, Ian Cooper 2008The Architecthttp://codebetter.com/blogs/ian_cooper/archive/2008/01/02/architects-back-to-the-future.aspx

visit us at http://fsug.orgThank you. Questions?Architecting Solutions for the Manycore FutureTalbott CrowellThirdM.comhttp://talbottc.spaces.live.comTwitter: @Talbott and @fsug

Architecting Solutions for the Manycore Future

More Related Content

What's hot (11)

Similar to Architecting Solutions for the Manycore Future (20)

More from Talbott Crowell (19)

Recently uploaded (20)

Architecting Solutions for the Manycore Future

Editor's Notes