Parallel Computing Initiative

 

The design models in hardware processing industry are changing. We are moving to multi core or multi CPU microprocessors instead of increasing the processing or clock speed of existing CPUs. Higher clock cycles results in more power consumption and heat emission and thus this shift.

Multi-Core or many-core systems provide both opportunity and challenges to software developers. These systems promise to increase the computing speed many times but only if software developed is capable of utilizing their power. Many-core systems will also help in increasing the power of High Performance Computing (HPC) Systems.

Serial programs which we write today are not capable of utilizing the power of multi-core or many-core systems. We need to write parallel programs which can scale up or down based on the computing power available on the systems.

While it is possible today to write parallel programs, it’s a complicated task and require special skills. Also maintaining and debugging such programs requires lot of effort.

Microsoft recently established the Parallel Computing Initiative that promises to deliver the following:

·         Applications: Applications that harness the power of many-core systems and deliver the next generation of experiences.

·         Domain Libraries: System blocks like Math library or image-processing library which deliver faster results by utilizing the power of many-core systems automatically.

·         Programming Models and Languages: that will enable developers of all levels to successfully design, develop, debug and deliver parallel software.

·         Developer Tooling: Integrated development environment for application lifecycle management of parallel computing era.

·         Runtime, platform and operating systems: enabling them to better manage the requests arising from applications for many-core resources.

As part of Parallel Computing Initiative, Microsoft recently released the CTP of Parallel Extensions to .NET framework which aims to deliver following benefits:

·         Enable developers to express parallelism easily.

·         Improve the efficiency and scalability of parallel applications.

·         Simplify debugging parallel applications.

Parallel programming framework will be core part of .NET framework 4.0 and Visual Studio 2010 debugger will be enhanced to identify concurrency issues.

Parallel extensions to programming languages and native runtime are also planned.

Parallel Extension to .NET Framework

Overview

·         Parallel Extensions is a managed library and requires the .NET Framework 3.5 to be installed on the system

·         Provides performance gains on multi core or multi processor machines.

·         Automatically scales based on the number of cores or processors available on the machine.

·         Introduces minimum overheads even if used on single processor machine.

·         Recommended to be installed directly on hardware and not using virtualization technologies.

·         Is a framework extension and not language extension. So can be used by programming languages like c#, VB.NET or F#.

·         Parallel programming framework will be core part of .NET 4.0 framework.

·         Visual Studio 2010: The debugger has been enhanced to identify concurrency issues. Two new features added are MultiStack and a Task List views. Introduction of concurrency analysis report for performance analysis of applications.

Approaches to Parallelism

Parallel extensions to .NET framework provide three ways to introduce parallelism into execution of code and based on the situation and requirement appropriate one can be selected.

1.    Declarative data parallelism: Applicable while querying data. This approach uses Parallel Language Integrated Query or Parallel LINQ or PLINQ which is an implementation of LINQ-to-Objects that executes queries in parallel.

 

2.    Imperative data parallelism: Applicable to common data oriented operations such as for and foreach loops.

 

3.    Imperative task parallelism: While the above two were data centered or oriented ways of parallelism, this one more for performing parallel tasks via expressions and statements.