Single Instruction Multiple Data (SIMD)

What is SIMD (Single Instruction Multiple Data)?

SIMD, or Single Instruction, Multiple Data refers to a specific type of multi-processing architecture or system. Technically, it performs a single operation at the same time on multiple pieces of data.

These units typically receive their inputs as two vectors, with each having its own set of operands. They perform similar operations on both these operands and produce one vector with the results.

Understanding SIMD (Single Instruction Multiple Data)

What is SIMD (Single Instruction Multiple Data)

Single Instruction, Multiple Data signifies the hardware components that carry out the same operations on several data operands at the same time.

In simpler words, SIMD refers to the specific type of organization that comprises several processing units that are under the management of a common Control Unit.

This means that the Control Unit issues the same instruction to all the processors, which, however, works on different types of data.

The SIMD units also share the memory unit but that has multiple modules. This helps in communicating with all the processors at the same time.

Ideally, SIMD is mainly used in array processing machines, though these are also found in vector processors. Therefore, the parallel processing form of SIMD is also referred to as array processing.

In these types of processors, there is a 2D grid of processing elements. These transmit a stream of instructions from the CPU, and while doing that, all the elements carry them out simultaneously.

The capacity of these processors need not be very high or complex to perform such calculations. This is achieved as follows:

Every processing element is coupled to its respective four adjacent neighbors for exchanging data. These connections around the end may be on both rows and columns.
Every element exchanges values following a specific and simple data path with each of its neighbors.
All of these processing elements have a few registers and little local memory for storing data.
There is also a network register that helps in moving the values from the respective neighbors.
The central processor, all the time, broadcasts instructions for moving the values across the network and registering a step up, down, right, or left.
There is an ALU or Arithmetic Logic Unit in each processing element which helps in performing arithmetic instructions that are given by the control processor.

With all these features used, a series of instructions is sent repeatedly for implementing reiterative loops.

The control processor determines that every processing element has its own constituent of temperature to achieve the requisite accuracy.

This is done by setting the internal status bit of each element to 1, which indicates the condition.

There is a grid interconnect feature included that lets the controller identify these status bits and make sure that they have been set at the conclusion of an iteration.

All these features of SIMD make the array processors much more efficient to perform operations simultaneously and highly specialized to perform numerical problems. These can be expressed either in a vector format or as a matrix.

Why is SIMD Fast?

The main reason that SIMD instructions are fast is that they allow for vector instructions. This helps the codes, in particular, run much faster.

Typically, the vector instructions are a special type of instruction which can handle vectors of shorter length, often between 2 and 16, of integers, characters, and floats in parallel.

The operations are carried out simultaneously with the help of the additional bits of space available.

This type of vectorization is often referred to as auto-vectorisation in which the compiler vectorized the code that is needed to be executed. However, this may not be an easy process when more complex codes are involved.

This is because the compiler then may not be able to identify what code in particular is to be vectorized automatically.

In such a situation, the vectorization of the codes is to be done manually with the help of SIMD vectorisation. Even then, the process will not be slowed down drastically.

Why is SIMD More Efficient?

The efficiency of the SIMD instructions is enhanced by their ability to vectorize codes. This helps in constructing a more efficient way to explore data-level parallelism.

It is because it allows carrying out several data operations at the same time with the help of only one instruction.

It is very performance sensitive, and therefore, when done correctly using the correct vector intrinsic, it helps supplement the C or C++ codes and offers an exceptionally good performance.

In fact, the modern processors with vectors under the hood process one-dimensional sets of data individually, but the intrinsics are implemented in the compilers directly, unlike library functions.

Therefore, codes written must be tailored to these vectors in order to maximize the performance.

The vector versions can operate anywhere from three to eight times faster than the scalar codes. This is because the vector registers can fit twice as many 16-bit integers as 32-bit floats.

This allows the system to process double the number of pixels in parallel in about the same amount of time, thereby increasing its efficiency.

The compiler is also able to hide some latency that is typically associated with instruction reordering. This enhances the efficiency of the code in performing different activities such as:

Computing the product
Incrementing the pointers
Adding the product to the accumulator
Testing for loop exit conditions

All these help in interleaving the scalar and vector instructions and hiding the latency between the two, which increases efficiency.

Are Graphics Cards SIMD?

Yes, usually the graphics cards use a SIMD model. This makes the Graphics Processing Unit hardware more efficient and cheaper. However, the multiple added constraints in the SIMD model make programming a bit harder.

This particular instruction issuing mechanism is very useful and it is no coincidence that the graphics cards gain a lot in different areas such as:

In performance
In die area
In efficiency

In fact, it is no surprise that graphics cards have been using SIMD units since their early days in order to put vector instructions into practice.

Ideally, 3D workloads are basically everything about vector operations.

Therefore, it is no surprise that they have programmable shaders using an assembly-like language for issuing shading instructions, especially those particular instructions that operate normally on 4-part vectors.

Specific linear transformations are required for rendering 3D scenes especially for particular attributes such as:

Normal
Position
Texture coordinates

All these involve vector-matrix multiplications for performing several vector-vector operations on dot products.

These are, more often than not, best performed on a 4-part vector that represents homogeneous coordinates.

Use of these instructions helps in performing various 3-part and 4-part vector operations that eventually help in multiple ways such as:

In determining the color of individual pixels and vertices
In making difficult lighting calculations
In representing colors in RGB or RGBA format
In determining the directions of the incoming light, surface normal and reflection
In offering more refined support for shader control flow
In processing multiple shader invocations or threads, when needed, in parallel
In implementing less intuitive control flow
In handling larger numbers of instructions by the shaders in parallel by feeding various SIMD blocks.

Therefore, SIMD in the GPUs enables using complex instructions comprising multiple operations that are needed to be executed in parallel and leveraging the benefits in a variety of interesting ways.

However, this technique should not be considered the same as SMT or Simultaneous Multithreading technology, which is also used by the GPUs often, in which the CPU schedules the instructions of other waves.

Here the waves have to wait for instructions and a long-latency operation, such as a memory read, to complete.

SIMD Architecture Example

A good example of the SIMD architecture is the Wireless MMX unit which resides in a SIMD coprocessor. This is an extension of the XScale microarchitecture.

This 64-bit programming model typically characterizes three types of packed data such as an 8-bit byte, a 16-bit half word, and a 32-bit word. There is also a 64-bit double word.

Some other noteworthy examples of SIMD applications are as follows:

SIMD capabilities are also found in supercomputer systems. In fact, it was first found in these systems as early as the 1970s.
Several CPUs today also use SIMD to help perform several multimedia functions such as adjusting the color and brightness of a video or the volume of audio.
Function Multi Versioning or FMV, which is a subroutine in a library or the program, uses it to duplicate and compile several instruction set extensions and also to decide which particular instruction is to be used during run time.
Library Multi Versioning or LMV also uses it to duplicate the whole programming library for several instruction set extensions. The program or the operating system decides which particular instruction is to be used during run time.

SIMD instruction sets are also used to create a high-performing interface for the Dart programming language to benefit web programs.

It was used for the first time in 2013 by John McCutchan and consisted of two types of interfaces namely, Float32x4, a 4 single-precision floating point values, and Int32x4, a 4 32-bit integer values.

GAAP or Generally Accepted Accounting Principles is one significant commercial application of SIMD-only processors.

Developed by Lockheed Martin, the modern incarnations of it help in real-time video processing applications such as:

De-interlacing
Image noise reduction
Image enhancement
3D graphics rendering
Adaptive video compression

It also helps in the conversion between different frame rates and video standards like NTSC to PAL and vice versa, NTSC to HDTV and vice versa and others.

In video games, SIMD seems to have a more ubiquitous presence, wherein almost every modern video game console designed since 1998 has a SIMD processor incorporated at some place in its architecture.

Is SIMD Single Core?

Yes, it is, but the SIMD instructions allow carrying out multiple calculations at the same time even on a single core if a register is used that is several times larger than the data that is being processed.

This means that a system can perform as many as eight 32-bit calculations by using a register of 256 bits with only one machine code instruction.

What are the Uses of SIMD?

The SIMD instructions are more commonly used in processing 3D graphics. Modern graphics cards usually come with embedded SIMD today and have taken over this specific task from the CPU for the most part.

However, there are a few systems that also consist of permute operations.

These have the essentials inside the vectors which makes them ideal and useful for data processing and compression in particular.

Advantages

Helps in multimedia applications
Helps in better control flow
Helps in loading a large number of value at once in blocks
No waiting required to retrieve the following instruction or pixel
Instructions operate on a single operation for all data
Creation, analysis and debugging of the programs are easy
Simpler and faster operation
Inter-process communication is more effective due to implicit synchronization
Several scalar operations and control flow instructions may overlap over the Control Unit
Needs less memory since only one copy of the instruction is stored in it
Needs single instruction decoder which reduces cost

Disadvantages

Vectorizing all algorithms is not easy
The register files are larger which increases chip area and power consumption
Most compilers do not produce these instructions from a C program
Restrictions on data alignment
Need for permute operations to gather data in the SIMD registers and scatter it to the right locations may be inefficient and tricky
Unavailability of specific instructions such as three-operand addition or rotation
Less efficient in carrying out complex instructions
Architecture-specific instruction sets
Several implementations of vector codes required due to the different sizes of the registers provided by the architectures

SIMD vs MIMD

SIMD uses a single instruction for multiple data, but in comparison, MIMD uses multiple instructions for multiple data.
SIMD needs smaller memory, but in comparison, MIMD uses more memory for operations.
SIMD is less costly than MIMD
There is a single decoder in SIMD, but in comparison, MIMD uses multiple decoders.
SIMD implements tacit or latent synchronization, but MIMD uses accurate or explicit synchronization.
In SIMD, synchronous programming is followed, while in MIMD asynchronous programming is followed.
SIMD operations are much simpler in comparison to those in MIMD.
In terms of performance, SIMD is less efficient than MIMD.
SIMD requires hardware-related optimizations and support for software, but in comparison, MIMD requires communication-related optimizations and support for the same.
A SIMD machine carries out the same operation on different data items, but a MIMD machine executes different operations on a potentially diverse set of data.
The SIMD machines typically have simpler data paths, but the MIMD machines do not.
The SIMD machines are not as flexible as the MIMD machines.
In SIMD, the processing units share one Control Unit but are mutually separate, but in comparison, in MIMD, each processing unit performs just like an individual processor having its own Control Unit.
A few specific types of problems map much better in SIMD than in MIMD.
Most modern GPUs are SIMD machines, but in comparison, most modern CPUs are MIMD machines.
In SIMD, problems that typically need a lot of computation are solved by processors carrying out the same operation in parallel, but in MIMD, they are broken down into parts and each assigned to a different processor for simultaneous execution.
The SIMD processors are normally smaller, simpler, and faster in comparison to the MIMD processors.

How Are SIMD Architectures Employed?

Typically, SIMD architectures are used by exploiting parallelism. This is done by using concurrent operations across a massive set of data.

This pattern is most useful to solve those particular issues that have multiple data and need to be upgraded on a regular and wholesale basis.

The result is a more dynamic and powerful operation that helps in doing multiple scientific calculations.

Do Supercomputers Use SIMD?

Yes, it can be said that in a way, modern supercomputers use SIMD instructions.

In most cases, modern supercomputers typically are referred to as a bunch of Multiple Instruction Multiple Data or MIMD computers, where these instructions use a shorter vector of SIMD instructions.

Conclusion

So, reaching to the end of this article, now you know how Single Instruction, Multiple Data helps in computing, and how it is efficiently used by both the CPUs and GPUs.

It needs less memory and a single decoder to operate and also has a simpler data path which makes it efficient, fast, and less costly.