MAGMA binding: intro and gesvd

This is my first blog post for the JSoC project: MAGMA binding.

In this post, I will try introducing the background of our project, and reveal some recent progresses.

For those who are not so familiar with Julia, you may want to read the official documentation Julia

Remix this to get started with Julia .

MAGMA (Matrix Algebra on GPU and Multicore Architectures) is a collection of next-generation linear algebra libraries for heterogeneous architectures. By completing the Julia interface of MAGMA, we can provide powerful heterogeneous computing toolkits to users who require a large amount of linear algebra computation and face great computational complexity along with huge memory costs, such as condensed matter physicists who focus on tensor networks and machine learning.

Julia is becoming more and more popular and excellent for big data analytics as well as high-performance computation. Also, it is suitable for CPU and GPU computing especially with Julia packages for CUDA (e.g., Project JuliaGPU).

Meanwhile, recent activities of major chip manufacturers, such as Intel, AMD, IBM,and NVIDIA, make it more evident than ever that future designs of microprocessors and large HPC systems will be hybrid/heterogeneous in nature, relying on the integration (in varying proportions) of two major types of components:

While the relative balance between these component types in future designs is not clear, and will likely to vary over time, there seems to be no doubt that future generations of computer systems, ranging from laptops to supercomputers, will consist of a composition of heterogeneous components.

Among the various libraries used for CPU/GPU computing, MAGMA (Matrix Algebra on GPU and Multicore Architectures) is a collection of next-generation linear algebra libraries for heterogeneous architectures. MAGMA is designed and implemented by the team that developed LAPACK and ScaLAPACK, incorporating the latest developments in hybrid synchronization- and communication-avoiding algorithms, as well as dynamic runtime systems. Interfaces for the current LAPACK and BLAS standards are supported to allow computational scientists to seamlessly port any linear algebra reliant software components to heterogeneous architectures.

MAGMA allows applications to fully exploit the power of current heterogeneous systems of multi/many-core CPUs and multi-GPUs to deliver the fastest possible time to an accurate solution within given energy constraints. MAGMA can help scientists and engineers with different work in machine learning and other popular computing techniques.

For the ease of Julia users' CPU/GPU linear algebra computing, one natural thought is to implement the Julia binding for MAGMA. By binding MAGMA to Julia and providing corresponding BinaryProvider.jl for installation, both of which should be fitted in with existing Julia GPU project, we will definitely benefit many MAGMA and Julia users sustainably.

Recently, we have developed the simplest but available wrappers for the routines gesvd. All our work are open-sourced on GitHub

As all the other versions in LAPACK, ScaLAPACK or CUBLAS, gesvd routines use QR iteration to carry the singular value decomposition.

In MAGMA, the gesvd routines are divided into four different subroutines: Sgesvd (for matrices with elements Float32), Dgesvd (Float64), Cgesvd (Complex F32) and Zgesvd (Complex F64).

For Julia users, there will be no need for one task to own distinct methods so instead we used the macro @eval to realize the multiple dispatch. The final method is named by 'gesvd!', and will be able to receive three arguments: jobu and jobvt, which should belong to 'AbstractChar' and indicate how the U or VT should be returned (see MAGMA to find the details), and A which should be an 'AbstractMatrix'.

After one puts these three arguments into the method gesvd!, the method itself will help deal with everything between Julia and MAGMA. The only thing you need to do is waiting for the returned U, S, VT that

or for the complex situations

Currently we use the gesvd in the Julia stdlib to test whether the MAGMA binding works well or not.

The idea is that we first randomly generate a matrix with certain scale such as 2 × 2, and then get the standard answer S from stdlib and get the testing answer s from our wrappers. The criteria for the right calculation is that norm(S .- s) < 1e-7, and if it fails there will be tons of information telling you how it went wrong.

I believe the most common troubles lies in the environment.