Using the Parallel Computing Toolbox with MATLAB on the Eagle System

Learn how to use the Parallel Computing Toolbox (PCT) with MATLAB software on the Eagle system.

Note: Due to an issue with the scheduler and software licenses, we strongly recommend the use of compiled MATLAB code for batch processing. Using the PCT with MATLAB in batch mode may lead to failed jobs due to unavailability of licenses.

PCT provides the simplest way for users to run parallel MATLAB code on a single, multi-core compute node. Here, we describe how to configure your local MATLAB settings to utilize the PCT and provide some basic examples of running parallel code on Eagle.

For more extensive examples of PCT usage and code examples, see the MathWorks documentation.

Configuration in MATLAB R2018b

Configuration of the PCT is done most easily through the interactive GUI. However, the opening of parallel pools can be significantly slower in interactive mode than in non-interactive (batch) mode. For this reason, the interactive GUI will only be used to set up your local configuration. Runtime examples will include batch scripts that submit jobs directly to the scheduler.

To configure your local parallel settings, start an interactive MATLAB session with X11 forwarding (see Running Interactive Jobs on Eagle and Environment Modules on the Eagle System). Open MATLAB R2018b and do the following:

  1. Under the Home tab, go to Parallel > Parallel Preferences.
  2. In the Parallel Pool box, set the "Preferred number of workers in a parallel pool" to at least 36 (the max number of cores currently available on a Eagle compute node).
  3. Click OK.
  4. Exit MATLAB.

For various reasons, you might not have 36 workers available at runtime. In this case, MATLAB will just use the largest number available.

Note: Specifiying the number of tasks for an interactive job, i.e. using

srun ... --ntasks=...

to start your interactive job will interfere with parallel computing toolbox. We recommend not specifying the number of tasks.

Examples

Here we demonstrate how to use the PCT on a single compute node on Eagle. Learn how to open a local parallel pool with some examples of how to use it for parallel computations. Because the opening of parallel pools can be extremely slow in interactive sessions, the examples here will be restricted to non-interactive (batch) job submission.

Note: Each example below will check out one "MATLAB" and one "Distrib_Computing_Toolbox" license at runtime.

Hello World Example

In this example, a parallel pool is opened and each worker identifies itself via spmd ("single program multiple data"). Create the MATLAB script helloWorld.m: 

% open the local cluster profile
  p = parcluster('local');
  
  % open the parallel pool, recording the time it takes
  tic;
  parpool(p); % open the pool
  fprintf('Opening the parallel pool took %g seconds.\n', toc)
  
  % "single program multiple data"
  spmd
    fprintf('Worker %d says Hello World!\n', labindex)
  end
  
  delete(gcp); % close the parallel pool
  exit 

To run the script on a compute node, create the file helloWorld.sb:

#!/bin/bash
#SBATCH --time=05:00
#SBATCH --nodes=1
#SBATCH --job-name=helloWorld
#SBATCH --account=<account_string>
  
# load modules
module purge
module load matlab/R2018b 
  
# define an environment variable for the MATLAB script and output
BASE_MFILE_NAME=helloWorld
MATLAB_OUTPUT=${BASE_MFILE_NAME}.out

# execute code
cd $SLURM_SUBMIT_DIR
matlab -nodisplay -r $BASE_MFILE_NAME > $MATLAB_OUTPUT

where, again, the fields in < > must be properly specified.  Finally, at the terminal prompt, submit the job to the scheduler: 

$ sbatch helloWorld.sb

The output file helloWorld.out should contain messages about the parallel pool and a "Hello World" message from each of the available workers.

Example of Speed-Up Using Parfor

MATLAB's parfor ("parallel for-loop") can be used to parallelize tasks that require no communication between workers. In this example, the aim is to solve a stiff, one-parameter system of ordinary differential equations (ODE) for different (randomly sampled) values of the parameter and to compare the compute time when using serial and parfor loops. This is a quintessential example of Monte Carlo simulation that is suitable for parfor: the solution for each value of the parameter is time-consuming to compute but can be computed independently of the other values.

First, create a MATLAB function stiffODEfun.m that defines the right-hand side of the ODE system: 

function dy = stiffODEfun(t,y,c)
  % This is a modified example from MATLAB's documentation at:
  % http://www.mathworks.com/help/matlab/ref/ode15s.html
  % The difference here is that the coefficient c is passed as an argument.
    dy = zeros(2,1);
    dy(1) = y(2);
    dy(2) = c*(1 - y(1)^2)*y(2) - y(1);
  end

Second, create a driver file stiffODE.m that samples the input parameter and solves the ODE using the ode15s function. 

%{ 
   This script samples a parameter of a stiff ODE and solves it both in
   serial and parallel (via parfor), comparing both the run times and the
   max absolute values of the computed solutions. The code -- especially the
   serial part -- will take several minutes to run on Eagle.
  %}
  
  % open the local cluster profile
  p = parcluster('local');
  
  % open the parallel pool, recording the time it takes
  time_pool = tic;
  parpool(p);
  time_pool = toc(time_pool);
  fprintf('Opening the parallel pool took %g seconds.\n', time_pool)
  
  % create vector of random coefficients on the interval [975,1050]
  nsamples = 100; % number of samples
  coef = 975 + 50*rand(nsamples,1); % randomly generated coefficients
  
  % compute solutions within serial loop
  time_ser = tic;
  y_ser = cell(nsamples,1); % cell to save the serial solutions
  for i = 1:nsamples
    if mod(i,10)==0
      fprintf('Serial for loop, i = %d\n', i);
    end
    [~,y_ser{i}] = ode15s(@(t,y) stiffODEfun(t,y,coef(i)) ,[0 10000],[2 0]);
  end
  time_ser = toc(time_ser);
  
  % compute solutions within parfor
  time_parfor = tic;
  y_par = cell(nsamples,1); % cell to save the parallel solutions
  err = zeros(nsamples,1); % vector of errors between serial and parallel solutions
  parfor i = 1:nsamples
    if mod(i,10)==0
      fprintf('Parfor loop, i = %d\n', i);
    end
    [~,y_par{i}] = ode15s(@(t,y) stiffODEfun(t,y,coef(i)) ,[0 10000],[2 0]);
    err(i) = norm(y_par{i}-y_ser{i}); % error between serial and parallel solutions
  end
  time_parfor = toc(time_parfor);
  time_par = time_parfor + time_pool;
  
  % print results
  fprintf('RESULTS\n\n')
  fprintf('Serial time : %g\n', time_ser)
  fprintf('Parfor time : %g\n', time_par)
  fprintf('Speedup : %g\n\n', time_ser/time_par)
  fprintf('Max error between serial and parallel solutions = %e\n', max(abs(err)))
  
  % close the parallel pool
  delete(gcp)
  exit

Finally, create the batch script stiffODE.sb:

#!/bin/bash
#SBATCH --time=20:00
#SBATCH --nodes=1
#SBATCH --job-name=stiffODE
#SBATCH --account=<account_string>

# load modules
module purge
module load matlab/R2018b

# define environment variables for MATLAB script and output
BASE_MFILE_NAME=stiffODE
MATLAB_OUTPUT=${BASE_MFILE_NAME}.out

# execute code
cd $SLURM_SUBMIT_DIR
matlab -nodisplay -r $BASE_MFILE_NAME > MATLAB_OUTPUT

Next, submit the job (which will take several minutes to finish on Eagle):

$ sbatch stiffODE.sb

If the code executed correctly, the end of the text file stiffODE.out should contain the times needed to compute the solutions in serial and parallel as well as the error between the serial and parallel solutions (which should be 0!). There should be a significant speed-up — how much depends on the runtime environment — for the parallelized computation.

Share