Designed and developed By DrupalNetworks.com. Drupal Networks is a division of YAS Global.

Creating a Batch API operation to parse a large CSV file

n this code snippet, I'll show how you can parse a (large) CSV file using Drupal's Batch API. The purpose of batching an operation is to avoid PHP memory limits and time outs. Before you begin, I recommend reviewing the following two articles. Be sure to review the additional batch parameters outlined in the documentation, you might need to use them.

function MYMODULE_callback_csv_import() {
  // define path to CSV file
  $csv_file_path = file_directory_path() . '/import_path/myfile.csv';
  // define a redirect path upon batch completion
  $redirect_path = 'admin/import-csv';
  // define batch array structure
  // NOTE: minimal parameters defined to simplify code
  $batch = array(
    'title' => t('Reading File'),
    'operations' => array(
      array(
        '_MYMODULE_batch_read', array($csv_file_path),
      ),
    ),
  );
  // set batch
  batch_set($batch);
  // process batch
batch_process($redirect_path);
}

Next, we'll define the batch callback function. This function will be called repeatedly until the $context['finished'] variable is set to "1".
<?php function _MYMODULE_batch_read($csv_file_path, &$context) {
  
// define batch limit
 
$batch_limit = 100;
 
// assume the batch process has not completed
 
$context['finished'] = 0;
 
// open the file for reading
 
$file_handle = fopen($csv_file_path, 'r');
 
// check if file pointer position exists in the sandbox, and jump to location in file
 
if ($context['sandbox']['file_pointer_position']) {
   
fseek($file_handle, $context['sandbox']['file_pointer_position']);
  }
 
// loop through the file and stop at batch limit
 
for ($i = 0; $i < $batch_limit; $i++) {
   
// get file line as csv
   
$csv_line = fgetcsv($file_handle);
   
// NOTE: at this point, do what ever you'd like with the CSV array data!
   
if (is_array($csv_line)) {
     
// db_query(), etc  
   
}
   
// retain current file pointer position
   
$context['sandbox']['file_pointer_position'] = ftell($file_handle);
   
// check for EOF
   
if (feof($file_handle)) {
     
// complete the batch process
     
$context['finished'] = 1;
    
// end loop
     
break;
    }
}
}
?>

Follow Us

Aliya's Tags