read_csv

Contents

read_csv#

Read a CSV file into a tensor

Note

This function requires the optional MATX_ENABLE_FILEIO compile flag

template<typename TensorType>
void matx::io::read_csv(TensorType &t, const std::string fname, const std::string delimiter, bool skip_header = true)#

Read a CSV file into a tensor view. 

  Read a CSV file into a tensor view

  CSV files typically have a large number of rows with columns representing
individual features/fields. Since these columns are typically accessed together, this function will take CSV columns and lay them out as rows in the tensor view. This allows for contiguous memory accesses on the data within a field. Currently this routine will allocate twice the amount of memory that’s needed to hold the data since the cuDF function used for reading will put the data in its own format, which we have to convert ours to.

CSVs are currently read in using the Python interpreter through pybind11. This has a startup performance hit, but CSV reading is intended to be a slow-path function, so this is not a critical component to speed up. Currently 1D and 2D tensors are supported only.

Template Parameters:

TensorType – Data type of tensor

Parameters:

  • t – Tensor to read data into

  • fname – File path of .csv file

  • delimiter – Delimiter to use for CSV file

  • skip_header – Skip the header row of the CSV file, default as true.

Examples#

io::read_csv(this->Av, this->small_csv, ",");
Contents