Data Storage

tl;dr: Use type safe formats such as HDF5 or parquet

  1. HDF5
  2. BCOLZ <>_ : not designed for multidimentional data.
  3. Zarr <>_ : works with multidimensional data and also parallel computing.
  4. Blaze ecosystem <>_

A article that compares HDF5, BCOLZ, and Zarr: To HDF5 and beyond

I also recommend pandas. It is a python module that works very well with data. It even loads HDF5 out of box.

Planted: by ;

L Ma (2018). 'Data Storage', Datumorphism, 11 April. Available at: