Show HN: OpenDAL, one API to access all the storages (S3, Azblob, HDFS, etc.)

[tison]

https://github.com/apache/opendal

[antman]

Does it work with seek requests for partially accessing files for backends that support it, or does it download the whole file each time?

Python similar: smart_open, universal_pathlib based on fsspec

[xuanwo → antman]

Yes, opendal supports seek.

For example:

  use std::io;
  use std::io::SeekFrom;
  use futures::io::AsyncReadExt;
  use opendal::Operator;
  use opendal::Result;
  async fn test(op: Operator) -> io::Result<()> {
      let mut r = op
          .reader("hello.txt")
          .await?
          // Only access range (0, 8*1024*1024 )
          .into_futures_async_read(0..8*1024*1024)
          .await?;

      // Seek to 1024.
      r.seek(SeekFrom::Start(1024)).await?;
      let mut bs = Vec::new();

      r.read_to_end(&mut bs).await?;
      Ok(())
  }