Package org.apache.arrow.datafusion

Interface DataFrame

public interface DataFrame
A dataframe is a rectangle shaped data that holds columns and rows, and can be collected into batches and read via reader.

  • Method Details

    • collect

      CompletableFuture<org.apache.arrow.vector.ipc.ArrowReader> collect(org.apache.arrow.memory.BufferAllocator allocator)
      Collect dataframe into a list of record batches
      Parameters:
      allocator - buffer allocator to allocate vectors within Reader
      Returns:
      reader instance to extract the data, you are expected to ArrowReader.close() it after usage to release memory

    • executeStream

      CompletableFuture<RecordBatchStream> executeStream(org.apache.arrow.memory.BufferAllocator allocator)
      Execute this DataFrame and return a stream of the result data
      Parameters:
      allocator - buffer allocator to allocate vectors for the stream
      Returns:
      Stream of results

    • show

      CompletableFuture<Void> show()
      Print results.
      Returns:
      null

    • writeParquet

      CompletableFuture<Void> writeParquet(Path path)
      Write results to a parquet file.
      Parameters:
      path - path to write parquet file to
      Returns:
      null

    • writeCsv

      CompletableFuture<Void> writeCsv(Path path)
      Write results to a csv file.
      Parameters:
      path - path to write csv file to
      Returns:
      null

    • intoView

      TableProvider intoView()
      Converts this DataFrame into a TableProvider that can be registered as a table view using SessionContext.registerParquet(String, Path)
      Returns:
      the table provider ready to be e.g. registered.

    • getPointer

      long getPointer()
      Get a pointer to the native object
      Returns:
      Pointer value as a long