Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!
Get Started for Free

FileSystem

FileSystem classes


Description

FileSystem is an abstract file system API, LocalFileSystem is an implementation accessing files on the local machine. SubTreeFileSystem is an implementation that delegates to another implementation after prepending a fixed base path

Factory

LocalFileSystem$create() returns the object and takes no arguments.

SubTreeFileSystem$create() takes the following arguments:

  • base_path, a string path

  • base_fs, a FileSystem object

S3FileSystem$create() optionally takes arguments:

  • anonymous: logical, default FALSE. If true, will not attempt to look up credentials using standard AWS configuration methods.

  • access_key, secret_key: authentication credentials. If one is provided, the other must be as well. If both are provided, they will override any AWS configuration set at the environment level.

  • session_token: optional string for authentication along with access_key and secret_key

  • role_arn: string AWS ARN of an AccessRole. If provided instead of access_key and secret_key, temporary credentials will be fetched by assuming this role.

  • session_name: optional string identifier for the assumed role session.

  • external_id: optional unique string identifier that might be required when you assume a role in another account.

  • load_frequency: integer, frequency (in seconds) with which temporary credentials from an assumed role session will be refreshed. Default is 900 (i.e. 15 minutes)

  • region: AWS region to connect to. If omitted, the AWS library will provide a sensible default based on client configuration, falling back to "us-east-1" if no other alternatives are found.

  • endpoint_override: If non-empty, override region with a connect string such as "localhost:9000". This is useful for connecting to file systems that emulate S3.

  • scheme: S3 connection transport (default "https")

  • background_writes: logical, whether OutputStream writes will be issued in the background, without blocking (default TRUE)

Methods

  • $GetFileInfo(x): x may be a FileSelector or a character vector of paths. Returns a list of FileInfo

  • $CreateDir(path, recursive = TRUE): Create a directory and subdirectories.

  • $DeleteDir(path): Delete a directory and its contents, recursively.

  • $DeleteDirContents(path): Delete a directory's contents, recursively. Like $DeleteDir(), but doesn't delete the directory itself. Passing an empty path ("") will wipe the entire filesystem tree.

  • $DeleteFile(path) : Delete a file.

  • $DeleteFiles(paths) : Delete many files. The default implementation issues individual delete operations in sequence.

  • $Move(src, dest): Move / rename a file or directory. If the destination exists: if it is a non-empty directory, an error is returned otherwise, if it has the same type as the source, it is replaced otherwise, behavior is unspecified (implementation-dependent).

  • $CopyFile(src, dest): Copy a file. If the destination exists and is a directory, an error is returned. Otherwise, it is replaced.

  • $OpenInputStream(path): Open an input stream for sequential reading.

  • $OpenInputFile(path): Open an input file for random access reading.

  • $OpenOutputStream(path): Open an output stream for sequential writing.

  • $OpenAppendStream(path): Open an output stream for appending.

Active bindings

  • $type_name: string filesystem type name, such as "local", "s3", etc.

  • $region: string AWS region, for S3FileSystem and SubTreeFileSystem containing a S3FileSystem

  • $base_fs: for SubTreeFileSystem, the FileSystem it contains

  • $base_path: for SubTreeFileSystem, the path in $base_fs which is considered root in this SubTreeFileSystem.


arrow

Integration to 'Apache' 'Arrow'

v4.0.0.1
Apache License (>= 2.0)
Authors
Neal Richardson [aut, cre], Ian Cook [aut], Jonathan Keane [aut], Romain François [aut] (<https://orcid.org/0000-0002-2444-4226>), Jeroen Ooms [aut], Javier Luraschi [ctb], Jeffrey Wong [ctb], Apache Arrow [aut, cph]
Initial release

We don't support your browser anymore

Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.