wszystkie opcje
bullseye  ] [  bookworm  ] [  trixie  ] [  sid  ]
[ Pakiet źródłowy: r-cran-ff  ]

Pakiet: r-cran-ff (4.0.12+ds-1)

Odnośniki dla r-cran-ff

Screenshot

Zasoby systemu Debian:

Pobieranie pakietu źródłowego r-cran-ff:

Opiekunowie:

Zasoby zewnętrzne:

Podobne pakiety:

Memory-Efficient Fast-Access Storage of Large Data

The ff package provides data structures that are stored on disk but behave (almost) as if they were in RAM by transparently mapping only a section (pagesize) in main memory - the effective virtual memory consumption per ff object. ff supports R's standard atomic data types 'double', 'logical', 'raw' and 'integer' and non-standard atomic types boolean (1 bit), quad (2 bit unsigned), nibble (4 bit unsigned), byte (1 byte signed with NAs), ubyte (1 byte unsigned), short (2 byte signed with NAs), ushort (2 byte unsigned), single (4 byte float with NAs). For example 'quad' allows efficient storage of genomic data as an 'A','T','G','C' factor. The unsigned types support 'circular' arithmetic. There is also support for close-to-atomic types 'factor', 'ordered', 'POSIXct', 'Date' and custom close-to-atomic types.

ff not only has native C-support for vectors, matrices and arrays with flexible dimorder (major column-order, major row-order and generalizations for arrays). There is also a ffdf class not unlike data.frames and import/export filters for csv files. ff objects store raw data in binary flat files in native encoding, and complement this with metadata stored in R as physical and virtual attributes. ff objects have well-defined hybrid copying semantics, which gives rise to certain performance improvements through virtualization. ff objects can be stored and reopened across R sessions. ff files can be shared by multiple ff R objects (using different data en/de-coding schemes) in the same process or from multiple R processes to exploit parallelism. A wide choice of finalizer options allows one to work with 'permanent' files as well as creating/removing 'temporary' ff files completely transparent to the user. On certain OS/Filesystem combinations, creating the ff files works without notable delay thanks to using sparse file allocation. Several access optimization techniques such as Hybrid Index Preprocessing and Virtualization are implemented to achieve good performance even with large datasets, for example virtual matrix transpose without touching a single byte on disk. Further, to reduce disk I/O, 'logicals' and non-standard data types get stored native and compact on binary flat files i.e. logicals take up exactly 2 bits to represent TRUE, FALSE and NA.

Beyond basic access functions, the ff package also provides compatibility functions that facilitate writing code for ff and ram objects and support for batch processing on ff objects (e.g. as.ram, as.ff, ffapply). ff interfaces closely with functionality from package 'bit': chunked looping, fast bit operations and coercions between different objects that can store subscript information ('bit', 'bitwhich', ff 'boolean', ri range index, hi hybrid index). This allows to work interactively with selections of large datasets and quickly modify selection criteria.

Inne pakiety związane z r-cran-ff

  • wymaga
  • poleca
  • sugeruje
  • enhances

Pobieranie r-cran-ff

Pobierz dla wszystkich dostępnych architektur
Architektura Rozmiar pakietu Rozmiar po instalacji Pliki
alpha (port nieoficjalny) 942,2 KiB1 695,0 KiB [lista plików]
amd64 950,3 KiB1 567,0 KiB [lista plików]
arm64 937,5 KiB1 568,0 KiB [lista plików]
armel 933,3 KiB1 465,0 KiB [lista plików]
armhf 935,0 KiB1 333,0 KiB [lista plików]
hppa (port nieoficjalny) 946,7 KiB1 641,0 KiB [lista plików]
i386 964,0 KiB1 648,0 KiB [lista plików]
ia64 (port nieoficjalny) 946,9 KiB2 147,0 KiB [lista plików]
m68k (port nieoficjalny) 933,8 KiB1 468,0 KiB [lista plików]
mips64el 922,3 KiB1 655,0 KiB [lista plików]
ppc64 (port nieoficjalny) 945,9 KiB1 767,0 KiB [lista plików]
ppc64el 950,4 KiB1 760,0 KiB [lista plików]
riscv64 949,5 KiB1 432,0 KiB [lista plików]
s390x 957,2 KiB1 648,0 KiB [lista plików]
sh4 (port nieoficjalny) 957,7 KiB1 500,0 KiB [lista plików]
sparc64 (port nieoficjalny) 931,8 KiB2 028,0 KiB [lista plików]
x32 (port nieoficjalny) 953,5 KiB1 553,0 KiB [lista plików]