Package: r-cran-tokenizers (0.3.0-1 and others)

[ Source: r-cran-tokenizers  ]

GNU R fast, consistent tokenization of natural language text

Convert natural language text into tokens. Includes tokenizers for shingled n-grams, skip n-grams, words, word stems, sentences, paragraphs, characters, shingled characters, lines, tweets, Penn Treebank, regular expressions, as well as functions for counting characters, words, and sentences, and a function for splitting longer texts into separate documents, each with the same number of words. The tokenizers have a consistent interface, and the package is built on the 'stringi' and 'Rcpp' packages for fast yet correct tokenization in 'UTF-8'.

Other Packages Related to r-cran-tokenizers

• dep: libc6 (>= 2.14) [amd64]

  GNU C Library: Shared libraries
  also a virtual package provided by libc6-udeb

  dep: libc6 (>= 2.16) [x32]

  dep: libc6 (>= 2.17) [arm64, ppc64el]

  dep: libc6 (>= 2.2) [hppa]

  dep: libc6 (>= 2.27) [riscv64]

  dep: libc6 (>= 2.36) [sh4]

  dep: libc6 (>= 2.4) [armel, armhf, i386, m68k, mips64el, ppc64, s390x, sparc64]

• dep: libc6.1 (>= 2.1.3) [alpha]

  GNU C Library: Shared libraries
  also a virtual package provided by libc6.1-udeb

  dep: libc6.1 (>= 2.36) [ia64]

• dep: libgcc-s1 (>= 3.0) [not armel, armhf, hppa, ia64, m68k]

  GCC support library

  dep: libgcc-s1 (>= 3.5) [armel, armhf]

• dep: libgcc-s2 (>= 4.2.1) [m68k]

  GCC support library

• dep: libgcc-s4 (>= 4.1.1) [hppa]

  GCC support library

• dep: libstdc++6 (>= 11) [not riscv64]

  GNU Standard C++ Library v3

  dep: libstdc++6 (>= 13.1) [riscv64]

• dep: libunwind8 [ia64]

  library to determine the call-chain of a program - runtime

• dep: r-api-4.0

  virtual package provided by r-base-core

• dep: r-base-core (>= 4.2.2.20221110-1) [not riscv64]

  GNU R core of statistical computation and graphics system

• dep: r-cran-rcpp (>= 0.12.3)

  GNU R package for Seamless R and C++ Integration

• dep: r-cran-snowballc (>= 0.5.1)

  Snowball stemmers based on the C libstemmer UTF-8 library

• dep: r-cran-stringi (>= 1.0.1)

  GNU R character string processing facilities

• rec: r-cran-testthat

  GNU R testsuite

• sug: r-cran-covr

  test coverage for GNU R packages

• sug: r-cran-knitr

  GNU R package for dynamic report generation using Literate Programming

• sug: r-cran-rmarkdown

  convert R markdown documents into a variety of formats

Download r-cran-tokenizers

Download for all available architectures

Architecture	Version	Package Size	Installed Size	Files
alpha (unofficial port)	0.3.0-1	641.0 kB	925.0 kB	[list of files]
amd64	0.3.0-1	641.6 kB	845.0 kB	[list of files]
arm64	0.3.0-1	636.3 kB	861.0 kB	[list of files]
armel	0.3.0-1	636.2 kB	860.0 kB	[list of files]
armhf	0.3.0-1	637.6 kB	860.0 kB	[list of files]
hppa (unofficial port)	0.3.0-1	643.2 kB	858.0 kB	[list of files]
i386	0.3.0-1	642.9 kB	847.0 kB	[list of files]
ia64 (unofficial port)	0.3.0-1	650.1 kB	950.0 kB	[list of files]
m68k (unofficial port)	0.3.0-1	639.4 kB	836.0 kB	[list of files]
mips64el	0.3.0-1	635.5 kB	869.0 kB	[list of files]
ppc64 (unofficial port)	0.3.0-1	642.4 kB	927.0 kB	[list of files]
ppc64el	0.3.0-1	642.1 kB	925.0 kB	[list of files]
riscv64	0.3.0-1+b1	640.2 kB	821.0 kB	[list of files]
s390x	0.3.0-1	637.7 kB	849.0 kB	[list of files]
sh4 (unofficial port)	0.3.0-1	647.6 kB	860.0 kB	[list of files]
sparc64 (unofficial port)	0.3.0-1	634.7 kB	1,761.0 kB	[list of files]
x32 (unofficial port)	0.3.0-1	642.2 kB	836.0 kB	[list of files]