licenses.dev

rust_tokenizers v3.0.1

High performance tokenizers for Rust

Published on

Licenses

Learn more about license information.

Package Licenses

Apache-2.0

Dependency Licenses

Apache-2.0 OR MIT44
PackageRelationDependenciesPublished
bitflags • 1.3.2A macro to generate structures which behave like bitflags. INDIRECT0
bitflags • 2.6.0A macro to generate structures which behave like bitflags. INDIRECT0
cfg-if • 1.0.0A macro to ergonomically define an item depending on a large number of #[cfg] parameters. Structured like an if-else chain, the first matching branch is the item that gets emitted. INDIRECT0
crossbeam-deque • 0.8.5Concurrent work-stealing dequeINDIRECT2
crossbeam-epoch • 0.9.18Epoch-based garbage collectionINDIRECT1
crossbeam-utils • 0.8.20Utilities for concurrent programmingINDIRECT0
either • 1.13.0The enum `Either` with variants `Left` and `Right` is a general purpose sum type with two cases. INDIRECT0
errno • 0.3.9Cross-platform interface to the `errno` variable.INDIRECT11
fastrand • 2.1.1A simple and fast random number generatorINDIRECT0
itertools • 0.8.2Extra iterator adaptors, iterator methods, free functions, and macros.DIRECT1
itoa • 1.0.11Fast integer primitive to string conversionINDIRECT0
lazy_static • 1.5.0A macro for declaring lazily evaluated statics in Rust.DIRECT0
libc • 0.2.161Raw FFI bindings to platform libraries like libc. INDIRECT0
proc-macro2 • 1.0.89A substitute implementation of the compiler's `proc_macro` API to decouple token-based libraries from the procedural macro use case.INDIRECT1
quote • 1.0.37Quasi-quoting macro quote!(...)INDIRECT2
rayon-core • 1.12.1Core APIs for RayonINDIRECT3
rayon • 1.10.0Simple work-stealing parallelism for RustDIRECT5
regex-syntax • 0.6.29A regular expression parser.INDIRECT0
regex • 1.7.3An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs. DIRECT3
serde_derive • 1.0.210Macros 1.1 implementation of #[derive(Serialize, Deserialize)]INDIRECT4
serde_json • 1.0.132A JSON serialization file formatDIRECT9
serde • 1.0.210A generic serialization/deserialization frameworkDIRECT5
syn • 2.0.56Parser for Rust source codeINDIRECT3
tempfile • 3.8.0A library for managing temporary files and directories.DIRECT28
unicode-normalization • 0.1.24This crate provides functions for normalization of Unicode strings, including Canonical and Compatible Decomposition and Recomposition, as described in Unicode Standard Annex #15. DIRECT2
windows_aarch64_gnullvm • 0.48.5Import lib for WindowsINDIRECT0
windows_aarch64_gnullvm • 0.52.6Import lib for WindowsINDIRECT0
windows_aarch64_msvc • 0.48.5Import lib for WindowsINDIRECT0
windows_aarch64_msvc • 0.52.6Import lib for WindowsINDIRECT0
windows_i686_gnu • 0.48.5Import lib for WindowsINDIRECT0
windows_i686_gnu • 0.52.6Import lib for WindowsINDIRECT0
windows_i686_gnullvm • 0.52.6Import lib for WindowsINDIRECT0
windows_i686_msvc • 0.48.5Import lib for WindowsINDIRECT0
windows_i686_msvc • 0.52.6Import lib for WindowsINDIRECT0
windows_x86_64_gnu • 0.48.5Import lib for WindowsINDIRECT0
windows_x86_64_gnu • 0.52.6Import lib for WindowsINDIRECT0
windows_x86_64_gnullvm • 0.48.5Import lib for WindowsINDIRECT0
windows_x86_64_gnullvm • 0.52.6Import lib for WindowsINDIRECT0
windows_x86_64_msvc • 0.48.5Import lib for WindowsINDIRECT0
windows_x86_64_msvc • 0.52.6Import lib for WindowsINDIRECT0
windows-sys • 0.48.0Rust for WindowsINDIRECT8
windows-sys • 0.52.0Rust for WindowsINDIRECT9
windows-targets • 0.48.5Import libs for WindowsINDIRECT7
windows-targets • 0.52.6Import libs for WindowsINDIRECT8
MIT OR Unlicense4
PackageRelationDependenciesPublished
aho-corasick • 0.7.20Fast multiple substring searching.INDIRECT1
csv-core • 0.1.11Bare bones CSV parsing with no_std support.INDIRECT1
csv • 1.3.0Fast CSV parsing with support for serde.DIRECT10
memchr • 2.7.4Provides extremely fast (uses SIMD on x86_64, aarch64 and wasm32) routines for 1, 2 or 3 byte search and single substring search. INDIRECT0
Apache-2.0 OR MIT OR Apache-2.0 WITH LLVM-exception2
PackageRelationDependenciesPublished
linux-raw-sys • 0.4.14Generated bindings for Linux's userspace APIINDIRECT0
rustix • 0.38.10Safe Rust bindings to POSIX/Unix/Linux/Winsock-like syscallsINDIRECT23
Apache-2.0 OR MIT OR Zlib2
PackageRelationDependenciesPublished
tinyvec_macros • 0.1.1Some macros for tiny containersINDIRECT0
tinyvec • 1.8.0`tinyvec` provides 100% safe vec-like data structures.INDIRECT1
MIT1
PackageRelationDependenciesPublished
redox_syscall • 0.3.5A Rust library to access raw Redox system callsINDIRECT1
Apache-2.0 OR BSL-1.01
PackageRelationDependenciesPublished
ryu • 1.0.18Fast floating point to string conversionINDIRECT0
Unicode-DFS-2016 AND (Apache-2.0 OR MIT)1
PackageRelationDependenciesPublished
unicode-ident • 1.0.13Determine whether characters have the XID_Start or XID_Continue properties according to Unicode Standard Annex #31INDIRECT0
Last updated on

Copy the licenses badge code for the latest version:

[![licenses](https://licenses.dev/b/cargo/rust_tokenizers)](https://licenses.dev/cargo/rust_tokenizers)
<a href="https://licenses.dev/cargo/rust_tokenizers">
  <img src="https://licenses.dev/b/cargo/rust_tokenizers" alt="licenses" />
</a>

Or the licenses badge code for the 3.0.1 version:

[![licenses](https://licenses.dev/b/cargo/rust_tokenizers/3.0.1)](https://licenses.dev/cargo/rust_tokenizers/3.0.1)
<a href="https://licenses.dev/cargo/rust_tokenizers/3.0.1">
  <img src="https://licenses.dev/b/cargo/rust_tokenizers/3.0.1" alt="licenses" />
</a>
GitHub