rpolars.github.io
Open in
urlscan Pro
2606:50c0:8000::153
Public Scan
Submitted URL: http://rpolars.github.io/
Effective URL: https://rpolars.github.io/
Submission: On April 19 via manual from US — Scanned from DE
Effective URL: https://rpolars.github.io/
Submission: On April 19 via manual from US — Scanned from DE
Form analysis 1 forms found in the DOM
<form class="form-inline my-2 my-lg-0" role="search">
<span class="algolia-autocomplete" style="position: relative; display: inline-block; direction: ltr;"><input type="search" class="form-control me-sm-2 aa-input" aria-label="Toggle navigation" name="search-input" data-search-index="search.json"
id="search-input" placeholder="Search for" autocomplete="off" spellcheck="false" role="combobox" aria-autocomplete="list" aria-expanded="false" aria-owns="algolia-autocomplete-listbox-0" dir="auto"
style="position: relative; vertical-align: top;">
<pre aria-hidden="true"
style="position: absolute; visibility: hidden; white-space: pre; font-family: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", "Noto Sans", "Liberation Sans", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; word-spacing: 0px; letter-spacing: 0px; text-indent: 0px; text-rendering: auto; text-transform: none;"></pre>
<span class="aa-dropdown-menu" role="listbox" id="algolia-autocomplete-listbox-0" style="position: absolute; top: 100%; left: 0px; z-index: 100; display: none; right: auto;">
<div class="aa-dataset-content"></div>
</span>
</span>
</form>Text Content
Skip to contents
polars 0.5.0
* Get started
* Reference
* Changelog
*
POLARS
The goal of this project is to bring the blazingly fast Polars data manipulation
library to R. The underlying computation engine is written in Rust and this R
implementation has no other dependencies than R itself (≥ 4.1.0).
Documentation can be found on the r-polars homepage.
The primary developer of the upstream Polars project is Ritchie Vink
(@ritchie46). This R port is maintained by Søren Welling (@sorhawell), together
with other contributors. Consider joining our Discord (subchannel) for
additional help and discussion.
Update: As of March 2023, polars has now reached nearly 100% coverage of the
underlying “lazy” Expr syntax. While translation of the “eager” syntax is still
a little further behind, you should be able to do just about everything using
$select() + $with_columns().
INSTALL
The package is not yet available on CRAN. But we provide convenient installation
options for a variety of operating systems:
R-UNIVERSE
R-universe provides pre-compiled polars binaries for Windows and MacOS (x86_64),
with source builds for other platforms. Please see the GitHub release option
below for binary install options on Linux.
install.packages("polars", repos = "https://rpolars.r-universe.dev")
Special thanks to Jeroen Ooms (@jeroen) for the excellent R-universe support.
GITHUB RELEASES
We also provide pre-compiled binaries for various operating systems, as well as
source installs, on our GitHub releases page. You can download and install these
files manually, or install directly from R. Simply match the URL for your
operating system and the desired release. For example, to install the latest
release of polars on Linux (x86_64) one would use:
install.packages(
"https://github.com/pola-rs/r-polars/releases/latest/download/polars__x86_64-pc-linux-gnu.gz",
repos = NULL
)
Similarly for Windows (URL and MacOS (x86_64, URL). Just remember to invoke the
repos = NULL argument if you are installing these binary builds directly from
within R.
One exception worth noting is MacOS (arm64), i.e. systems based on the new M1/M2
“Silicon” chips. To install polars on one of these machines, we need to build
the package from source and this requires Xcode (xcode-select --install). Once
Xcode is installed, you can run the below code chunk to build polars. The
corresponding Makevars script will download a ~200MB cross-compiled object file,
while your machine links and builds the final R package.
# install.packages("remotes")
remotes::install_github(
"https://github.com/pola-rs/r-polars",
ref = "long_arms64", force =TRUE
)
Please file an issue if you require a different target or operating system
build. Finally, see the bottom of this README for details on how to install rust
to build from source (only relevant for developers, or users of unsupported
operating systems).
QUICKSTART EXAMPLE
The introductory vignette (vignette("polars")) contains a series of detailed
examples. But here is a quick illustration of polars in action.
Start by loading the package and creating a Polars DataFrame object. Similar to
the Python implementation, note that we use the pl$ prefix to specify a Polars
constructor.
library(polars)
dat = pl$DataFrame(mtcars)
dat
#> polars DataFrame: shape: (32, 11)
#> ┌──────┬─────┬───────┬───────┬─────┬─────┬─────┬──────┬──────┐
#> │ mpg ┆ cyl ┆ disp ┆ hp ┆ ... ┆ vs ┆ am ┆ gear ┆ carb │
#> │ --- ┆ --- ┆ --- ┆ --- ┆ ┆ --- ┆ --- ┆ --- ┆ --- │
#> │ f64 ┆ f64 ┆ f64 ┆ f64 ┆ ┆ f64 ┆ f64 ┆ f64 ┆ f64 │
#> ╞══════╪═════╪═══════╪═══════╪═════╪═════╪═════╪══════╪══════╡
#> │ 21.0 ┆ 6.0 ┆ 160.0 ┆ 110.0 ┆ ... ┆ 0.0 ┆ 1.0 ┆ 4.0 ┆ 4.0 │
#> │ 21.0 ┆ 6.0 ┆ 160.0 ┆ 110.0 ┆ ... ┆ 0.0 ┆ 1.0 ┆ 4.0 ┆ 4.0 │
#> │ 22.8 ┆ 4.0 ┆ 108.0 ┆ 93.0 ┆ ... ┆ 1.0 ┆ 1.0 ┆ 4.0 ┆ 1.0 │
#> │ 21.4 ┆ 6.0 ┆ 258.0 ┆ 110.0 ┆ ... ┆ 1.0 ┆ 0.0 ┆ 3.0 ┆ 1.0 │
#> │ ... ┆ ... ┆ ... ┆ ... ┆ ... ┆ ... ┆ ... ┆ ... ┆ ... │
#> │ 15.8 ┆ 8.0 ┆ 351.0 ┆ 264.0 ┆ ... ┆ 0.0 ┆ 1.0 ┆ 5.0 ┆ 4.0 │
#> │ 19.7 ┆ 6.0 ┆ 145.0 ┆ 175.0 ┆ ... ┆ 0.0 ┆ 1.0 ┆ 5.0 ┆ 6.0 │
#> │ 15.0 ┆ 8.0 ┆ 301.0 ┆ 335.0 ┆ ... ┆ 0.0 ┆ 1.0 ┆ 5.0 ┆ 8.0 │
#> │ 21.4 ┆ 4.0 ┆ 121.0 ┆ 109.0 ┆ ... ┆ 1.0 ┆ 1.0 ┆ 4.0 ┆ 2.0 │
#> └──────┴─────┴───────┴───────┴─────┴─────┴─────┴──────┴──────┘
Once our Polars DataFrame has been created, we can chain together a series of
data manipulations as part of the same query. For example:
dat$filter(
pl$col("cyl")>=6
)$groupby(
"cyl", "am"
)$agg(
pl$col("mpg")$mean()$alias("mean_mpg"),
pl$col("hp")$median()$alias("med_hp")
)
#> polars DataFrame: shape: (4, 4)
#> ┌─────┬─────┬───────────┬────────┐
#> │ cyl ┆ am ┆ mean_mpg ┆ med_hp │
#> │ --- ┆ --- ┆ --- ┆ --- │
#> │ f64 ┆ f64 ┆ f64 ┆ f64 │
#> ╞═════╪═════╪═══════════╪════════╡
#> │ 6.0 ┆ 1.0 ┆ 20.566667 ┆ 110.0 │
#> │ 6.0 ┆ 0.0 ┆ 19.125 ┆ 116.5 │
#> │ 8.0 ┆ 0.0 ┆ 15.05 ┆ 180.0 │
#> │ 8.0 ┆ 1.0 ┆ 15.4 ┆ 299.5 │
#> └─────┴─────┴───────────┴────────┘
The above is an example of Polars’ eager execution engine. But for maximum
performance, it is preferable to use Polars’ lazy execution mode, which allows
the package to apply additional query optimizations.
ldat = dat$lazy()
ldat$filter(
pl$col("cyl")>=6
)$groupby(
"cyl", "am"
)$agg(
pl$col("mpg")$mean()$alias("mean_mpg"),
pl$col("hp")$median()$alias("med_hp")
)$collect()
#> polars DataFrame: shape: (4, 4)
#> ┌─────┬─────┬───────────┬────────┐
#> │ cyl ┆ am ┆ mean_mpg ┆ med_hp │
#> │ --- ┆ --- ┆ --- ┆ --- │
#> │ f64 ┆ f64 ┆ f64 ┆ f64 │
#> ╞═════╪═════╪═══════════╪════════╡
#> │ 6.0 ┆ 1.0 ┆ 20.566667 ┆ 110.0 │
#> │ 6.0 ┆ 0.0 ┆ 19.125 ┆ 116.5 │
#> │ 8.0 ┆ 0.0 ┆ 15.05 ┆ 180.0 │
#> │ 8.0 ┆ 1.0 ┆ 15.4 ┆ 299.5 │
#> └─────┴─────┴───────────┴────────┘
CONTRIBUTE
Contributions are very welcome!
Here are the steps required for an example contribution, where we are
implementing the cosine expression:
* Look up the polars.Expr.cos method in py-polars documentation.
* Press the [source] button to see the Python impl
* Find the cos py-polars rust implementation (likely just a simple call to the
Rust-Polars API)
* Adapt the Rust part and place it here.
* Adapt the Python frontend syntax to R and place it here. Add the roxygen docs
+ examples above.
* Notice we use Expr_cos = "use_extendr_wrapper", it means we’re just using
unmodified the extendr auto-generated wrapper
* Write a test here.
* Run renv::restore() and resolve all R packages
* Run rextendr::document() to recompile and confirm the added method functions
as intended,
e.g. pl$DataFrame(a=c(0,pi/2,pi,NA_real_))$select(pl$col("a")$cos())
* Run devtools::test(). See below for how to set up your development
environment correctly.
Note that PRs to polars will be automatically be built and tested on all
platforms as part of our GitHub Actions workflow. A more detailed description of
the development environment and workflow for local builds is provided below.
DEVELOPMENT ENVIRONMENT AND WORKFLOW
SYSTEM DEPENDENCIES
Rust toolchain
* Install rustup, the cross-platform Rust installer. Then:
rustup toolchain install nightly
rustup default nightly
* Windows: Make sure the latest version of Rtools is installed and on your
PATH.
* MacOS: Make sure Xcode is installed.
* Install CMake and added it to your PATH.
DEVELOPMENT WORKFLOW
Assuming the system dependencies have been met (above), the typical polars
development workflow is as follows:
Step 1: Fork the polars repo on GitHub and then clone it locally.
git clone git@github.com:<YOUR-GITHUB-ACCOUNT>/r-polars.git
cd r-polars
Step 2: Build the package and install the suggested package dependencies.
* Option A: Using devtools.
Rscript -e 'devtools::install(pkg = ".", dependencies = TRUE)'
* Option B: Using renv.
# Rscript -e 'install.packages("renv")'
Rscript -e 'renv::activate(); renv::restore()'
Step 3: Make your proposed changes to the R and/or Rust code. Don’t forget to
run:
rextendr::document() # compile Rust code + update wrappers & docs
devtools::test() # run all unit tests
Step 4 (optional): Build the package locally.
R CMD INSTALL --no-multiarch --with-keep.source polars
Step 5: Commit your changes and submit a PR to the main polars repo.
* As aside, notice that ./renv.lock sets all R packages during the server
build.
Tip: To speed up the local R CMD check, run the following:
devtools::check(
env_vars = list(RPOLARS_RUST_SOURCE="/YOUR/OWN/ABSOLUTE/PATH/r-polars/src/rust"),
check_dir = "./check/"
)
source("./inst/misc/filter_rcmdcheck.R")
Sys.sleep(5)
unlink("check",recursive = TRUE, force =TRUE)
* The RPOLARS_RUST_SOURCE environment variable allows polars to recover the
Cargo cache even if source files have been moved. Replace with your own
absolute path to your local clone!
* filter_rcmdcheck.R removes known warnings from final check report.
* unlink("check") cleans up.
LINKS
* Browse source code
LICENSE
* Full license
* MIT + file LICENSE
CITATION
* Citing polars
DEVELOPERS
* Soren Welling
Author, maintainer
DEV STATUS
*
*
*
Developed by Soren Welling.
Site built with pkgdown 2.0.7.