{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "623a6fd2", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'1.33.0'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import polars as pl\n", "pl.__version__ # The book is built with Polars version 1.20.0" ] }, { "cell_type": "markdown", "id": "1aef94a4", "metadata": {}, "source": [ "# Polars Data Types and Missing Values\n", "\n", "This notebook covers the fundamental data types in Polars, including nested types like Arrays, Lists, and Structs. \n", "It also dives deep into handling missing data (`null`) and special floating point values (`NaN`), as well as data type conversions." ] }, { "cell_type": "markdown", "id": "06781dc5-86cf-49d2-b00a-cec9acf86435", "metadata": {}, "source": [ "# Polars Series, Dataframe, Lazyframe" ] }, { "cell_type": "markdown", "id": "66da48de", "metadata": {}, "source": [ "Polars provides three main data structures:\n", "- **Series**: A one-dimensional homogeneous array with a name.\n", "- **DataFrame**: A two-dimensional table with named columns of potentially different types.\n", "- **LazyFrame**: A representation of a query plan that hasn't been executed yet. It allows Polars to optimize queries before running them." ] }, { "cell_type": "code", "execution_count": 2, "id": "db705e6f", "metadata": {}, "outputs": [], "source": [ "sales_series = pl.Series([150.00,300.00,250.00])" ] }, { "cell_type": "code", "execution_count": 3, "id": "cab68e97", "metadata": {}, "outputs": [], "source": [ "sales_df = pl.DataFrame(\n", " {\n", " \"sales\":sales_series,\n", " \"customer_id\":[24,25,26]\n", " }\n", ")" ] }, { "cell_type": "code", "execution_count": 4, "id": "9b4884e5", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 2)
salescustomer_id
f64i64
150.024
300.025
250.026
" ], "text/plain": [ "shape: (3, 2)\n", "┌───────┬─────────────┐\n", "│ sales ┆ customer_id │\n", "│ --- ┆ --- │\n", "│ f64 ┆ i64 │\n", "╞═══════╪═════════════╡\n", "│ 150.0 ┆ 24 │\n", "│ 300.0 ┆ 25 │\n", "│ 250.0 ┆ 26 │\n", "└───────┴─────────────┘" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sales_df" ] }, { "cell_type": "code", "execution_count": 5, "id": "c090668a", "metadata": {}, "outputs": [], "source": [ "lazy_df = pl.scan_csv(\"data/fruit.csv\").with_columns(is_heavy = pl.col(\"weight\")>200)" ] }, { "cell_type": "code", "execution_count": 6, "id": "6f6e6d13", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "polars_query\n", "\n", "\n", "\n", "p2\n", "\n", "Csv SCAN [data/fruit.csv]\n", "π */5;\n", "\n", "\n", "\n", "p1\n", "\n", "WITH COLUMNS [[(col(\"weight\")) > (200)].alias(\"is_heavy\")]\n", "\n", "\n", "\n", "p2->p1\n", "\n", "\n", "\n", "\n", "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "lazy_df.show_graph()" ] }, { "cell_type": "code", "execution_count": 7, "id": "13f71120", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 6)
nameweightcoloris_roundoriginis_heavy
stri64strboolstrbool
"Avocado"200"green"false"South America"false
"Banana"120"yellow"false"Asia"false
"Blueberry"1"blue"false"North America"false
"Cantaloupe"2500"orange"true"Africa"true
"Cranberry"2"red"false"North America"false
"Elderberry"1"black"false"Europe"false
"Orange"130"orange"true"Asia"false
"Papaya"1000"orange"false"South America"true
"Peach"150"orange"true"Asia"false
"Watermelon"5000"green"true"Africa"true
" ], "text/plain": [ "shape: (10, 6)\n", "┌────────────┬────────┬────────┬──────────┬───────────────┬──────────┐\n", "│ name ┆ weight ┆ color ┆ is_round ┆ origin ┆ is_heavy │\n", "│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │\n", "│ str ┆ i64 ┆ str ┆ bool ┆ str ┆ bool │\n", "╞════════════╪════════╪════════╪══════════╪═══════════════╪══════════╡\n", "│ Avocado ┆ 200 ┆ green ┆ false ┆ South America ┆ false │\n", "│ Banana ┆ 120 ┆ yellow ┆ false ┆ Asia ┆ false │\n", "│ Blueberry ┆ 1 ┆ blue ┆ false ┆ North America ┆ false │\n", "│ Cantaloupe ┆ 2500 ┆ orange ┆ true ┆ Africa ┆ true │\n", "│ Cranberry ┆ 2 ┆ red ┆ false ┆ North America ┆ false │\n", "│ Elderberry ┆ 1 ┆ black ┆ false ┆ Europe ┆ false │\n", "│ Orange ┆ 130 ┆ orange ┆ true ┆ Asia ┆ false │\n", "│ Papaya ┆ 1000 ┆ orange ┆ false ┆ South America ┆ true │\n", "│ Peach ┆ 150 ┆ orange ┆ true ┆ Asia ┆ false │\n", "│ Watermelon ┆ 5000 ┆ green ┆ true ┆ Africa ┆ true │\n", "└────────────┴────────┴────────┴──────────┴───────────────┴──────────┘" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lazy_df.collect()" ] }, { "cell_type": "markdown", "id": "e195a598-2a18-4c01-934e-9cb4c1d5dafe", "metadata": {}, "source": [ "## Polars Array" ] }, { "cell_type": "markdown", "id": "a805c83a", "metadata": {}, "source": [ "The `Array` data type in Polars represents **fixed-size** lists. \n", "Unlike the `List` type, `Array` enforces that every element has the same number of items. \n", "This allows for more memory-efficient storage and execution." ] }, { "cell_type": "code", "execution_count": 8, "id": "a229fc21", "metadata": {}, "outputs": [], "source": [ "coordinates = pl.DataFrame(\n", " [\n", " pl.Series('point2d',[[1,3],[2,3]]),\n", " pl.Series('point3d',[[1,3,4],[4,5,6]]),\n", " ],\n", " schema={\n", " 'point2d':pl.Array(shape=2, inner=pl.Int64),\n", " 'point3d':pl.Array(shape=3, inner=pl.Int64),\n", " }\n", ")\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "b9709965", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (2, 2)
point2dpoint3d
array[i64, 2]array[i64, 3]
[1, 3][1, 3, 4]
[2, 3][4, 5, 6]
" ], "text/plain": [ "shape: (2, 2)\n", "┌───────────────┬───────────────┐\n", "│ point2d ┆ point3d │\n", "│ --- ┆ --- │\n", "│ array[i64, 2] ┆ array[i64, 3] │\n", "╞═══════════════╪═══════════════╡\n", "│ [1, 3] ┆ [1, 3, 4] │\n", "│ [2, 3] ┆ [4, 5, 6] │\n", "└───────────────┴───────────────┘" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "coordinates" ] }, { "cell_type": "markdown", "id": "778c496e-205b-414d-b081-369b5c929658", "metadata": {}, "source": [ "## Polars List" ] }, { "cell_type": "markdown", "id": "27457f96", "metadata": {}, "source": [ "The `List` data type allows for **variable-length** arrays within a column. \n", "This is useful for storing sequences of data, such as daily temperature readings or a list of tags." ] }, { "cell_type": "code", "execution_count": 10, "id": "1edf8dea", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (2, 2)
temperaturewind_speed
list[f64]list[i64]
[72.5, 75.0, 77.3][15, 20]
[68.0, 70.2][10, 12, … 16]
" ], "text/plain": [ "shape: (2, 2)\n", "┌────────────────────┬────────────────┐\n", "│ temperature ┆ wind_speed │\n", "│ --- ┆ --- │\n", "│ list[f64] ┆ list[i64] │\n", "╞════════════════════╪════════════════╡\n", "│ [72.5, 75.0, 77.3] ┆ [15, 20] │\n", "│ [68.0, 70.2] ┆ [10, 12, … 16] │\n", "└────────────────────┴────────────────┘" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "weather_readings = pl.DataFrame(\n", "{\n", "\"temperature\": [[72.5, 75.0, 77.3], [68.0, 70.2]],\n", "\"wind_speed\": [[15, 20], [10, 12, 14, 16]],\n", "}\n", ")\n", "weather_readings" ] }, { "cell_type": "markdown", "id": "25dbd815-b8bd-402f-bba7-78c35224c395", "metadata": {}, "source": [ "## Polars Struct" ] }, { "cell_type": "markdown", "id": "724bca57", "metadata": {}, "source": [ "The `Struct` data type is similar to a dictionary or a row nested within a cell. \n", "It contains named fields, each with its own data type. Structs are useful for grouping related data together." ] }, { "cell_type": "code", "execution_count": 11, "id": "83a9c5c0", "metadata": {}, "outputs": [], "source": [ "rating_series = pl.Series(\n", " \"rating\",[\n", " { \"Movies\":\"Cars\",\"Theater\":\"NE\",\"Avg_rating\":4.5},\n", " {\"Movies\":\"Toy Story\",\"Theater\":\"ME\",\"Avg_rating\":4.9},\n", " ],\n", ")" ] }, { "cell_type": "code", "execution_count": 12, "id": "f89a8e1b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (2,)
rating
struct[3]
{"Cars","NE",4.5}
{"Toy Story","ME",4.9}
" ], "text/plain": [ "shape: (2,)\n", "Series: 'rating' [struct[3]]\n", "[\n", "\t{\"Cars\",\"NE\",4.5}\n", "\t{\"Toy Story\",\"ME\",4.9}\n", "]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "rating_series" ] }, { "cell_type": "markdown", "id": "478a20f4-cf2b-4958-b3c8-52ff605f1d0b", "metadata": {}, "source": [ "## Missing Data" ] }, { "cell_type": "markdown", "id": "78953e43", "metadata": {}, "source": [ "Missing data in Polars is represented by `null`. This is distinct from `NaN` (Not a Number). \n", "Polars provides extensive functionality to handle nulls, including filling them with specific values, strategies, or expressions." ] }, { "cell_type": "markdown", "id": "40e4b2f9-82d9-40b7-a1de-1b078b68b3ff", "metadata": {}, "source": [ "### Missing single value, strategy and expresession" ] }, { "cell_type": "code", "execution_count": 13, "id": "eb5e4a39", "metadata": {}, "outputs": [], "source": [ "missing_df = pl.DataFrame(\n", "{\n", "\"value\": [None, 2, 3, 4, None, None, 7, 8, 9, None],\n", "},\n", ")" ] }, { "cell_type": "code", "execution_count": 16, "id": "949b2d0d-e19f-49fe-8926-07ce78c34a2a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 1)
value
i64
null
2
3
4
null
null
7
8
9
null
" ], "text/plain": [ "shape: (10, 1)\n", "┌───────┐\n", "│ value │\n", "│ --- │\n", "│ i64 │\n", "╞═══════╡\n", "│ null │\n", "│ 2 │\n", "│ 3 │\n", "│ 4 │\n", "│ null │\n", "│ null │\n", "│ 7 │\n", "│ 8 │\n", "│ 9 │\n", "│ null │\n", "└───────┘" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df" ] }, { "cell_type": "code", "execution_count": 15, "id": "18b9b6bf", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (1, 1)
value
u32
4
" ], "text/plain": [ "shape: (1, 1)\n", "┌───────┐\n", "│ value │\n", "│ --- │\n", "│ u32 │\n", "╞═══════╡\n", "│ 4 │\n", "└───────┘" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df.null_count()" ] }, { "cell_type": "code", "execution_count": 20, "id": "13bab1fb", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 2)
valuesingle_value_fill
i64i64
null-1
22
33
44
null-1
null-1
77
88
99
null-1
" ], "text/plain": [ "shape: (10, 2)\n", "┌───────┬───────────────────┐\n", "│ value ┆ single_value_fill │\n", "│ --- ┆ --- │\n", "│ i64 ┆ i64 │\n", "╞═══════╪═══════════════════╡\n", "│ null ┆ -1 │\n", "│ 2 ┆ 2 │\n", "│ 3 ┆ 3 │\n", "│ 4 ┆ 4 │\n", "│ null ┆ -1 │\n", "│ null ┆ -1 │\n", "│ 7 ┆ 7 │\n", "│ 8 ┆ 8 │\n", "│ 9 ┆ 9 │\n", "│ null ┆ -1 │\n", "└───────┴───────────────────┘" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df.with_columns(single_value_fill = pl.col('value').fill_null(-1))" ] }, { "cell_type": "code", "execution_count": 21, "id": "c71e373f", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 8)
valueforwardbackwardminmaxmeanzeroone
i64i64i64i64i64i64i64i64
nullnull229501
22222222
33333333
44444444
null4729501
null4729501
77777777
88888888
99999999
null9null29501
" ], "text/plain": [ "shape: (10, 8)\n", "┌───────┬─────────┬──────────┬─────┬─────┬──────┬──────┬─────┐\n", "│ value ┆ forward ┆ backward ┆ min ┆ max ┆ mean ┆ zero ┆ one │\n", "│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │\n", "│ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │\n", "╞═══════╪═════════╪══════════╪═════╪═════╪══════╪══════╪═════╡\n", "│ null ┆ null ┆ 2 ┆ 2 ┆ 9 ┆ 5 ┆ 0 ┆ 1 │\n", "│ 2 ┆ 2 ┆ 2 ┆ 2 ┆ 2 ┆ 2 ┆ 2 ┆ 2 │\n", "│ 3 ┆ 3 ┆ 3 ┆ 3 ┆ 3 ┆ 3 ┆ 3 ┆ 3 │\n", "│ 4 ┆ 4 ┆ 4 ┆ 4 ┆ 4 ┆ 4 ┆ 4 ┆ 4 │\n", "│ null ┆ 4 ┆ 7 ┆ 2 ┆ 9 ┆ 5 ┆ 0 ┆ 1 │\n", "│ null ┆ 4 ┆ 7 ┆ 2 ┆ 9 ┆ 5 ┆ 0 ┆ 1 │\n", "│ 7 ┆ 7 ┆ 7 ┆ 7 ┆ 7 ┆ 7 ┆ 7 ┆ 7 │\n", "│ 8 ┆ 8 ┆ 8 ┆ 8 ┆ 8 ┆ 8 ┆ 8 ┆ 8 │\n", "│ 9 ┆ 9 ┆ 9 ┆ 9 ┆ 9 ┆ 9 ┆ 9 ┆ 9 │\n", "│ null ┆ 9 ┆ null ┆ 2 ┆ 9 ┆ 5 ┆ 0 ┆ 1 │\n", "└───────┴─────────┴──────────┴─────┴─────┴──────┴──────┴─────┘" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df.with_columns(\n", " forward=pl.col(\"value\").fill_null(strategy=\"forward\"),\n", " backward=pl.col(\"value\").fill_null(strategy=\"backward\"),\n", " min=pl.col(\"value\").fill_null(strategy=\"min\"),\n", " max=pl.col(\"value\").fill_null(strategy=\"max\"),\n", " mean=pl.col(\"value\").fill_null(strategy=\"mean\"),\n", " zero=pl.col(\"value\").fill_null(strategy=\"zero\"),\n", " one=pl.col(\"value\").fill_null(strategy=\"one\"),\n", ")" ] }, { "cell_type": "code", "execution_count": 22, "id": "4740ddf0-66b9-4d72-ac5f-2ed2145a525f", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 2)
valueexpression_mean
i64f64
null5.5
22.0
33.0
44.0
null5.5
null5.5
77.0
88.0
99.0
null5.5
" ], "text/plain": [ "shape: (10, 2)\n", "┌───────┬─────────────────┐\n", "│ value ┆ expression_mean │\n", "│ --- ┆ --- │\n", "│ i64 ┆ f64 │\n", "╞═══════╪═════════════════╡\n", "│ null ┆ 5.5 │\n", "│ 2 ┆ 2.0 │\n", "│ 3 ┆ 3.0 │\n", "│ 4 ┆ 4.0 │\n", "│ null ┆ 5.5 │\n", "│ null ┆ 5.5 │\n", "│ 7 ┆ 7.0 │\n", "│ 8 ┆ 8.0 │\n", "│ 9 ┆ 9.0 │\n", "│ null ┆ 5.5 │\n", "└───────┴─────────────────┘" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df.with_columns(\n", "expression_mean=pl.col(\"value\").fill_null(pl.col(\"value\").mean())\n", ")" ] }, { "cell_type": "code", "execution_count": 23, "id": "49fcbf6a-ce14-4e7e-a3de-2e897da3ab5a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (10, 1)
value
f64
null
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
null
" ], "text/plain": [ "shape: (10, 1)\n", "┌───────┐\n", "│ value │\n", "│ --- │\n", "│ f64 │\n", "╞═══════╡\n", "│ null │\n", "│ 2.0 │\n", "│ 3.0 │\n", "│ 4.0 │\n", "│ 5.0 │\n", "│ 6.0 │\n", "│ 7.0 │\n", "│ 8.0 │\n", "│ 9.0 │\n", "│ null │\n", "└───────┘" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "missing_df.interpolate()" ] }, { "cell_type": "markdown", "id": "c9aa5d2b-3a8e-4192-b7e9-50512d95d429", "metadata": {}, "source": [ "# NULL vs Not a Number (NaN)" ] }, { "cell_type": "markdown", "id": "a7414b43", "metadata": {}, "source": [ "It is crucial to distinguish between `null` and `NaN`:\n", "- **`null`**: Represents missing data. It applies to all data types.\n", "- **`NaN` (Not a Number)**: A special floating-point value representing undefined results (e.g., 0/0). It only applies to floating-point columns.\n", "\n", "Polars handles them differently. `null` is ignored in aggregations (like mean), while `NaN` propagates (NaN + 1 = NaN)." ] }, { "cell_type": "code", "execution_count": null, "id": "1811f375", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "nan_df = pl.DataFrame({\n", " \"value\": [1.0, np.nan, None, 4.0]\n", "})\n", "print(nan_df)" ] }, { "cell_type": "markdown", "id": "c1f870aa", "metadata": {}, "source": [ "You can check for these values using `is_nan()` and `is_null()`." ] }, { "cell_type": "code", "execution_count": null, "id": "891e6717", "metadata": {}, "outputs": [], "source": [ "nan_df.with_columns(\n", " is_nan = pl.col(\"value\").is_nan(),\n", " is_null = pl.col(\"value\").is_null()\n", ")" ] }, { "cell_type": "markdown", "id": "af7eba83", "metadata": {}, "source": [ "To handle `NaN` values, you can use `fill_nan()`. Note that `fill_null()` does NOT affect `NaN` values." ] }, { "cell_type": "code", "execution_count": null, "id": "df0a297f", "metadata": {}, "outputs": [], "source": [ "nan_df.with_columns(\n", " filled_nan = pl.col(\"value\").fill_nan(0.0),\n", " filled_null = pl.col(\"value\").fill_null(0.0)\n", ")" ] }, { "cell_type": "markdown", "id": "5077f6a4-6944-48a6-93b7-3fc82bcd322b", "metadata": {}, "source": [ "# Data Type Conversion" ] }, { "cell_type": "markdown", "id": "7ffbe84b", "metadata": {}, "source": [ "Changing data types (casting) is a common operation. Polars uses the `.cast()` method.\n", "By default, casting is **strict**. If a value cannot be converted (e.g., casting \"abc\" to Integer), Polars will raise an error." ] }, { "cell_type": "code", "execution_count": 24, "id": "5e7f42b9-d4ba-44a8-9099-f6134e1d9d69", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "shape: (3, 1)\n", "┌───────┐\n", "│ id │\n", "│ --- │\n", "│ str │\n", "╞═══════╡\n", "│ 10000 │\n", "│ 20000 │\n", "│ 30000 │\n", "└───────┘\n", "Estimated size: 15 bytes\n" ] } ], "source": [ "string_df = pl.DataFrame({\"id\": [\"10000\", \"20000\", \"30000\"]})\n", "print(string_df)\n", "print(f\"Estimated size: {string_df.estimated_size('b')} bytes\")" ] }, { "cell_type": "code", "execution_count": 27, "id": "ac16083f-9af7-4350-b681-5f22bd3f2759", "metadata": {}, "outputs": [], "source": [ "string_df_int = string_df.select(pl.col(\"id\").cast(pl.UInt16))" ] }, { "cell_type": "code", "execution_count": 28, "id": "da0a1402-26b3-41dd-b3f8-e9fd6cf800e0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimated size: 6 bytes\n" ] } ], "source": [ "print(f\"Estimated size: {string_df_int.estimated_size('b')} bytes\")" ] }, { "cell_type": "code", "execution_count": 29, "id": "4dcfab05-85e1-446e-a173-6b6c74b15c61", "metadata": {}, "outputs": [], "source": [ "data_types_df = pl.DataFrame(\n", "{\n", "\"id\": [10000, 20000, 30000],\n", "\"value\": [1.0, 2.0, 3.0],\n", "\"value2\": [\"1\", \"2\", \"3\"],\n", "}\n", ")" ] }, { "cell_type": "code", "execution_count": 30, "id": "2aec813a-31a9-4072-be5c-9ce9c6129200", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 3)
idvaluevalue2
i64f64str
100001.0"1"
200002.0"2"
300003.0"3"
" ], "text/plain": [ "shape: (3, 3)\n", "┌───────┬───────┬────────┐\n", "│ id ┆ value ┆ value2 │\n", "│ --- ┆ --- ┆ --- │\n", "│ i64 ┆ f64 ┆ str │\n", "╞═══════╪═══════╪════════╡\n", "│ 10000 ┆ 1.0 ┆ 1 │\n", "│ 20000 ┆ 2.0 ┆ 2 │\n", "│ 30000 ┆ 3.0 ┆ 3 │\n", "└───────┴───────┴────────┘" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_types_df" ] }, { "cell_type": "code", "execution_count": 31, "id": "2028233a-9294-47bb-b712-29f44fc93765", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 3)
idvaluevalue2
u16u16u16
1000011
2000022
3000033
" ], "text/plain": [ "shape: (3, 3)\n", "┌───────┬───────┬────────┐\n", "│ id ┆ value ┆ value2 │\n", "│ --- ┆ --- ┆ --- │\n", "│ u16 ┆ u16 ┆ u16 │\n", "╞═══════╪═══════╪════════╡\n", "│ 10000 ┆ 1 ┆ 1 │\n", "│ 20000 ┆ 2 ┆ 2 │\n", "│ 30000 ┆ 3 ┆ 3 │\n", "└───────┴───────┴────────┘" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_types_df.cast(pl.UInt16)" ] }, { "cell_type": "code", "execution_count": 32, "id": "2987c280", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 3)
idvaluevalue2
u16f32u8
100001.01
200002.02
300003.03
" ], "text/plain": [ "shape: (3, 3)\n", "┌───────┬───────┬────────┐\n", "│ id ┆ value ┆ value2 │\n", "│ --- ┆ --- ┆ --- │\n", "│ u16 ┆ f32 ┆ u8 │\n", "╞═══════╪═══════╪════════╡\n", "│ 10000 ┆ 1.0 ┆ 1 │\n", "│ 20000 ┆ 2.0 ┆ 2 │\n", "│ 30000 ┆ 3.0 ┆ 3 │\n", "└───────┴───────┴────────┘" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_types_df.cast({\"id\": pl.UInt16, \"value\": pl.Float32, \"value2\": pl.UInt8})" ] }, { "cell_type": "code", "execution_count": 33, "id": "cc26f3be", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 3)
idvaluevalue2
i64f32u8
100001.01
200002.02
300003.03
" ], "text/plain": [ "shape: (3, 3)\n", "┌───────┬───────┬────────┐\n", "│ id ┆ value ┆ value2 │\n", "│ --- ┆ --- ┆ --- │\n", "│ i64 ┆ f32 ┆ u8 │\n", "╞═══════╪═══════╪════════╡\n", "│ 10000 ┆ 1.0 ┆ 1 │\n", "│ 20000 ┆ 2.0 ┆ 2 │\n", "│ 30000 ┆ 3.0 ┆ 3 │\n", "└───────┴───────┴────────┘" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_types_df.cast({pl.Float64: pl.Float32, pl.String: pl.UInt8})" ] }, { "cell_type": "code", "execution_count": 34, "id": "3093d75b", "metadata": {}, "outputs": [], "source": [ "import polars.selectors as cs" ] }, { "cell_type": "code", "execution_count": 35, "id": "3b764020", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "shape: (3, 3)
idvaluevalue2
u16u16str
100001"1"
200002"2"
300003"3"
" ], "text/plain": [ "shape: (3, 3)\n", "┌───────┬───────┬────────┐\n", "│ id ┆ value ┆ value2 │\n", "│ --- ┆ --- ┆ --- │\n", "│ u16 ┆ u16 ┆ str │\n", "╞═══════╪═══════╪════════╡\n", "│ 10000 ┆ 1 ┆ 1 │\n", "│ 20000 ┆ 2 ┆ 2 │\n", "│ 30000 ┆ 3 ┆ 3 │\n", "└───────┴───────┴────────┘" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_types_df.cast({cs.numeric():pl.UInt16})" ] }, { "cell_type": "code", "execution_count": null, "id": "6a1b5f90", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "3be7332c", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "86d925a8", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "34f0e09a", "metadata": {}, "source": [ "### Strict vs Non-Strict Casting\n", "You can disable strict mode to convert failing casts into `null` values instead of raising an error." ] }, { "cell_type": "code", "execution_count": null, "id": "473c888a", "metadata": {}, "outputs": [], "source": [ "strict_df = pl.DataFrame({\"val\": [\"1\", \"2\", \"a\"]})\n", "\n", "# This would raise an error:\n", "# strict_df.select(pl.col(\"val\").cast(pl.Int64))\n", "\n", "# Non-strict casting replaces errors with null:\n", "strict_df.select(pl.col(\"val\").cast(pl.Int64, strict=False))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.2" } }, "nbformat": 4, "nbformat_minor": 5 }