Why is Polars returning unexpected results with unique() on lists?
Asked by
CuriousCoder42
I'm having an issue with Polars where the `unique()` function doesn't seem to work as expected. For example, when I apply it to a column with lists, the result for the first row, which contains only `null`, should be `null`, but Polars returns `[false, true, null]`. Here's some output from my code:
```
|list_col|unique(list_col)|
|:-|:-|
|[null]|[false, true, null]|
|[null, null, … true]|[true, null]|
```
I'm using Polars version 1.29.0, and I've already submitted a bug report on GitHub. Here's the code I'm using to reproduce this issue:
```python
import polars as pl # type: ignore
print("polars version: ", pl.__version__)
(
pl.DataFrame(
{"list_col": [[None], [None, None, None, True, None, None, None, True, True]]}
).with_columns(pl.col("list_col").list.unique().alias("unique(list_col)"))
)
```
Can someone explain why this is happening?
Answers
You can actually get around this by using `.list.eval()` until they fix it. Here’s an example of how to do that:
```python
import polars as pl
print("polars version: ", pl.__version__)
(
pl.DataFrame(
{"list_col": [[None], [None, None, None, True, None, None, None, True, True]]}
).with_columns(pl.col("list_col").list.eval(pl.element().unique()))
)
```
This should give you the expected result!
Reply to this answer
Add your answer
Answers are reviewed before publication.
CodeSleuth: I see where you're coming from! But I don't think the original code is necessarily broken; it might just be a case of needing to access list elements properly in Polars. Using `eval` helps with that, and it’s good to keep Polars syntax in mind!