ENH(string dtype): Make str.decode return str dtype #60709
Conversation
rhshadrach
changed the title
| @@ -566,7 +566,7 @@ def test_string_slice_out_of_bounds(any_string_dtype): | |||
| def test_encode_decode(any_string_dtype): | |||
| ser = Series(["a", "b", "a\xe4"], dtype=any_string_dtype).str.encode("utf-8") | |||
| result = ser.str.decode("utf-8") | |||
| expected = ser.map(lambda x: x.decode("utf-8")).astype(object) | |||
| expected = Series(["a", "b", "a\xe4"], dtype="str") | |||
There was a problem hiding this comment.
The change from ser.map to using Series is just to make this test a bit more explicit. Using ser.map(...).astype("str") also passes.
pandas/io/pytables.py
Outdated
| if get_option("future.infer_string"): | ||
| data = ser.to_numpy() | ||
| else: | ||
| data = ser._values |
There was a problem hiding this comment.
You can probably simplify this and always to .to_numpy()? (or np.asarray(..))
In the case of object dtype in the else branch, that will return the same (and be as cheap) as _values I think
|
@rhshadrach can you update this? |
|
@jorisvandenbossche - the issue with Perhaps there could be a way (e.g. |
|
Hmm, good point. Ideally we would be able to solve this without using private APIs, I think, because it is a good case study for what also other people (external code) could run into. So I think what we have said before is that downstream users could do But this also makes me wonder if we should re-discuss if we have to add some keyword to |
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.