Requoting unquotes %27

[albertvillanova]

Describe the bug

When requoting a redirection URL, you use yarl.URL and the single quotation mark gets unquoted: %27 => '
https://github.com/aio-libs/aiohttp/blob/4635161ee8e7ad321cca46e01ce5bfeb1ad8bf26/aiohttp/client.py#L578-L580

In [1]: from yarl import URL

In [2]: url = "https://netloc/path?param=param%27%27value"

In [3]: print(url)
Out[3]: 'https://netloc/path?param=param%27%27value'

In [4]: print(str(URL(url)))
Out[4]: "https://netloc/path?param=param''value"

However, the requests library does not unquote %27: %27 => %27

In [1]: from requests.utils import requote_uri

In [2]: url = "https://netloc/path?param=param%27%27value"

In [3]: print(url)
Out[3]: 'https://netloc/path?param=param%27%27value'

In [4]: print(requote_uri(url))
Out[4]: 'https://netloc/path?param=param%27%27value'

To Reproduce

from requests.utils import requote_uri
from yarl import URL

url = "https://netloc/path?param=param%27%27value"

print(url)

print(requote_uri(url))

print(URL(url))

Expected behavior

I think both libraries should have the same behavior.

Logs/tracebacks

N/A

Python Version

$ python --version
Python 3.9.7

aiohttp Version

$ python -m pip show aiohttp
Version: 3.8.1

multidict Version

$ python -m pip show multidict
Version: 5.2.0

yarl Version

$ python -m pip show yarl
Version: 1.7.2

OS

Ubuntu Linux

Related component

Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[albertvillanova]

See the reasons argued by the requests team: psf/requests#6341 (comment)

Arguably the RFCs are in Requests' favor. It's safer to leave those quotes because they should be quoted by RFC 3986 and even the What WG URL spec suggests they should be. Not all servers are going to reject aiohttp's behavior but some will. It's safer to be conservative in what is sent (quote reserved characters, etc.) then it is not to.

[bizzyvinci]

Hi @albertvillanova, you'll get the same result as reqoute_uri if you use the encoded argument

>>> from yarl import URL
>>> url = "https://netloc/path?param=param%27%27value"
>>> URL(url)
URL('https://netloc/path?param=param''value')
>>> URL(url, encoded=True)
URL('https://netloc/path?param=param%27%27value')

When using in session

...
async with session.get(URL(url, encoded = True)) as response:
...

[albertvillanova]

@bizzyvinci please note that it is your code in client.py which should properly handle the requoting of the redirection URL by default (as it is the case for the default behavior with requests). See the code line in the issue description above.

However, your default behavior unquotes %27.

[albertvillanova]

I show the corresponding code lines again: https://github.com/aio-libs/aiohttp/blob/4635161ee8e7ad321cca46e01ce5bfeb1ad8bf26/aiohttp/client.py#L578-L580

[Dreamsorcerer]

Would also be fixed by: #1073 (comment)