Skip to content

Improve docutils.parsers #14191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 3, 2025
Merged

Conversation

donBarbos
Copy link
Contributor

No description provided.

This comment has been minimized.

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked into the primer hits, although find them a bit concerning. For now, some remarks below.

class Parser(Component):
settings_spec: ClassVar[tuple[Any, ...]]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's Any here? Most Anys need a comment explaining their use.

Copy link
Contributor Author

@donBarbos donBarbos May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to remove this line because this attribute is already described in the parent class, and I added comment there:
# Mixed tuple structure; uses Any for flexibility in nested option definitions

__docformat__: Final = "reStructuredText"

class BasePseudoSection(Directive):
option_spec: ClassVar[dict[str, Callable[[str], Any]]]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above. More instances below. Maybe use a TypeAlias, so you only need to explain Any once?

Copy link
Contributor Author

@donBarbos donBarbos May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, it comes from the parent Directive class whose argument has already been defined using Any, I just wanted to show that dict value can't be None (only Callable)

but I sent this PR not to fix errors, but to add missing objects, trying not to leave Incomplete

Copy link
Contributor Author

@donBarbos donBarbos May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks to you I noticed this and now I have shown that argument of Callable can be str | None, and the result str | list[str] and most of the similar functions are taken from docutils/parsers/rst/directives/__init__.py module

in particular, this is why we got more effects from mypy_primer

This comment has been minimized.

Harry-Lees added a commit to Harry-Lees/typeshed that referenced this pull request Jun 1, 2025
@srittau srittau closed this Jun 2, 2025
@srittau srittau reopened this Jun 2, 2025

This comment has been minimized.

@@ -27,22 +30,23 @@ class Directive:
required_arguments: ClassVar[int]
optional_arguments: ClassVar[int]
final_argument_whitespace: ClassVar[bool]
option_spec: ClassVar[dict[str, Callable[[str], Any]] | None]
option_spec: ClassVar[dict[str, Callable[[str | None], Incomplete]] | None]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most of the primer hits are due to the changed option_spec definition. I would suggest to create a type alias and use it here in other places where option_spec is defined, but use the old definition for now. (Except for Any -> Incomplete.) Then we can look at this problem in depth in a separate PR if necessary.

Copy link
Contributor

github-actions bot commented Jun 2, 2025

Diff from mypy_primer, showing the effect of this PR on open source code:

sphinx (https://github.com/sphinx-doc/sphinx)
- sphinx/directives/other.py:10: note: In module imported here:
+ sphinx/util/docutils.py: note: In member "get_source_info" of class "SphinxDirective":
+ sphinx/util/docutils.py:488:16: error: Incompatible return value type (got "tuple[str, int] | tuple[None, None]", expected "tuple[str, int]")  [return-value]
+ sphinx/directives/__init__.py: note: In member "run" of class "DefaultRole":
+ sphinx/directives/__init__.py:332:24: error: Argument 2 to "role" has incompatible type "_RstLanguageModule"; expected "_LanguageModule"  [arg-type]
+ sphinx/directives/patches.py:33: error: Unused "type: ignore" comment  [unused-ignore]
+ sphinx/directives/patches.py: note: In member "run" of class "Figure":
+ sphinx/directives/patches.py:42:20: error: Incompatible return value type (got "Sequence[Node]", expected "list[Node]")  [return-value]
+ sphinx/directives/patches.py: note: At top level:
+ sphinx/directives/patches.py:59: error: Unused "type: ignore" comment  [unused-ignore]
+ sphinx/directives/patches.py: note: In member "run" of class "CSVTable":
+ sphinx/directives/patches.py:82:16: error: Call to untyped function "run" in typed context  [no-untyped-call]
+ sphinx/directives/other.py: note: In member "run" of class "Include":
+ sphinx/directives/other.py:409:13: error: Cannot assign to a method  [method-assign]
+ sphinx/directives/other.py:409:47: error: Incompatible types in assignment (expression has type "Callable[[list[str], str], None]", variable has type "Callable[[list[str] | StringList, str], None]")  [assignment]
+ sphinx/directives/code.py: note: In member "run" of class "CodeBlock":
+ sphinx/directives/code.py:146:74: error: Argument "location" to "dedent_lines" has incompatible type "tuple[str, int] | tuple[None, None]"; expected "tuple[str, int] | None"  [arg-type]
+ sphinx/directives/code.py: note: In member "run" of class "LiteralInclude":
+ sphinx/directives/code.py:462:48: error: Argument "location" to "read" of "LiteralIncludeReader" has incompatible type "tuple[str, int] | tuple[None, None]"; expected "tuple[str, int] | None"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
- src/bokeh/sphinxext/bokeh_palette_group.py:53: note: In module imported here:

alectryon (https://github.com/cpitclaudel/alectryon)
+ alectryon/docutils.py:818: error: Dict entry 1 has incompatible type "str": "Callable[[str], int]"; expected "str": "Callable[[str], str | list[str]]"  [dict-item]
+ alectryon/docutils.py:819: error: Dict entry 2 has incompatible type "str": "Callable[[str | None], None]"; expected "str": "Callable[[str], str | list[str]]"  [dict-item]
- alectryon/docutils.py:1534: error: Unused "type: ignore" comment  [unused-ignore]

@srittau
Copy link
Collaborator

srittau commented Jun 3, 2025

The new alectryon primer hit is due to a previously untyped module. The sphinx errors are a mixture, but mostly errors or problems on sphinx's side as far as I can see.

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@srittau srittau merged commit 2a742fd into python:main Jun 3, 2025
49 checks passed
Harry-Lees added a commit to Harry-Lees/typeshed that referenced this pull request Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants