py-libp2p Revamp: Organizational and Code Structure Contributions

[seetadev]

@arcinston : Your recent contributions and insights into improving py-libp2p—both at the organization level and in code architecture—have been truly fantastic. 🙌 We're excited about the direction you're helping to shape and believe your ideas can significantly elevate the project.

We’d love to invite you to share your thoughts, suggestions, or even RFC-style proposals here with the broader team and community.

Feel free to start threads on:

Architectural improvements you're implementing or envisioning

Design patterns or refactors that have helped

Ideas on structuring the project more cleanly for contributors

Anything that could use community input!

Looking forward to your input and further collaboration. 🚀

[arcinston]

1. migration from setup.py to pyproject.toml (which is the current standard )to bring native support for all python package managers (including pip , poetry , uv) to make managing packages seammless
2. current CI / Pre-commit hook fails to catch a lot of basic type issues and possible runtime errors (also pretty bloated imo) , using a standard linter/formatter to do this which a seperate config file makes the whole process cleaner , i would recommend using Ruff (as both linter and formatter) - v fast and seemless - i will ensure that the previously used linting and formating rules are used

examples of this

1. tcp.py (pyright throwing type error)

2. echo.py

the above issue is pretty imp to resolve - as broken code can possibly be pushed

low-priority

1. personally find pre-commit hooks not as great - prevent using the version control system locally , would be in favour of replacing them with Github Actions for CI
   Ruff has a github action for typechecks linting & formatting https://github.com/astral-sh/ruff-action
   can add more soon to have much better DevEx

[arcinston]

https://github.com/arcinston/py-libp2p - have a Work In Progress fork here with the above mentioned changes

[seetadev]

@arcinston : That's awesome progress — only 2 type errors left! 😄 Feels like we're nearly there. Thanks for digging into both issues with clear notes.

About autonat_pb2_grpc.py:

You're right — that grpc.experimental error usually comes up after a grpcio version bump. The module structure changes subtly between versions, and generated code from older protoc runs can become incompatible. Regenerating all protobuf/gRPC files sounds like the cleanest fix. It's definitely orthogonal enough that someone else could take it on without blocking current work. We could even add a small script or Makefile target to make regeneration easier for everyone going forward.

I'll ask @Winter-Soren (Soham wrote the autonat module) and @sumanjeet0012 to collaborate here with you.

About BasicHost.run:

Appreciate you taking a stab at that override issue — ABCs with complex inheritance hierarchies can be surprisingly tricky, especially when protocols or abstract base classes are involved. If the parent class defines run() with a different signature or annotations, even a small mismatch can trigger the bad-override warning.

Might be worth double-checking:

• That the IHost.run() method has the exact same signature (including types and whether it’s async)
• Whether BasicHost.run() is missing @override (if you're using typing.override) or other annotations that help the type checker infer intent

I'll ask @AkMo3, and @acul71 to help here as they have been working with us on this part of the stack.

Great progress indeed 🚀We’re in a great spot overall!

[arcinston]

can add the correct annotation which satisfies pyright then add a type ignore for pyrefly

pyrefly will probably fix it soon!

[AkMo3]

Yup, raised the PR to your fork fixing the type for IHost

[acul71]

Fix in
arcinston#31

[pacrob]

We could even add a small script or Makefile target to make regeneration easier for everyone going forward.

This exists in the Makefile targets protobufs (generate) and clean-proto (delete).

[acul71]

We could even add a small script or Makefile target to make regeneration easier for everyone going forward.

This exists in the Makefile targets protobufs (generate) and clean-proto (delete).

make or make all runs make protobufs
Better add this instructions to the Makefile help ?

help:
    @echo "clean-build - remove build artifacts"
    @echo "clean-pyc - remove Python file artifacts"
    @echo "clean - run clean-build and clean-pyc"
    @echo "dist - build package and cat contents of the dist directory"
    @echo "fix - fix formatting & linting issues with ruff"
    @echo "lint - fix linting issues with pre-commit"
    @echo "test - run tests quickly with the default Python"
    @echo "docs - generate docs and open in browser (linux-docs for version on linux)"
    @echo "package-test - build package and install it in a venv for manual testing"
    @echo "notes - consume towncrier newsfragments and update release notes in docs - requires bump to be set"
    @echo "release - package and upload a release (does not run notes target) - requires bump to be set"

[guha-rahul]

Integrating a pattern like this example from Reth into the project's Makefile can improve developer experience by consolidating common commands (e.g., make lint, make typecheck) into a single, user-friendly target. This will probably help streamline onboarding and reduces friction when running routine checks. Would really love to know your thoughts on this.

[guha-rahul]

update: I put up a pr and it got merged #672 (comment)

[Winter-Soren]

I have fixed some pyrefly type checking issues in the PR #611 :

In tests/core/security/test_security_multistream.py: Resolved "Object of class IMuxedConn has no attribute" errors by implementing a more flexible approach to access secured connections, using isinstance() checks and graceful fallbacks for different muxed connection implementations.

This fix went beyond the original scope of just the relay module - identified and addressed type checking issues in the test suite.