Skip to content

Show origin of conflicting PyPI dependencies #3799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

noib3
Copy link

@noib3 noib3 commented May 20, 2025

Closes #3405.

@noib3 noib3 force-pushed the dependencies-origin branch from 3626dbf to 5dc4a43 Compare May 20, 2025 15:09
@noib3 noib3 force-pushed the dependencies-origin branch from 5dc4a43 to 6577915 Compare May 21, 2025 07:15
@noib3
Copy link
Author

noib3 commented May 21, 2025

@tdejager after parsing uv_resolver's ResolveError, is there a sort of dependency graph one can walk bottom-up to figure out whether a dependency is direct or transitive?

@tdejager
Copy link
Contributor

Yeah good question. So what I can find from reading the code is that it depends on the Error if you get a NoSolutionError it contains a DerivationGraph through PubGrub. But those fields are not public.

We do have the PyPI Dependencies that come in through that function, which are essentially the dependencies that the user requests for that environment. However, that can also contain source dependencies which in turn may define a couple of dependencies.

We could try making an improvent for the simple case first? The case that is also present in the issue I think :)

@noib3
Copy link
Author

noib3 commented May 25, 2025

The requirements and constraints that are given to uv's resolver are:

requirements: [
    Requirement {
        name: PackageName("numpy"),
        extras: [],
        groups: [],
        marker: true,
        source: Registry {
            specifier: VersionSpecifiers(
                [
                    VersionSpecifier {
                        operator: LessThan,
                        version: "2.2",
                    },
                ],
            ),
            index: None,
            conflict: None,
        },
        origin: None,
    },
]
constraints: Constraints(
    {
        PackageName("gdal"): [
            Requirement {
                name: PackageName("gdal"),
                extras: [],
                groups: [],
                marker: true,
                source: Registry {
                    specifier: VersionSpecifiers(
                        [
                            VersionSpecifier {
                                operator: Equal,
                                version: "3.11.0",
                            },
                        ],
                    ),
                    index: None,
                    conflict: None,
                },
                origin: None,
            },
        ],
        PackageName("numpy"): [
            Requirement {
                name: PackageName("numpy"),
                extras: [],
                groups: [],
                marker: true,
                source: Registry {
                    specifier: VersionSpecifiers(
                        [
                            VersionSpecifier {
                                operator: Equal,
                                version: "2.2.6",
                            },
                        ],
                    ),
                    index: None,
                    conflict: None,
                },
                origin: None,
            },
        ],
    },
)

So the inputs to the resolver are already "linearized", i.e. I don't think it's possible to tell that python==2.2.6 is gdal's dependency at that point.

I tried dumping out the contents of locked_pixi_records -- which AFAIU contains metadata about every package in the entire dependency graph -- to see if I could find something useful. I found the following entry for gdal:

RepoDataRecord {
    package_record: PackageRecord {
        arch: None,
        build: "py313ha0b673b_3",
        build_number: 3,
        constrains: [],
        depends: [
            "__osx >=10.13",
            "libcxx >=18",
            "libgdal-core 3.11.0.*",
            "numpy >=1.21,<3",
            "python >=3.13,<3.14.0a0",
            "python_abi 3.13.* *_cp313",
        ],
        extra_depends: {},
        features: None,
        legacy_bz2_md5: None,
        legacy_bz2_size: None,
        license: Some("MIT"),
        license_family: Some("MIT"),
        md5: Some(..),
        name: PackageName {
            normalized: None,
            source: "gdal",
        },
        noarch: NoArchType(None),
        platform: None,
        purls: Some(
            {
                GenericPurl {
                    package_type: "pypi",
                    parts: PurlParts {
                        namespace: "",
                        name: "gdal",
                        version: "",
                        qualifiers: Qualifiers {
                            qualifiers: [
                                (
                                    QualifierKey(
                                        "source",
                                    ),
                                    "hash-mapping",
                                ),
                            ],
                        },
                        subpath: "",
                    },
                },
            },
        ),
        python_site_packages_path: None,
        run_exports: None,
        sha256: Some(..),
        size: Some(1800066),
        subdir: "osx-64",
        timestamp: Some(2025-05-15T15:06:10.970Z,,
        track_features: [],
        version: VersionWithSource {
            version: Version {
                version: [[0], [3], [11], [0]],
                local: [],
            },
            source: Some("3.11.0")},
    },
    file_name: "gdal-3.11.0-py313ha0b673b_3.conda",
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(Domain("conda.anaconda.org")),
        port: None,
        path: "/conda-forge/osx-64/gdal-3.11.0-py313ha0b673b_3.conda",
        query: None,
        fragment: None,
    },
    channel: Some("https://conda.anaconda.org/conda-forge"),
},

Which, even though it lists numpy as one of its dependencies, it just reports it as "numpy >=1.21,<3" without a fully resolved version, which would match both numpy<2.2 and numpy==2.2.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

unsuccessful solve should show origins of conflicting dependencies
2 participants