feat: Add the ability to skip install of local source dependencies

[olivier-lacroix]

Fixes #1652

This PR implements an additional --skip-local-sources argument for pixi install, that will skip installation of local path dependencies.

As this argument is intended to be used in cases where these local dependencies may be absent, we do not want to update the lockfile, and the use of --skip-local-sources requires the "--frozen" argument.

With this PR, a dockerfile using pixi may look like

FROM python:3.12.2-slim

WORKDIR /app

COPY pixi.lock pixi.toml .

RUN pixi install --frozen --skip-local-sources

COPY sample-package . 

RUN pixi install --frozen

[ruben-arts]

Thanks @olivier-lacroix, would you be able to add a test to validate that this works for both conda and PyPI path dependencies. And add the CLI documentation?

[olivier-lacroix]

Thanks @olivier-lacroix, would you be able to add a test to validate that this works for both conda and PyPI path dependencies. And add the CLI documentation?

Can do yes :-)

[olivier-lacroix]

@ruben-arts what sort of test(s) do you think would be best? unit test(s) and/or integration tests?

[ruben-arts]

For Pypi you might use the src/install_pypi/plan/test code. But for the full picture I would write an integration test. Please try to keep these fast, small packages from the local test data channels.

[olivier-lacroix]

Made this public to be able to test if a package was in the prefix. Is this ok?

[olivier-lacroix]

@ruben-arts integration test added. Let me know what you think.

[tdejager]

Yeah you are right that should be fine :) only skipping the folders then, I suppose. You would still want to install sdists from paths and direct urls right? Or do you want to skip those as well with the flag set?

…

On Tue, 18 Feb 2025 at 22:30, Olivier Lacroix ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/lock_file/update.rs <#3092 (comment)>: > @@ -483,6 +498,10 @@ impl<'p> LockFileDerivedData<'p> { Ok(locked_env .conda_packages(platform) + .map(|packages| { + packages + .filter(|p| !no_path_dependencies || matches!(p.location(), UrlOrPath::Url(_))) @tdejager <https://github.com/tdejager> would detecting if the path is a folder or not work? — Reply to this email directly, view it on GitHub <#3092 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADF4XRTROOCA7B45NFCVQD2QORADAVCNFSM6AAAAABWYRQH5OVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDMMRVGA3TMMZXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[olivier-lacroix]

Yeah you are right that should be fine :) only skipping the folders then, I suppose. You would still want to install sdists from paths and direct urls right? Or do you want to skip those as well with the flag set?

I guess so. the intent is really to skip packages that are being developed.

Quick one, is there a straightforward way to know if a Utf8TypedPathBuf is a directory? not too sure how to convert this to a type that can tell me if its a directory, without going through strings, which, I assume, could be pretty brittle.

[tdejager]

Utf8TypedPathBuf

I think as_std_path().is_dir(), does that work? Also, this does pivot the PR a bit but we could also have a --skip-install <pkg> that could skip the installation of any package, do you think that would make sense?

[olivier-lacroix]

I think as_std_path().is_dir(), does that work?

That'd be nice, but this is available in the camino crate, not the typed-path crate that is in use here . I have based the logic on file extension: A location is deemed a 'local source' if it is a path that is NOT a wheel or an egg file

[olivier-lacroix]

Alright, @tdejager @ruben-arts this PR is about ready to be merged I think.

@tdejager , a --skip flag could definitely be added if there is demand for it. Not sure what use cases there would be for ignoring a specific package though. I propose to defer adding this until some users ask for it? I created #3182 to track

I was not 100% happy with the flag name --no-path-dependencies, as local built artifact are not skipped, and switched to --skip-local-sources. this would make it consistent with a potential future --skip flag as well 😃 . Happy with that?

[tdejager]

Hey @olivier-lacroix, @ruben-arts and myself have been discussing this PR offline just now, and we wanted to report back. So while we do think the idea is good there are a couple of things, also because of my suggestion that do not work out. So supposing you still want to skip development dependencies:

1. I wrongly suggested as_source as that also includes git dependencies that you do want to include in the initial layering.
2. I think you also still want to include the sdists that would not be excluded, e.g: .zip, .tar.gz and maybe other extensions as well: https://github.com/astral-sh/uv/blob/f9c1684b96d1b3a88c4d1f4adb4db5e018a7d54f/crates/uv-distribution-filename/src/extension.rs#L29

Additionally, @ruben-arts and I discussed that we think that a general --skip makes more sense, as it would simplify some of the logic and you would need to be very explicit what you skip.

We are really sorry about the tardiness regarding this PR, and totally understand that it would be to much work to change this so happy to take this over from you if you do not want to make the change.

[olivier-lacroix]

@tdejager fair enough. I'll implement it.