rpms / uv

Clone
Source Code
GIT

#18 Install a default system-wide uv.toml
Merged 7 months ago by music. Opened 7 months ago by music.
rpms/ music/uv uv-toml  into  rawhide

file modified
+9 
@@ -146,6 +146,9 @@ 
 

  # LICENSE.dependencies contains a full license breakdown
 

  URL:            https://github.com/astral-sh/uv
 

  Source0:        %{url}/archive/%{version}/uv-%{version}.tar.gz
 

+ # Default system-wide configuration file
 

+ # https://docs.astral.sh/uv/configuration/files
 

+ Source1:        uv.toml
 

  

  # Currently, uv must use a fork of async_zip, as explained in:
 

  #   Restore central directory buffering
 
@@ -684,6 +687,9 @@ 
 

    install -Dpm 0644 _${cmd} -t %{buildroot}/%{zsh_completions_dir}
 

  done
 

  

+ # Install a default system-wide configuration file
 

+ install -t '%{buildroot}%{_sysconfdir}/uv' -p -m 0644 -D '%{SOURCE1}'
 

+ 

  

  %check
 

  %if %{with check}
 
@@ -721,6 +727,9 @@ 
 

  %{fish_completions_dir}/{uv,uvx}.fish
 

  %{zsh_completions_dir}/_{uv,uvx}
 

  

+ %dir %{_sysconfdir}/uv
 

+ %config(noreplace) %{_sysconfdir}/uv/uv.toml
 

+ 

  

  %files -n python3-uv -f %{pyproject_files}

file added
+22 
@@ -0,0 +1,22 @@ 
 

+ # The RPM-packaged uv deviates from the default configuration in two ways.
 

+ #
 

+ # First, we set "python-downloads" to "manual" in order to avoid unintended
 

+ # Python downloads. We suggest using RPM-packaged (system) Pythons that benefit
 

+ # from distribution maintenance and integration. Use "uv python install" to
 

+ # manually install managed Pythons.
 

+ #
 

+ # Second, we set "python-preference" to "system" instead of "managed".
 

+ # Otherwise, any managed Python would be used for uv operations where no
 

+ # particular Python is specified, even if the only available managed Python
 

+ # were much older than the primary system Python.
 

+ #
 

+ # No choices can be appropriate for all users and applications. To restore the
 

+ # default behavior, comment out settings in this file or override them in a
 

+ # configuration file with higher precedence, such as a user-level configuration
 

+ # file. See https://docs.astral.sh/uv/configuration/files/ for details on the
 

+ # interaction of project-, user-, and system-level configuration files.
 

+ #
 

+ # https://docs.astral.sh/uv/reference/settings/#python-downloads
 

+ python-downloads = "manual"
 

+ # https://docs.astral.sh/uv/reference/settings/#python-preference
 

+ python-preference = "system"

Configures python-downloads = "manual", which keeps uv from automatically downloading precompiled Python interpreters from the Internet without asking; users can still ask to install these interpreters with uv python install.

Original discussion and upstream documentation links in https://src.fedoraproject.org/rpms/uv/pull-request/16.

One choice I make here is to install the configuration file in the XDG-specified path /etc/xdg/uv/uv.toml rather than the also-supported /etc/uv/uv.toml. That seemed like the right choice to me, but I admit that /etc/xdg/ is not widely used in practice.

Another choice is not to set python-preference = "system", at least for now. As @churchyard noted, users who have manually downloaded Python interpreters with uv python install possibly/probably expect that they will be used. When I raised the various choices involved in this config file in the Fedora Python Matrix channel, a couple of people spoke up in favor of adding python-preference = "system", but didn’t offer a rationale. This could be revisited in the future if someone comes up with a good justification.

Sample behaviour on Fedora 41 with python3.13 installed (of course) and with python3.12 not installed.

$ uv venv _py313 --python 3.13
Using CPython 3.13.0 interpreter at: /usr/bin/python3
Creating virtual environment at: _py313
Activate with: source _py313/bin/activate
$ uv venv _py312 --python 3.12
  × No interpreter found for Python 3.12 in managed installations or system path

Before the config file with python-downloads = "manual" was added, the above would have automatically downloaded and installed a managed interpreter. The user now has two choices:

$ sudo dnf install python3.12
[…]
$ uv venv _py312 --python 3.12
Using CPython 3.12.7 interpreter at: /usr/bin/python3.12
Creating virtual environment at: _py312
Activate with: source _py312/bin/activate

Or:

$ uv python install 3.12
Installed Python 3.12.7 in 1.85s
 + cpython-3.12.7-linux-x86_64-gnu
$ uv venv _py312 --python 3.12
Using CPython 3.12.7
Creating virtual environment at: _py312
Activate with: source _py312/bin/activate

Disabling automatic Python interpreter downloads could be seen as a breaking change for some users, but there is some compromise required when we’re chasing a relatively new and fast-moving target, and the availability of system-wide configuration in uv is brand new.

I like the default values.

About the path/location: When both /etc/uv.toml and /etc/xdg/uv.toml exists, which one is preferred? If it's /etc/uv.toml, it allows users/administrators to override our config without overriding it on disk, which would be nice.

I like the default values.

About the path/location: When both /etc/uv.toml and /etc/xdg/uv.toml exists, which one is preferred? If it's /etc/uv.toml, it allows users/administrators to override our config without overriding it on disk, which would be nice.

From https://docs.astral.sh/uv/configuration/files/, “If multiple system-level configuration files are found, e.g., at both /etc/uv/uv.toml and $XDG_CONFIG_DIRS/uv/uv.toml, only the first-discovered file will be used, with XDG taking priority.”

Note that for both paths, there is a uv directory, i.e. /etc/uv/uv.toml rather than /etc/uv.toml.

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/7167be44f9094c3ba4355b3c659928c8

Isn't the ability to automatically download Pythons kind of a central feature? "Download Python versions as needed" is documented in upstream's README. I am generally in support of this for the reasons already mentioned but worry about the user impact.

About the path/location: […]

As a follow-up, any system configuration file will still be overridden (merged at a per-setting level) by user-level or project-level configuration.

Isn't the ability to automatically download Pythons kind of a central feature? "Download Python versions as needed" is documented in upstream's README. I am generally in support of this for the reasons already mentioned but worry about the user impact.

It’s certainly a major feature, and anything we do to override defaults has the potential to create surprise and frustration. Whatever we choose should be the minimum change we consider necessary.

This kind of discussion is one reason I’m trying to be rather “noisy” about this and not rush to ship something. If we add a configuration file to stable releases, I’d like to feel reasonably confident it’s something the Fedora Python community can collectively live with rather than something that keeps getting fiddled with as more and more people are caught by surprise and chime in with new perspectives that perhaps don’t align with initial feedback.

Perhaps we could work with upstream to improve the "interpreter not found" case?

$ uv venv _py312 --python 3.12
  × No interpreter found for Python 3.12 in managed installations or system path
  × Not automatically downloading Python 3.12 because python-downloads is set to "manual" in /etc/xdg/uv/uv.toml. You can download it with: uv python install 3.12

About the path/location: When both /etc/uv.toml and /etc/xdg/uv.toml exists, which one is preferred? If it's /etc/uv.toml, it allows users/administrators to override our config without overriding it on disk, which would be nice.

From https://docs.astral.sh/uv/configuration/files/, “If multiple system-level configuration files are found, e.g., at both /etc/uv/uv.toml and $XDG_CONFIG_DIRS/uv/uv.toml, only the first-discovered file will be used, with XDG taking priority.”

In that case, I think we should install the file into /etc/uv/uv.toml. Giving the users a way to globally cover it it with $XDG_CONFIG_DIRS/uv/uv.toml. (Not a strong preference.)

Note that for both paths, there is a uv directory, i.e. /etc/uv/uv.toml rather than /etc/uv.toml.

Ack, sorry about that.

I'd also suggest including a comment near python-downloads = "manual" that says something like (feel free to reword):

The RPM-packaged uv deviates from the default by setting this to "manual" to avoid accidental Python downloads. There are plenty of RPM-packaged Pythons available that we believe are better suited for our users. Comment this line to use the default instead:

Also, I checked: removing and installing uv after deleting the config file restores it, which is not nice :/

Also, I checked: removing and installing uv after deleting the config file restores it, which is not nice :/

This is just standard RPM behavior for config files, though. I am not sure how RPM could remember that a removed previously-installed was missing a config file when it was uninstalled, but in any case users should expect uv’s config file to behave like all the other config files in the distribution.

In that case, I think we should install the file into /etc/uv/uv.toml. Giving the users a way to globally cover it it with $XDG_CONFIG_DIRS/uv/uv.toml. (Not a strong preference.)

I’m open to that. I have a few thoughts on it, though:

It’s a neat trick, but also kind of a hack. I think duplicate configuration files with one shadowing the other is surprising and not very discoverable unless it follows the pattern of having the default configuration files under /usr/lib and the override file under /etc as systemd and a number of other things do. I would not want to have to remember the priority order of two similarly-named config files under /etc if I were a sysadmin digging through someone else’s work.

Previous testing shows that adding a second configuration files path and its directory with %ghost doesn’t seem to be meaningfully different from not owning it at all. Maybe creating and owning the second directory, and only %ghosting the second config file path itself, would work better.

This is not to say that there is something wrong with choosing /etc/uv/uv.toml, in and of itself. We could always just choose that path, semi-arbitrary but with an eye toward your suggestion, not %ghosting the XDG path, and let people do what they like.

I would defer to precedent here if there were any, but

  • Fedora’s pip doesn’t install a default configuration file
  • Of the distros that have uv, I don’t know of any that already install a default config file (this is really bleeding-edge stuff)

Gentoo wants to and we can coordinate with them.

https://floss.social/@mgorny@treehouse.systems/113432309724108665

Gentoo wants to and we can coordinate with them.

https://floss.social/@mgorny@treehouse.systems/113432309724108665

Thanks, that’s good to know, and it’s nice for Fedora not to be considering this alone. It would be nice if at least Fedora and Gentoo could end up using the same config file paths.

Yeah, the UV's idea of using two different config file paths in /etc looks quite confusing. Perhaps it would be better to settle on one, and stop using the other.

As far as Gentoo does, I think we'll just settle on installing one file and expecting the users to modify it if they don't like our defaults. I don't think they should or need to shadow it — our config update mechanism should handle user changes well.

Yeah, the UV's idea of using two different config file paths in /etc looks quite confusing. Perhaps it would be better to settle on one, and stop using the other.

I was going to claim that the upstream goal was just to support XDG, rather than to support some kind of overriding scheme, but https://github.com/astral-sh/uv/pull/7851#issuecomment-2408700995 actually mentions the idea of overriding /etc/uv/uv.toml with the XDG path. I guess that’s an argument for using /etc/uv/uv.toml in a distribution. I am not sure we need to %ghost or otherwise acknowledge the XDG path in that case, but it gives users the option to use that pattern if they want to.

Yeah, the UV's idea of using two different config file paths in /etc looks quite confusing. Perhaps it would be better to settle on one, and stop using the other.

I was going to claim that the upstream goal was just to support XDG, rather than to support some kind of overriding scheme, but https://github.com/astral-sh/uv/pull/7851#issuecomment-2408700995 actually mentions the idea of overriding /etc/uv/uv.toml with the XDG path. I guess that’s an argument for using /etc/uv/uv.toml in a distribution. I am not sure we need to %ghost or otherwise acknowledge the XDG path in that case, but it gives users the option to use that pattern if they want to.

rebased onto 4abac29
7 months ago

I added text similar to what was suggested in https://src.fedoraproject.org/rpms/uv/pull-request/18#comment-228579 to the config file, removed the large comment blocks that duplicated uv’s documentation for python-downloads and python-preference, and changed the path from /etc/xdg/uv/uv.toml to /etc/uv/uv.toml.

I like it now, thanks!

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/10d6bd0915bb4ba3b0455ad5e35c4d4b

It looks like the current form of this PR is pretty well aligned with Gentoo.

I’m inclined to review the wording of the comment for clarity one more time and then plan to ship this to stable releases along with release 0.5.0, which brings a few other changes that could be considered breaking for some users.

Well, just to be precise, I've made our changes based on this pull request. Thanks for your work!

Okay, one thing I suspected and now confirmed is that this change breaks testing. I'm thinking of trying to use /etc/xdg directory instead, as that will be easier to workaround in tests (presuming uv respect XDG_CONFIG_DIRS).

Okay, one thing I suspected and now confirmed is that this change breaks testing. I'm thinking of trying to use /etc/xdg directory instead, as that will be easier to workaround in tests (presuming uv respect XDG_CONFIG_DIRS).

Can you clarify in exactly which way this breaks testing?

If I’m not misunderstanding what you’re trying to do, the easiest way to override this setting should in most cases be a user-level uv.toml file, at ~/.config/uv/uv.toml or $XDG_CONFIG_HOME/uv/uv.toml, which will be merged with the system-level file, with the user-level file overriding it with per-setting granularity.

Well, apparently there are a few show_settings tests that actually rely on specific settings, e.g.:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: resolve_uv_toml
Source: crates/uv/tests/it/show_settings.rs:40
────────────────────────────────────────────────────────────────────────────────
Expression: snapshot
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
   14    14 │     allow_insecure_host: [],
   15    15 │     show_settings: true,
   16    16 │     preview: Disabled,
   17    17 │     python_preference: Managed,
   18       │-    python_downloads: Automatic,
         18 │+    python_downloads: Manual,
   19    19 │     no_progress: false,
   20    20 │ }
   21    21 │ CacheSettings {
   22    22 │     no_cache: false,
────────────┴───────────────────────────────────────────────────────────────────

I mean, I could create another file during testing that reverses our changes, but it seems simpler to avoid loading the system config in the first place, by overriding XDG_CONFIG_DIRS, i.e. https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/uv/uv-0.5.1.ebuild?id=7b13268794fb101bc1c7a4cc37ee3fb61121acf5#n140

I guess if we install uv from this PR and run %check again, we might see the failure.

Normally, we don't have uv installed when we build uv, so this should not affect our own builds.

Snapshot: resolve_uv_toml
Source: crates/uv/tests/it/show_settings.rs:40

We already skip the integration tests as a group because almost all of them require the very specific Python interpreter versions that uv would download, down to the patch-release, and those version numbers don’t necessarily precisely match the system interpreters. It looks like the test that is failing for you is among those we are already skipping.

You can avoid specific interpreter tests by disabling python-patch feature. Overall, upstream is very cooperative.

You can avoid specific interpreter tests by disabling python-patch feature. Overall, upstream is very cooperative.

Thanks – that’s really helpful. I had discovered the pypi feature and had noticed the later introduction of the crates-io feature, but I missed python-patch. It looks like it has been default-disabled since 0.2.0, but looking at the uv crate features, it seems like I need to try disabling python-managed. I’ll try that out and see if I can start running more integration tests.

Normally, we don't have uv installed when we build uv, so this should not affect our own builds.

I think this will be correct, even if I get that test running based on the above suggestion. The uv we are testing should be looking for configuration files in the mock chroot’s /etc/uv/, not ${RPM_BUILD_ROOT}/etc/uv/, so it shouldn’t find our custom uv.toml.

In your case, I agree that setting XDG_CONFIG_DIRS to hide the custom uv.toml for testing should work, but looking at the implementation, it will still fall back to /etc/uv/uv.toml if it doesn’t actually find a uv.toml in any of the XDG_CONFIG_DIRS – so you will probably have to make sure there is a valid ${DIR}/uv/uv.toml for some DIR in XDG_CONFIG_DIRS. That doesn’t have to be in /etc/xdg/; it could be an empty file in a temporary directory as long as you tweak XDG_CONFIG_DIRS appropriately.

There might be some uv-specific environment variable you could set instead, but I suspect UV_NO_CONFIG and UV_CONFIG_FILE are too powerful since they cause uv to ignore user- and/or project-level configuration, which probably breaks some tests.

a couple of people spoke up in favor of adding python-preference = "system", but didn’t offer a rationale. This could be revisited in the future if someone comes up with a good justification.

I also think we should set python-preference = "system". On my system, I have all the Fedora Python interpreter packages installed and then python3.7 installed through uv. Without python-preference = "system", uv defaults to the managed python3.7 for every single operation instead of the system /usr/bin/python3. I expect uv installed from the Fedora repos to use the default system Python interpreter unless I explicitly request a different Python version.

So if even one managed Python is installed, it will be the “default” Python even if it is much older than the system Python. I can reproduce that using uv 0.4.29 built from this PR:

$ uv python install 3.8
$ uv python list
cpython-3.13.0-linux-x86_64-gnu    /usr/bin/python3.13
cpython-3.13.0-linux-x86_64-gnu    /usr/bin/python3 -> python3.13
cpython-3.8.20-linux-x86_64-gnu    .local/share/uv/python/cpython-3.8.20-linux-x86_64-gnu/bin/python3.8
$ uv venv _e
Using CPython 3.8.20
Creating virtual environment at: _e
Activate with: source _e/bin/activate

I agree, that looks like a real problem, and setting python-preference = "system" fixes it. I’m afraid having both of these set may be an unpleasant surprise for some people’s workflows, but it looks necessary.

Could you report that upstream? This behavior sounds a bit surprising to me too.

This behavior sounds a bit surprising to me too.

Which part? That an older version is preferred or that the system one is not used by the default? I think the bigger problem is that the system version is not used. The uv venv command in Fedora's uv package, for example, should default to creating a virtual environment using the Fedora system Python interpreter, even if the system Python is 3.12 and uv has 3.13 installed.

I think it's surprising that an older version is used by default also, but that's orthogonal. I guess upstream could add python-preference = "latest" option that would just use the latest, non-pre-release Python interpreter, regardless of whether it's a uv-managed interpreter or not, but, as a distribution, we should probably still default to system.

The former, I think — but I'm not sure anymore.

I mean, my logic was this: we don't want uv installing Python automatically. But if the user explicitly asked it to, then I suppose it makes sense for that to be preferred over the system Python (since the user asked for it). But when the user asked for it because it's an old Python version we don't have anymore, we don't want it used instead of newer, system version.

Still working on 0.4.29 for now, I tried removing python-managed and git from the default features and adding BuildRequires on the system packages for the upstream-supported Python interpreters. (It turns out that the git feature, described as “introduces a dependency on Git,” actually means “compiles and runs tests for git dependencies, requiring access to remote git repositories on GitHub or similar.”) That does does allow me to run quite a few of the integration tests. I’ve gated the new BuildRequires and additional tests behind a build conditional so we can still try testing with only the main system Python if we like.

However, I still see many integration tests trying and failing to access the network, like

---- branching_urls::root_package_splits_other_dependencies_too stdout ----

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Unfiltered output ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
----- stdout -----

----- stderr -----
error: Failed to fetch: `https://pypi.org/simple/iniconfig/`
  Caused by: Could not connect, are you offline?
  Caused by: Request failed after 3 retries
  Caused by: error sending request for url (https://pypi.org/simple/iniconfig/)
  Caused by: client error (Connect)
  Caused by: dns error: failed to lookup address information: Temporary failure in name resolution
  Caused by: failed to lookup address information: Temporary failure in name resolution

────────────────────────────────────────────────────────────────────────────────

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: root_package_splits_other_dependencies_too
Source: crates/uv/tests/it/branching_urls.rs:391
────────────────────────────────────────────────────────────────────────────────
Expression: snapshot
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0       │-success: true
    1       │-exit_code: 0
          0 │+success: false
          1 │+exit_code: 2
    2     2 │ ----- stdout -----
    3     3 │
    4     4 │ ----- stderr -----
    5       │-Resolved 9 packages in [TIME]
          5 │+error: Failed to fetch: `https://pypi.org/simple/iniconfig/`
          6 │+  Caused by: Could not connect, are you offline?
          7 │+  Caused by: Request failed after 3 retries
          8 │+  Caused by: error sending request for url (https://pypi.org/simple/iniconfig/)
          9 │+  Caused by: client error (Connect)
         10 │+  Caused by: dns error: failed to lookup address information: Temporary failure in name resolution
         11 │+  Caused by: failed to lookup address information: Temporary failure in name resolution
────────────┴───────────────────────────────────────────────────────────────────
To update snapshots run `cargo insta review`
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'branching_urls::root_package_splits_other_dependencies_too' panicked at /usr/share/cargo/registry/insta-1.40.0/src/runtime.rs:548:9:
snapshot assertion for 'root_package_splits_other_dependencies_too' failed in line 391

Am I missing something, or is this just a case where I need to send a PR upstream to gate more tests on the git and/or pypi features? @mgorny, did you see something similar?

Well, I've given up entirely on testing offline. I think <10% of the whole test suite can be run like that. Everything else requires fetching stuff from PyPI and elsewhere.

rebased onto 3c5efef
7 months ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci
https://fedora.softwarefactory-project.io/zuul/buildset/d765f85873e44f7788c5d057adc1c312

rebased onto b96c694
7 months ago

I’m going ahead and shipping a uv 0.4.30 update (last 0.4.x release) to F41 and F40, without a uv.toml file. F39 is close enough to end of life that I don’t plan to ship any more updates there.

It doesn’t feel like the discussion here is entirely finished, and I still have some dependency updates I’d like to deal with before diving into 0.5.x.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci
https://fedora.softwarefactory-project.io/zuul/buildset/977c302a130f44da84c5b2f136d3c50e

rebased onto b96c694
7 months ago

Now testing also setting python-preference = "system".

2 new commits added

  • Also configure python-preference = "system"
  • Install a default system-wide uv.toml
7 months ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci
https://fedora.softwarefactory-project.io/zuul/buildset/c395729c36c54589b0577a3aaf273ea3

I’m starting work on uv 0.5.x in Rawhide. I’m going to go ahead and merge this PR as-is first. There’s still room to test, discuss, and revise it before it goes out to stable releases in (I expect) about a week. Any testing that people can do is much appreciated.

Pull-Request has been merged by music
7 months ago

Thanks.

Any testing that people can do is much appreciated.

I've installed the config file form Rawhide on Fedora 41 and will use it for a while.

Any testing that people can do is much appreciated.

I've installed the config file form Rawhide on Fedora 41 and will use it for a while.

Thank you! Make sure you are using https://bodhi.fedoraproject.org/updates/FEDORA-2024-44381a2970 from updates-testing so you have a version of uv that respects the system-wide config file.
Metadata
None
No Tags
Changes Summary 2
+9 -0
file changed
uv.spec
+22
file added
uv.toml
Powered by Pagure 5.14.1
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.