So #setuptools fun: if your source tree contains a symbolic link to a directory, setuptools may randomly either include the original directory in the source distribution, or move it where the symlink was, disregarding the original location.

So if you have:

a/a
a/b
a/c
b -> a

you'll randomly end up with either:

a/a a/b a/c (and no "b")

or:

b/a b/b b/c (and no "a")

https://github.com/pypa/setuptools/issues/4937

So, now that the urgent fires have been put out for the time being, let's talk what other fun stuff #setuptools are doing these days.

For example, they have implemented PEP 639 recently (new license metadata). While doing that, they immediately deprecated their previous `license-field` field, giving people until 2026-02-18 to update their projects. The deprecation warnings also gives a helpful hint that you need setuptools 77 for the new field, which was released… a week ago.

So yeah, setuptools pretty much tells you that you need to choose between the old solution that means that all the versions of your package using it will stop building next February, or the new solution which means that your package will now require a week-old setuptools release (and effectively kill support for EOL Python versions, for which some projects apparently still care).

And of course the #Python community will tell you to solve the problem by pinning dependencies. And guess what happens when people put arbitrary pins (say, <77) to silence this deprecation warning, and other people have >=77 dependency, because they use the new variant.

So… would you like to talk about flit, perhaps? Or hatchling?

PS. The best joke is that they're pointing at a packaging guide that hasn't even been updated yet and still states that setuptools do not implement PEP 639.

https://github.com/pypa/setuptools/blob/6ead555c5fb29bc57fe6105b1bffc163f56fd558/setuptools/config/_apply_pyprojecttoml.py#L100-L106
https://web.archive.org/web/20250324035753/https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license-files

Domino meme:

1. #setuptools starting to finally normalize wheel filenames per PEP 491.
…
n. #PyCXX suddenly installing header files into `/usr/include/python*/cxx` instead of uppercase `CXX`.

Funny enough, it affects both #pip and #installer. Long story short, the include install path uses "distribution name", and in both cases the distribution name is apparently taken from the filename. Not sure if this is a bug or a feature, but I guess i'll file bugs. Installer even has a TODO item to determine whether they should be using original or normalized name.

https://discuss.python.org/t/installing-headers-from-wheel-should-the-directory-be-normalized-or-not/85990

setuptools team learned yesterday that "Postel's Law" is more like the law of gravity than software architecture guidance.

(Note: credit where credit is absolutely do: setuptools team stepped up and embraced the issue, putting together a very quick rollback to unbreak everybody and move the deprecation deadline further out. That was extremely responsible.)

"But it's not setuptools fault that other projects didn't respond to a deprecation warning from 2021..."

There's a concept of "Responsibility without fault." There's a pretty famous blog post, which I regrettably cannot lay hands upon, where an ex-Microsoft engineer talks about how when they upgraded Windows, testing showed they broke Photoshop. Drilling down revealed that they had changed the details of some C++-implemented APIs that resulted in a binary change to the API buffers, but that should have been irrelevant because the pattern to use those APIs was to request a buffer from the OS and then fill it in.

Adobe had optimized some cycles away by caching their own pre-filled copies of those buffers, which, of course, broke when the binary layout of the buffer changed.

Microsoft's solution? They reimplemented those buffers in C so they could maintain binary compatibility and not break Photoshop. Because if Photoshop broke in a new version of the OS, end-users wouldn't blame Adobe, they'd blame the thing that changed, and Microsoft is in the business of selling operating systems.

It's not about fault, really. It's about "as a software project, do you want to be the one people use or the one people route around?" And that's more of a social network challenge than an engineering challenge.

(This also goes to the question "How could they possibly have known they'd break other projects?" Well... Microsoft maintains a list of must-use software they test new OS versions against. If you're as big as setuptools, you may be big enough to maintain such a list for testing purposes).

https://social.treehouse.systems/@mgorny/114137227235533135

Yes, they did it. #setuptools literally made lots of #Python packages (such as requests) explode, apparently in order to resolve a problem with their own test suite. Sure, that stuff has been deprecated for a long time. But as I've said multiple times, *nobody* sees these deprecation warnings.

Well, unless they run #Gentoo, because we have literally the only Python package installer out there that catches and repeats setuptools deprecation warnings verbosely. But we don't have time to fix deprecations in upstream packages while upstreams are making sure to set up fires all over the place, all the time.

https://github.com/pypa/setuptools/issues/4910

A pamiętacie, jak myśleliście, że można ignorować te wszystkie ostrzeżenia generowane, kiedy #setuptools przypadkowo zmieniał nazwy pierdół, bo przecież nigdy nie usuną wstecznej zgodności?

https://github.com/pypa/setuptools/pull/4870

Remember when you thought it's fine to ignore all these deprecation warnings about #setuptools randomly renaming stuff, because they're never going to remove the backwards compatiblity?

https://github.com/pypa/setuptools/pull/4870

A tak na marginesie, zdaje się, że jedyny powód, dla którego #setuptools jeszcze istnieje, to dawanie ludziom wymówek, żeby tworzyli nowe backendy #PEP517. No, bo w porównaniu z setuptools to każdy wydaje się znacznie lepszy.

Also, I feel like the only purpose of #setuptools these days is to give people excuses to create new #PEP517 backends. Because obviously everything comes out better when compared to setuptools.

I suppose I could use my experience to give some #PEP517 build system recommendations.

For pure #Python packages:

1. #flit_core (https://pypi.org/project/flit-core/) — it's lightweight and simple, and has no dependencies (in modern Python versions, for older Pythons it vendors tomli).

2. #hatchling (https://pypi.org/project/hatchling/) — it's popular and quite powerful, but has many vendored dependencies and no stand-alone test suite (which makes it painful to maintain in #Gentoo).

For Python packages with C extensions: #meson-python (https://pypi.org/project/meson-python/) — which combines the power and correctness of meson build system with good very Python integration.

For Python packages with Rust extensions: #maturin (https://pypi.org/project/maturin/) — which is simply a good builder for precisely that kind of packages.

Now, I strongly discourage:

A. #setuptools — lots of vendored NIH dependencies (that can alternatively be unvendored for cyclic deps), lots of deprecations over time (we're still seeing tons of deprecation warnings all over the place), many unsolved bugs (e.g. parallel C extension builds are broken in a few ways), a lot of technical debt, and if all that wasn't enough, it's slow.

B. #poetry-core — a very tricky build system with lots of pitfalls (I've reported a lot of mistakes done when migrating to it).

C. Practically any other build system — writing new backends is trivial, so everyone and their grandmother must have one. And then, they often carry a lot of NIH dependencies (if you're reinventing a build system, you may reinvent everything else), lack experience and reintroduce the same bugs. And if that wasn't enough, packaging them in distributions is a lot of work for no real benefit to anyone.

Some time ago two #PEP517 build systems introduced #PyPI trove classifier verification. At a first glance, it makes sense. After all, if you made a mistake somewhere, you'd rather know early than when you try to upload the package. The problem is, that the verification fires for people building packages locally too — including #Gentoo users.

Now, this function was based on the #Python "trove-classifiers" package. Whenever a new classifier is introduced, a new release of the package is made. When you're building a package using tools such as `build` or `pip` (isolated build), the newest version of this package is being installed from the Internet. On the other, a Gentoo user may have an old version, unless we enforce an upgrade via package dependencies. Then building packages that use newer classifiers will fail, and with a confusing message too. Confusing because: 1) contrary to the message, the classifier is valid; and 2) even if it weren't, it doesn't affect us in any way.

And so we asked for an ability to disable this. While it took some time, the #Hatchling showed understanding and eventually merged my patch. On the other hand, the #setuptools maintainer… well, started a long and tedious debate that resulted in ignoring the trivial solution to the actual problem (as "unnecessary complexity"). Instead, we were given another option: we could entirely disable `pyproject.toml` validation. It's not really acceptable, for two reasons: 1) because setuptools actually rely on this validation (so removing it could result in broken package installs instead of an error, if the file is not valid), and 2) because it produces an awful warning on every package build. So we'd end up bullying Gentoo users with false warnings, and some of them would probably end up filing invalid bugs to various upstreams.

The bottom line is: don't use setuptools.

https://github.com/pypa/hatch/issues/1368
https://github.com/pypa/setuptools/issues/4459

Jakiś czas temu dwa systemy budowania #PEP517 dorobiły się funkcji weryfikacji klasyfikatorów #PyPI ("trove classifiers"). Na pierwszy rzut oka, ma to sens: wszak warto poinformować autorów paczek od razu, jeżeli gdzieś popełniono błąd i paczka zostanie odrzucona. Problem stanowi jednak fakt, że ta weryfikacja dotyczy również budowania paczek lokalnie — a więc użytkowników #Gentoo.

Funkcję zbudowano w oparciu o paczkę "trove-classifiers". Ilekroć nowy klasyfikator dodawany jest do PyPI, wydawana jest nowa wersja tej paczki. Jeżeli używamy narzędzi takich jak `build` czy `pip`, każdorazowo z sieci instalowana jest najnowsza wersja tej paczki. Na Gentoo natomiast, o ile nie wymusimy tego zależnościami, użytkownik może mieć przestarzałą. Jeżeli wówczas spróbuje zainstalować paczkę Pythona, która używa nowszych klasyfikatorów, otrzyma mylący błąd o błędnym klasyfikatorze. Mylący, bowiem: 1) wbrew komunikatowi, klasyfikator jest poprawny; 2) nawet gdyby nie był, to problem nas w ogóle nie dotyczy.

Dlatego też wystąpiliśmy z prośbą o możliwości wyłączenia tej funkcji. Choć zajęło to trochę czasu, opiekun paczki #Hatchling wykazał zrozumienie dla naszego problemu i zaakceptował moją łatkę. Z kolei opiekun #setuptools… no cóż, podjął się długiej i bezowocnej debaty, która poskutkowała zignorowaniem trywialnego rozwiązania głównego problemu ("niepotrzebna komplikacja"). Zamiast tego, zaproponowano nam rozwiązanie dosyć wątpliwe — całkowite wyłączenie weryfikacji poprawności pliku `pyproject.toml`. Rozwiązanie nieakceptowalne z dwóch powodów: 1) dlatego, że setuptools polega na tej weryfikacji (a więc przy błędach w pliku moglibyśmy dostać zepsutą instalację zamiast błędu); 2) dlatego, że przy budowaniu każdej paczki rzuca paskudnym ostrzeżeniem. W praktyce więc znęcalibyśmy się nad użytkownikami Gentoo, zarzucając ich fałszywymi ostrzeżeniami, i niektórzy z nich prawdopodobnie niepotrzebnie zgłaszaliby problem autorom właściwej paczki.

Nie używajcie setuptools.

https://github.com/pypa/hatch/issues/1368
https://github.com/pypa/setuptools/issues/4459

For two weeks now, the latest #setuptools release has a major race condition bug. Sure, it's not the first race condition in C extension builds. However, this one has a known cause, with clear explanation why the change was wrong, and causes a regression for many packages that weren't affected before.

And perhaps the most absurd part is that many people are using setuptools only for their C extension building support, and that one feature that's keeping the project still relevant today keeps being broken in so many ways. At this point, I'm going to suggest using #meson and its #PEP517 backend, meson-python.

https://github.com/pypa/setuptools/issues/4653

I proposed we finally update #setuptools in #Fedora 42. It breaks a lot of #Python packages :(

150+ packages use the now-removed `setup.py test` command, which was deprecated for 5 years.

I am looking forward to when `setup.py build` and `setup.py install` will be removed as well, that will certainly be fun. Perhaps we should deprecate the old Python RPM macros more loudly (with sleeps, ASCII art Jolly Rogers, and beeps).

https://fedoraproject.org/wiki/Changes/Setuptools_74+

It's release day again for setuptools-pyproject-migration! This project from @stuartl and myself helps you convert your setuptools configurations to modern standard-compliant pyproject.toml files. v0.3 brings many bug fixes; I think we are rapidly approaching the point where the thing actually works

We'd be very grateful if you try it and let us know how it works for you!

https://setuptools-pyproject-migration.readthedocs.io/en/stable/
https://github.com/diazona/setuptools-pyproject-migration

Let's ignore #setuptools deprecation warnings for years, and then *surprised pikachu* when deprecated stuff is removed and everything suddenly falls apart.

https://github.com/pypa/setuptools/issues/4519
https://github.com/psf/requests/issues/6775