I made some code to extract the backends declared in the pyproject.toml files (1)
I used @sethmlarson great article : "Querying every file in every release on the Python Package Index" (2)
I called the gist (3) to dowload the files and everything worked fine.
Only the root pyproject.toml is now analyzed.
LOG SCALED USED
#python #packaging-backends
(1) https://gitlab.liris.cnrs.fr/fconil-small-programs/packaging/get-pypi-packages-backends
(2) https://sethmlarson.dev/security-developer-in-residence-weekly-report-18
(3) https://gist.github.com/sethmlarson/852341a9b7899eda7d22d8c362c0a095
Here is the resulting bar chart (1/2)
@fcodvpt @sethmlarson That's not what I see on github:
path:pyproject.toml "setuptools.build_meta" : 28.7k files
path:pyproject.toml "poetry.core.masonry.api" : 96.8k files
@mdk @sethmlarson New chart published, if you want to check.
I only parsed the root pyproject.toml file now, this can explain why the is less occurrences than the github search (poetry has 22 pyprojects.toml files with a "build-backend" key, for test purposes)
@fcodvpt @sethmlarson Thanks for the ping! Yeah this looks more like what I expected.
What would be very interesting (to my eyes) is to see evolution over time. a bit like in https://git.afpy.org/mdk/python-versions, to see "migrations" from one backend to another, to see "birth" (or deaths?) of backends, and so on... Last time I did this kind of things I crawled github instead of PyPI, for a talk: https://git.afpy.org/mdk/talks/src/branch/main/2018-pyconfr-emergence.md (it took ages :D)
@mdk @sethmlarson I totally agree and I would like that too. I may not have the time right now, but if I can I will make it.
What do you think of making a graph removing the ":object" part of the backend ?
Thanks for you fast reaction on my first chart :)
@fcodvpt I don't know, some backends are so weird, I don't think it can even work. Like "setuptools", there's only 1, I don't even understand how it managed to land on PyPI :D
I think you could just exclude those with a single occurrence from the graph. They are interesting, some of them do work, they could be listed, but they "pollute' the graph a lot.
@mdk I made 2 charts (normal scale and log scale) where I removed the backend which were used less than 5 times