Further automation for releasing cherry-picker.py

[Mariatta]

Thanks to #262 (@webknjaz) I can publish to PyPI from Travis CI, all within GitHub web interface, and without going to the command line.

But I found myself still doing several manual steps:

• created the release branch
• drop the ".devX" from the version info (in __init__.py)
• create the tag
• merge the release branch
• bump the version number and add ".devX" to the version info.

Can this be made even simpler? Perhaps I was doing unnecessary steps?

I think that creating the release branch, dropping ".devX", and creating tag can be fully automated somehow.

[Mariatta]

GitMate.io thinks possibly related issues are python/core-workflow#113 (Questions about cherry-picker from Windows never-venv user), https://github.com/python/core-workflow/issues/268 ([cherry-picker] [idea 💡] Extend backport sources), python/core-workflow#249 ([cherry-picker] Document config options), python/core-workflow#88 (Make -h work for cherry-picker), and python/core-workflow#263 (Feedback on trying to adopt cherry-picker within @ansible/ansible).

[webknjaz]

workflow comments

So, in my projects based on pure setuptools with setuptools-scm plugin I don't have "created the release branch" step at all.
If the code is in master I can cut a tag right away (which will trigger testing and publishing machinery). But often before doing it I go through the diff of changes and update changelog. (And push tag after that)

drop the ".devX" from the version info (in __init__.py)

This is redundant in my flow because setuptools_scm identifies the current version as vLAST_TAG.dev0+commit_hash (configurable) at any commit. it's just TAG for tagged commits.

flit

I've checked its source code promtly and found out that it doesn't support plugins for redefining the way of identifying version.

But it looks like I can easily extend it injecting additional code (from setuptools-scm) here: https://github.com/takluyver/flit/blob/master/flit/common.py#L117

workaround

I could also call setuptools-scm manually from CI and then generate __init__.py with version from template.

While this is okayish solution, there's a drawback: it'll confuse flit -s mode, used during development.

[webknjaz]

workaround 2

I've noticed that flit has a fallback code for actually evaluating module/package to get the version, so theoretically we could put some code there, which would then dynamically calculate current version.

[webknjaz]

I've sent a PR to flit: pypa/flit#199

[webknjaz]

It seems like the maintainer doesn't want this feature. Let's see if I can talk him into adding a hook point at least. The code there is messy however :(

[Mariatta]

Thanks. I suspected that it would be rejected by flit. Thanks for the effort though.

Generating __init__.py seems interesting solution. Why would it confuse flit -s?
Wouldn't we always have .devX during development?

My thinking is:

• __init__.py says v X.Y.Z.dev0 throughout the dev process
• when ready to release, create v X.Y.Z tag
• auto-update __init__.py to v X.Y.Z
• publish to PyPI
• auto-update __init__.py to v X.Y.Z+1.dev0

[webknjaz]

I've prototyped a change to __init__.py, which does not require manual generation of it, but it needs a small improvement, though.

Regarding confusion, it's more about missing manual step of regeneration of this module.

[webknjaz]

I just noticed #285 and maybe it's a bit late but I realized why I didn't feel like the approach of having tags outside of master branch is the right thing to do.
Automation cannot detect those correctly, because it normally uses git describe from the main branch, which doesn't really detect commits, which don't belong there.
Also, people tend to remove branches after they are done with PRs, so the tag would show to commit, which doesn't belong to a named branch, but would be pointed from tag still remaining in Git tree (hopefully).

[webknjaz]

@Mariatta see #287 for a preview of SCM-based versioning. Feel free to test that locally.

You can try following:

git tag current commit
flit install -s
pip list | grep cherr  # to see the version (should come from tag)
add some commit
flit install -s
pip list | grep cherr  # to see the version (should be smth like tag + dev + commit data)

[webknjaz]

So it looks like flit is completely non-customizable. I've taken a look at poetry and it looks very promising. Do you want to try it out?
And there's a few non-rejected FRs for scm versioning support: python-poetry/poetry#185 + python-poetry/poetry#140.

[Mariatta]

Thanks for investigating. I want to stick with flit for now.

[webknjaz]

So what about other workarounds? We could generate __init__.py from git version (flit basically does the same for setup.py)

Also, I saw flows, where CI/CD, upon releasing new version, created a new commit on top of master with just a version bump (to new .dev0).

[webknjaz]

You might be interested in how it's done in mypy: https://github.com/python/mypy/blob/master/mypy/version.py

[hugovk]

Releasing is now automated via Trusted Publishing:

• Publish to PyPI using Trusted Publishers #94
• https://github.com/python/cherry-picker/blob/main/RELEASING.md