#13 WIP: Drop Python 2
Closed 3 months ago by opohorel. Opened 6 months ago by opohorel.
rpms/ opohorel/mercurial drop_py2_opohorel  into  master

file removed
-38
@@ -1,38 +0,0 @@ 

- From 65b6aac48e167645a1aa75d6a724811fb7e5c775 Mon Sep 17 00:00:00 2001

- From: =?UTF-8?q?Miro=20Hron=C4=8Dok?= <miro@hroncok.cz>

- Date: Tue, 26 Nov 2019 11:10:12 +0100

- Subject: [PATCH] Create hg2/hg3 bin/scripts to keep Py2 & Py3 mercurial

-  co-installable

- 

- This can be removed completely once the mercurial for Python 2 is removed.

- 

- Co-Authored-By: Petr Stodulka <pstodulk@redhat.com>

- ---

-  setup.py | 4 ++--

-  1 file changed, 2 insertions(+), 2 deletions(-)

- 

- diff --git a/setup.py b/setup.py

- index 932608b..8663e01 100644

- --- a/setup.py

- +++ b/setup.py

- @@ -196,7 +196,7 @@ def write_if_changed(path, content):

-              fh.write(content)

-  

-  

- -scripts = ['hg']

- +scripts = ['hg' + str(sys.version_info[0])]

-  if os.name == 'nt':

-      # We remove hg.bat if we are able to build hg.exe.

-      scripts.append('contrib/win32/hg.bat')

- @@ -317,7 +317,7 @@ def findhg():

-      # and disable localization for the same reasons.

-      hgenv['HGPLAIN'] = '1'

-      hgenv['LANGUAGE'] = 'C'

- -    hgcmd = ['hg']

- +    hgcmd = ['hg' + str(sys.version_info[0])]

-      # Run a simple "hg log" command just to see if using hg from the user's

-      # path works and can successfully interact with this repository.  Windows

-      # gives precedence to hg.exe in the current directory, so fall back to the

- -- 

- 2.23.0

- 

file modified
+37 -68
@@ -1,7 +1,7 @@ 

  Summary: Mercurial -- a distributed SCM

  Name: mercurial

  Version: 5.6.1

- Release: 4%{?dist}

+ Release: 5%{?dist}

  

  # Release: 1.rc1%%{?dist}

  
@@ -12,37 +12,37 @@ 

  URL: https://mercurial-scm.org/

  Source0: https://www.mercurial-scm.org/release/%{name}-%{upstreamversion}.tar.gz

  Source1: mercurial-site-start.el

- Patch2: 0001-setup-hg3.patch

- BuildRequires: python3-devel bash-completion

- BuildRequires: emacs-nox emacs-el pkgconfig gettext python3-docutils

+ BuildRequires: bash-completion

+ BuildRequires: emacs-el

+ BuildRequires: emacs-nox

  BuildRequires: gcc

+ BuildRequires: gettext

+ BuildRequires: pkgconfig

+ BuildRequires: python3-devel

+ BuildRequires: python3-docutils

(If we should keep one dependency on each line and sort the entries, we should do it consistently in the spec.)

+ #Requires(pre): %%{_sbindir}/update-alternatives

Why have this line?

+ 

+ Requires: python3

+ Requires: emacs-filesystem

+ Requires: %{name}-lang = %{version}-%{release}

  

- # Priorities for the alternatives system

- %global py3_priority 30

- 

- %description

- Mercurial is a fast, lightweight source control management system designed

- for efficient handling of very large distributed projects.

- 

- Quick start: http://www.selenic.com/mercurial/wiki/index.cgi/QuickStart

- Tutorial: http://www.selenic.com/mercurial/wiki/index.cgi/Tutorial

- Extensions: http://www.selenic.com/mercurial/wiki/index.cgi/CategoryExtension

- 

- 

- %package py3

- Summary: Mercurial -- a distributed SCM (Python 3 version)

+ # Provides and obsoletes related to previous releases of Fedora where we

+ # delivered mercurial for Py2 & Py3 in parallel. Can be reduced in F36+

  Provides: hg3 = %{version}-%{release}

  Provides: hg = %{version}-%{release}

  Provides: %{name} = %{version}-%{release}

This line also seems pointless now.

- Provides: %{name}%{?_isa} = %{version}-%{release}

- Requires: python3 emacs-filesystem

- Requires: %{name}-lang = %{version}-%{release}

- Obsoletes: emacs-mercurial <= 3.4.1, emacs-mercurial-el <= 3.4.1

+ Provides: %{name}-py3 = %{version}-%{release}

+ Provides: %{name}-py3%{?_isa} = %{version}-%{release}

What is the story behind providing the isa as well? Is that mandated by the guidelines somewhere?

+ 

  Obsoletes: %{name}-py2 < %{version}-%{release}

- Requires(post): %{_sbindir}/update-alternatives

- Requires(preun): %{_sbindir}/update-alternatives

+ Obsoletes: %{name}-py3 < %{version}-%{release}

+ Obsoletes: hg2

+ Obsoletes: hg3

These were never packages, so why obsolete them?

+ Obsoletes: hg-ssh2

+ Obsoletes: hg-ssh3

Why obsolete these? They were never provided.

Also, for fedpkg local:
warning: line 39: It's not recommended to have unversioned Obsoletes: Obsoletes: hg2
warning: line 40: It's not recommended to have unversioned Obsoletes: Obsoletes: hg3
warning: line 41: It's not recommended to have unversioned Obsoletes: Obsoletes: hg-ssh2
warning: line 42: It's not recommended to have unversioned Obsoletes: Obsoletes: hg-ssh3

+ 

  

- %description py3

+ %description

  Mercurial is a fast, lightweight source control management system designed

  for efficient handling of very large distributed projects.

  
@@ -57,8 +57,6 @@ 

  Summary:    Hgk interface for mercurial

  Requires:   hg = %{version}-%{release}, tk

  

- Requires:   hg3 = %{version}-%{release}, tk

- 

  %description hgk

  A Mercurial extension for displaying the change history graphically

  using Tcl/Tk.  Displays branches and merges in an easily
@@ -74,9 +72,6 @@ 

  Summary:    A fast client for Mercurial command server running on Unix. It saves time of slow Python startup.

  Requires:   hg = %{version}-%{release}

  

- # Let's default to Python 2, but allow some experiments

- Suggests:   hg3

- 

  %description chg

  chg is a C wrapper for the hg command. Typically, when you type hg, a new

  Python process is created, Mercurial is loaded, and your requested command runs
@@ -95,13 +90,10 @@ 

  

  %prep

  %setup -q -n %{name}-%{upstreamversion}

- 

- %patch2 -p1 -b .create_hg3

+ # NOTE: Use PYTHON envar instead of pathing Makefiles

+ # sed -ri 's|python\b|python2|' %{_builddir}/%{name}-%{version}/Makefile %{_builddir}/%{name}-%{version}/doc/Makefile

  

  %build

- # copy hg to hg2/hg3 to be able to create /usr/bin/hg(2|3) script

- cp -a hg hg3

- rm hg

  PYTHON=%{python3} make all

  

  # chg will invoke the 'hg' command - no direct Python dependency
@@ -110,8 +102,6 @@ 

  popd

  

  %install

- # NOTE: this is nasty, but if we want to have parallel Py2/Py3 mercurial..

- # ignoring now stuff related to *_completion, extensions

  %{python3} setup.py install -O1 --root %{buildroot} --prefix %{_prefix} --record=%{name}.files

  make install-doc DESTDIR=%{buildroot} MANDIR=%{_mandir}

  
@@ -147,7 +137,7 @@ 

  popd

  

  pushd contrib/chg

- make install DESTDIR=%{buildroot} PREFIX=%{_usr} MANDIR=%{_mandir}/man1

+ %make_install DESTDIR=%{buildroot} PREFIX=%{_usr} MANDIR=%{_mandir}/man1

AFAICS, the whole point with %make_install was to avoid DESTDIR?

  popd

  

  
@@ -179,34 +169,10 @@ 

  

  grep -v locale %{name}-base.files > %{name}-base-filtered.files

  

+ pathfix.py -pni "%{python3}" %{buildroot}%{_bindir}/hg-ssh

  

- # Move hg-ssh aside

- cp -p %{buildroot}%{_bindir}/hg-ssh %{buildroot}%{_bindir}/hg-ssh3

- rm %{buildroot}%{_bindir}/hg-ssh

- pathfix.py -pni "%{python3}" %{buildroot}%{_bindir}/hg-ssh3

  

- 

- # Touch the base executables for alternatives

- touch %{buildroot}%{_bindir}/hg

- touch %{buildroot}%{_bindir}/hg-ssh

- 

- %post py3

- for fname in %{_bindir}/{hg,hg-ssh}; do

-   if [ ! -L "$fname" ]; then

-     rm -f "$fname"

-   fi

- done

- %{_sbindir}/update-alternatives --install %{_bindir}/hg \

-   hg %{_bindir}/hg3 %{py3_priority} \

-   --slave %{_bindir}/hg-ssh hg-ssh %{_bindir}/hg-ssh3 || :

- 

- %preun py3

- if [ $1 -eq 0 ]; then

-   %{_sbindir}/update-alternatives --remove \

-     hg %{_bindir}/hg3 || :

- fi

- 

- %files py3 -f %{name}-base-filtered.files

+ %files -f %{name}-base-filtered.files

  %doc CONTRIBUTORS COPYING doc/README doc/hg*.txt doc/hg*.html *.cgi contrib/*.fcgi contrib/*.wsgi

  %doc %attr(644,root,root) %{_mandir}/man?/hg*.gz

  %doc %attr(644,root,root) contrib/*.svg
@@ -222,10 +188,8 @@ 

  %{python3_sitearch}/hgdemandimport/

  %{_emacs_sitelispdir}/mercurial

  %{_emacs_sitestartdir}/*.el

- %ghost %{_bindir}/hg

- %ghost %{_bindir}/hg-ssh

- %{_bindir}/hg3

- %{_bindir}/hg-ssh3

People might have started to explicit use hg3 or hg-ssh3 - I thus think we have to keep providing these in a transition period.

+ %{_bindir}/hg

+ %{_bindir}/hg-ssh

  

  %config(noreplace) %{_sysconfdir}/mercurial/hgrc.d/certs.rc

  
@@ -241,6 +205,7 @@ 

  

  

  #%%check

+ # TODO: start to use the check section again

  # This will now fail everytime. Mercurial is not ported properly for Python3

  # and current split of mercurial for Py2 and Py3 brings additional problems

  # for extensions then pure Python3 packaging.
@@ -248,6 +213,10 @@ 

  

  

  %changelog

+ * Thu Dec 10 2020 Ondrej Pohorelsky <opohorel@redhat.com> - 5.6.1-5

+ - Drop Python 2 support

+ - Resolves: #1737931

+ 

  * Tue Dec  8 17:14:14 CET 2020 Mads Kiilerich <mads@kiilerich.com> - 5.6.1-4

  - Change mercurial-hgk to use py3

  - Use py3 for locales - py2 is going away

As pstodulk doesn't have time to work on dropping Python 2 from Mercurial, I've started working on it.

It is still work in progress. As I'm not that experienced with packaging and Mercurial in general, this might take some time.
Right now this PR consists of pstodulk changes[0] and suggested change by mhroncok. I'm going to continue working on it in this PR.
Luckily this package right now builds successfully[1], but there still is a lot work to do.
I'm not going to rebase to the newest Mercurial 5.5, before everything works on current version(5.4).

[0]https://src.fedoraproject.org/rpms/mercurial/pull-request/11
[1]https://koji.fedoraproject.org/koji/taskinfo?taskID=53434725

I can't see preserving update path and making sure there are no leftovers. E.g. the changelog states "hg[23] and hg-ssh[23] utilities are dropped" but I don't see them obsoleted.

1 new commit added

  • Add obsoletes to hg[23] and hg-ssh[23]
6 months ago

1 new commit added

  • Rebase to 5.6
5 months ago

So I rebased to version 5.6. This new version introduced quite a few Python compatibility fixes, which makes this work IMO easier. I also added %make_install macro that tstellar suggested in other PR.

Right now it looks like Mercurial builds without check section, upgrades both py2 and py3 versions, hgk seems to work and chg as well. Of course I haven't tested it properly yet as I don't use Mercurial regularly.

After enabling check section there are 2 failing tests:

Failed test-hghave.t: output changed
Failed test-merge-combination.t: timed out

And 75 skipped tests.
I'm going to investigate what is the cause and try to fix these.

1 new commit added

  • own hgdemandimport and hgext3rd directories
5 months ago

@opohorel Please rebase the branch against the master. So far, I played a little bit around and git-remote-hg seems to be working with this version of mercurial well.

rebased onto 867c705

4 months ago

I cleaned up the spec file and I think we are merge ready now.

@kiilerix
Thg is now updated to work with Mercurial 5.6. I don't see any blocker to drop Python 2 support now. Can we finally get rid of it?

So much is happening in this PR, making it hard to review the delicate parts.

I landed some of the trivial parts - including the trivial removal of the py2 package. The less trivial parts of your PR is the removal of support for split versions and renaming back to the old names. Can you rebase this so we can see mere clearly what is left?

rebased onto d51dd22

4 months ago

I rebased this PR to master

Why have this line?

Why obsolete these? They were never provided.

This line also seems pointless now.

What is the story behind providing the isa as well? Is that mandated by the guidelines somewhere?

AFAICS, the whole point with %make_install was to avoid DESTDIR?

These were never packages, so why obsolete them?

People might have started to explicit use hg3 or hg-ssh3 - I thus think we have to keep providing these in a transition period.

(If we should keep one dependency on each line and sort the entries, we should do it consistently in the spec.)

Also, for fedpkg local:
warning: line 39: It's not recommended to have unversioned Obsoletes: Obsoletes: hg2
warning: line 40: It's not recommended to have unversioned Obsoletes: Obsoletes: hg3
warning: line 41: It's not recommended to have unversioned Obsoletes: Obsoletes: hg-ssh2
warning: line 42: It's not recommended to have unversioned Obsoletes: Obsoletes: hg-ssh3

The upgrade path doesn't seem as clean as it should be:

$ rpm -qa | grep mercurial
mercurial-lang-5.4-4.fc33.x86_64
mercurial-py2-5.4-4.fc33.x86_64
mercurial-py3-5.4-4.fc33.x86_64

$ ll
-rw-r--r--. 1 mk mk 4236421 Dec 10 20:55 mercurial-5.6.1-5.fc33.x86_64.rpm
-rw-r--r--. 1 mk mk   26132 Dec 10 20:55 mercurial-chg-5.6.1-5.fc33.x86_64.rpm
-rw-r--r--. 1 mk mk   39905 Dec 10 20:55 mercurial-hgk-5.6.1-5.fc33.x86_64.rpm
-rw-r--r--. 1 mk mk 1034092 Dec 10 20:55 mercurial-lang-5.6.1-5.fc33.x86_64.rpm

$ sudo rpm -Uhv *
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:mercurial-lang-5.6.1-5.fc33      ################################# [ 14%]
   2:mercurial-5.6.1-5.fc33           ################################# [ 29%]
   3:mercurial-chg-5.6.1-5.fc33       ################################# [ 43%]
   4:mercurial-hgk-5.6.1-5.fc33       ################################# [ 57%]
failed to link /usr/bin/hg -> /etc/alternatives/hg: /usr/bin/hg exists and it is not a symlink
failed to link /usr/bin/hg-ssh -> /etc/alternatives/hg-ssh: /usr/bin/hg-ssh exists and it is not a symlink
Cleaning up / removing...
   5:mercurial-py3-5.4-4.fc33         ################################# [ 71%]
   6:mercurial-py2-5.4-4.fc33         ################################# [ 86%]
   7:mercurial-lang-5.4-4.fc33        ################################# [100%]
$ 

Thanks for rebasing and making the remaining changes more reviewable.

Note: I find it much easier if changesets only do one thing and do it cleanly.

The mercurial-py3 should be set in Provides and Obsoletes. Currently just obsoletes is set in upstream. Maybe it's worth to close this PR as everythin (or at least majority) is already pushed in the rawhide @opohorel @kiilerix ?

Yes, I planned to close this PR this week, because pretty much everything important is already in rawhide.

I'm closing it now, thank you for solving the issue!

Pull-Request has been closed by opohorel

3 months ago