#3 Switch to OpenJDK 11 as default JDK
Merged 7 months ago by mizdebsk. Opened 10 months ago by mizdebsk.
rpms/ mbi/javapackages-tools default-jdk11-change  into  master

file modified
+8 -2
@@ -9,7 +9,7 @@ 

  %bcond_without asciidoc

  %bcond_with gradle

  %bcond_without tests

- %bcond_with xmvn_javadoc

+ %bcond_without xmvn_javadoc

  

  %if 0%{?fedora} || 0%{?rhel} > 7

  %global python_prefix python3
@@ -25,7 +25,7 @@ 

  

  Name:           javapackages-tools

  Version:        5.3.0

- Release:        10%{?dist}

+ Release:        11%{?dist}

  

  Summary:        Macros and scripts for Java packaging support

  
@@ -204,6 +204,12 @@ 

  %license LICENSE

  

  %changelog

+ * Wed Jun 10 2020 Mikolaj Izdebski <mizdebsk@redhat.com> - 5.3.0-11

+ - Use XMvn Javadoc MOJO for generating API docs

+ 

+ * Tue May 26 2020 Miro Hrončok <mhroncok@redhat.com> - 5.3.0-10

+ - Rebuilt for Python 3.9

+ 

  * Tue Apr 28 2020 Mikolaj Izdebski <mizdebsk@redhat.com> - 5.3.0-10

  - Switch to OpenJDK 11 as default JDK

  

We are not going to merge this for now as we are currently investigating what will need to be fixed in packages. This is in prep for:
https://fedoraproject.org/wiki/Changes/Java11

Thanks @mizdebsk for the PR. I've added a comment and the DO NOT MERGE note.

Another useful change would be e13aca8. Together with related XMvn feature it allows to keep building packages with OpenJDK 8 by changing a single line in spec file, for example: plexus-utils

Can we also switch on xmvn_javadoc? In my testing it resolves almost all issues related to javadoc generation ("cannot find package / symbol" errors with maven-javadoc-plugin").

This cuts down the number of build failures massively ...

I don't mind to switch to using XMvn Javadoc, as long as this is expected by Java SIG.

In the last two weeks, there was no opposition to switching on xmvn-javadoc. Since it fixes multiple problems (fixes almost all javadoc build failures with OpenJDK 11 AND it produces more consistent documentation), I ask that we enable it when switching to java-11-openjdk by default.

1 new commit added

  • Use XMvn Javadoc MOJO for generating API docs
8 months ago

Sorry, I'm just getting back into this. What is the hold up on this?

This PR implements Java 11 change.
javapackages-tools with this patch will be built in a side tag.
I will merge this PR once side tag is created by Release Engineering and change owner ACKs the PR.

Time is here - https://pagure.io/releng/issue/9574 - f33-java11 was created, jdks with necessary changes are already built or building. This needs to go now, mavenis next, and then follow everything else.

rebased onto d9fa1a9

7 months ago

Pull-Request has been merged by mizdebsk

7 months ago

Looking to the changes Fabio is doing, eg:
https://src.fedoraproject.org/rpms/apache-commons-modeler/c/c9764961e7950019c500e59fd63918e6d83a0db2?branch=master

And considering, that during next bump of build JDK (jdk17) somewhere around f36/37, we will have to do it again, because lowest source/target will be 9, I'm heavily suggesting, the control of
source/target is moved to
- jpackage-tools (eg macro with the value)
- and that is directly used by xmvn (eg via the -Dmaven.compiler.source=%{jsource} -Dmaven.compiler.target=%{jtarget}) - with possibility to get overriden

Writing here, due to CCed people,. I had already raised it with few people, but not got answers. Unless strictly forbidden already here, will rise it on java-devel list sooner or later.

To do this now, will save quite a lot of work to Fabio and others. Thoughts?

xmvn used to override the minimum source/target levels automatically, but that feature was removed:

https://github.com/fedora-java/xmvn/commit/c64c35a

Not sure if reintroducing some source/target = min(1.8, source/target) functionality is going to help here for the future, since it only affects packages built with maven (-compiler-plugin), and not anything built by e.g. ant.

Hi! thanx for reply.
As 1/2 of javastack is maven, it will downsize the workload a lot.
Also having %{jtarget}/%{jsource} macros, will allow us to do it once. Instead of -Dmaven.compiler.source=1.8 -Dmaven.compiler.target=1.8 or sed s/javac.target=1.5/ javac.target=1.8

You will use -Dmaven.compiler.source=%{jsource} -Dmaven.compiler.target=%{jtarget} and sed s/javac.target=1.5/ javac.target=%{jtarget}

Thus next time we eill have quite a pece.

That is of course java packaging guidelines change proposal, and would vote it to be optional (except mavne, where it is override-able)

Looking to the issue with dogtag-pki, that woul dnot happen with above applied.

Hmm. There are at least two packages whic ich have to build with jdk8, but runs fine with jdk11. Those packages desperately needs the above suggested macros. Right now, each of them have at least one dependece, which have bytecode of jdk 11 version, thus they can not use it.

@jvanek I'm tracking this issue here:
https://pagure.io/java-maint-sig/issue/7

And the macro support you suggested would not help in all the cases I found.

hmm. Sad to hear that:( Unluckily I fail to see the issue in the proposal. Anyway, you are very experienced java engineer and java packager. Do you have any other suggestion how to prevent it both happening in future?
- humans to edit source/target
- have packages built with wrong target

TBH the majority of the cases I've fixed are ant packages e.g.: https://src.fedoraproject.org/rpms/javacc/c/1f1404aae15c97eb3ce03f18e840d93b1c917998

There is an %ant macro provided by something that could be modified, but almost no package uses it.

What I would really like is a way to properly use "javac --release" to prevent incorrect cross-compilation from causing runtime linkage errors such as I fixed in aqute-bnd: https://src.fedoraproject.org/rpms/aqute-bnd/c/df192255992dcffb7e7c17174d8efbc4a8142385

For maven it can be passed as maven.compiler.release (but ONLY on JDK >= 9) and for ant I am not aware of a way to pass it without editing the build.xml, but with ant you can pass it for any JDK and I think it will be ignored -- I think we'd need to patch ant itself to pull this value from a system property or something

TBH the majority of the cases I've fixed are ant packages e.g.: https://src.fedoraproject.org/rpms/javacc/c/1f1404aae15c97eb3ce03f18e840d93b1c917998

Does that work always, or only when the build target is named build?

Also from what I've seen, this doesn't work at all if the property is already set in build.xml, it looks like command line arguments do not override this in ant :frowning:

There is an %ant macro provided by something that could be modified, but almost no package uses it.
What I would really like is a way to properly use "javac --release" to prevent incorrect cross-compilation from causing runtime linkage errors such as I fixed in aqute-bnd: https://src.fedoraproject.org/rpms/aqute-bnd/c/df192255992dcffb7e7c17174d8efbc4a8142385

Yeah, this is what we discussed in the "EasyFix" ticket - once we've moved to JDK >= 9 (which is now?), we can start to set the --release flag in maven-compiler-plugin.

For maven it can be passed as maven.compiler.release (but ONLY on JDK >= 9) and for ant I am not aware of a way to pass it without editing the build.xml, but with ant you can pass it for any JDK and I think it will be ignored -- I think we'd need to patch ant itself to pull this value from a system property or something

Yeah but maven has the same problem as ant - the command line arguments for -Dmaven.compiler.source/target/release are only honoured if they aren't already set in the pom.xml file configuration for maven-compiler-plugin. So in most cases I had to %pom_xpath_remove the old configuration for the new one passed on the command line to take effect :disappointed:

That's why I said: It's not so simple :(

I mean, we could come up with a magic script that detects broken build configurations, removes them from the upstream build files, and injects the correct values on the command line. But I don't think this is feasible for ant (build.xml files are way too diverse for this to work), but maybe it is doable for maven (at least the maven-compiler-plugin configuration is always at the same XPATH).

My initial idea was much more simple, it was about providing simply macros with expected versions (1.8 this time) Then it is on packager to use them or not. Point was to not do it again in two years.
I understand the diversity of build files quite well, and thats why there was no full automation. As you mention, it is doable for maven, and already that woudl save half ot he work, and you understand it mch better then me.
For all other cases, manual touch will be needed. But being it sed, patch or call to xpath, there everywhere can be used amcro with the real system value, instead of hardcoded numbers, which we will need to fix every release.

TBH the majority of the cases I've fixed are ant packages e.g.: https://src.fedoraproject.org/rpms/javacc/c/1f1404aae15c97eb3ce03f18e840d93b1c917998

Does that work always, or only when the build target is named build?
Also from what I've seen, this doesn't work at all if the property is already set in build.xml, it looks like command line arguments do not override this in ant 😦

It's magic system properties that ant checks if the source="" and target="" attributes are not set at all in the <javac> task -- but this is the case that produces the Java 11 bytecode. If the build.xml does source="${some.other.property}" then obviously you need to set -Dsome.other.property instead and invariably it's already set to something lower than 11 in the build.xml.

yes, and that -Dsome.other.property needs a value, and better one then hardcoded one.

FWIW, defining %{jsource} and %{jtarget} seems like a good idea to me :-)

Edit: Maybe also %{jrelease}

I prefer the following approach:

  • packages that need to work with OpenJDK 8 (or both 8 and 11) are built with OpenJDK 8; cross-compiling with OpenJDK 11 and --release 8 does not solve all the issues
  • XMvn has a new feature - toolchain manager that allows to use OpenJDK 8 during package build; with the toolchain active XMvn itself still runs with OpenJDK 11, but uses all the tools (javac, javadoc etc.) from OpenJDK 8; unit tests are executed with OpenJDK 8 too
  • javapackages-tools adds maven-local-openjdk8 subpackage that allows for easy use of XMvn toolchain; you simply replace BuildRequires on maven-local with maven-local-openjdk8
  • packages that use Maven for building and need to be build with OpenJDK 8 need very minimal changes like this
  • packages that use Ant can use %ant macro that configures JAVA_HOME for Ant to use
  • packages that don't use any build system but invoke javac/javadoc etc. directly from spec file can use %java/%javac/%javadoc/%jar etc. macros to select OpenJDK 8
  • packages that invoke javac/javadoc/etc from custom build scripts or tools (eg. Makefile) can export JAVA_HOME=%java_home or even export PATH=$JAVA_HOME/bin:$PATH

That is interesting approach, however do not solve the issue with bytecde version (which I'm not sure if is really issue ore temporary complication).
I'm afraid, that making build by jdk8 to easy, will sound like a proffered way to do so, Which is not. Then we happily could just bump jdk11 provides, and left javastack to rot.
According to current patching, mostly decathorpe does, the build with set source/target currently works,and if it is just temporary complication then it will do its job and will fade into history.

Afaik all what is necessary to solve, is to not force javapackagers to bump sourcve/target next time we bump jdk. It will be around f36 to jdk17. That bump should work out of the box... all except source/target. Which will be minimal on jdk11, and will thus kill all jdk8 remaining parts

JAVA_HOME is workaround which works already now. And is moreover necessary.

Both you and decatrhrope are much better java packagers, so the decision is actually on you, I can just be tweeting arround.

I mean, yeah, we need some solution for this.

Ideally, we would fix upstream projects. But since a lot of those are either FUBAR or AWOL that's going to be difficult.

That is interesting approach, however do not solve the issue with bytecde version (which I'm not sure if is really issue ore temporary complication).

I'm not sure what you mean here. If you build something with JDK 8 then it will have bytecode compatible with both JDK 8 and 11.

I'm afraid, that making build by jdk8 to easy, will sound like a proffered way to do so, Which is not.

I would argue with that. If compatibility with JDK 8 is a goal then the package should be built with JDK 8. When cross-compiling with JDK 11 the build may succeed, but produce bytecode that can't run with JDK 8 - that happened before for Maven, see eg. private bug 1782282. Otherwise, when JDK 8 compatibility is non-goal, you can build the package with default JDK 11 without explicit source/target/release setting and it should work with JDK 11.