PR#2: Read lexer from .wasm files instead of base64-encoded .js - rpms/nodejs-cjs-module-lexer

rpms / nodejs-cjs-module-lexer

#2 Read lexer from .wasm files instead of base64-encoded .js

Opened 4 months ago by mochaa. Modified 15 days ago

rpms/ mochaa/nodejs-cjs-module-lexer rawhide into rawhide

Read lexer from .wasm files instead of base64-encoded .js

Zephyr Lykos • 17 days ago

be09892

.gitignore

file modified

+5 -3

		`@@ -1,3 +1,5 @@`
		`- /cjs-module-lexer-1.2.3.tar.gz`
		`- /cjs-module-lexer-1.2.3-nm-dev.tgz`
		`- /cjs-module-lexer-1.2.3-bundled-licenses.txt`
		`+ /cjs-module-lexer-*.tar.gz`
		`+ /cjs-module-lexer-*.tgz`
		`+ /cjs-module-lexer-*-bundled-licenses.txt`
		`+`
		`+ /94.patch`

0001-parametrize-wasm-compilation-process.patch

file removed

-89

		`@@ -1,89 +0,0 @@`
		`- From 182e2768a4029365144d287257be0ca606c0ab40 Mon Sep 17 00:00:00 2001`
		`- From: =?UTF-8?q?Jan=20Stan=C4=9Bk?= <jstanek@redhat.com>`
		`- Date: Mon, 26 Jun 2023 09:05:38 +0200`
		`- Subject: [PATCH] Parametrize WASM compilation process (#87)`
		`- MIME-Version: 1.0`
		`- Content-Type: text/plain; charset=UTF-8`
		`- Content-Transfer-Encoding: 8bit`
		`-`
		`- * Makefile: extract compilation flags to variables`
		`-`
		`- Previously, the WASM compilation step was hard to adjust for different`
		`- systems, as it required re-writing the actual rule.`
		`-`
		`- This change extracts all the flags into make variables,`
		`- and separates them into system- and project-specific ones.`
		`-`
		`- The system-specific ones are expected to change from machine to machine,`
		`- and now can be easily overridden:`
		`-`
		`- $ make WASM_CC=clang WASM_CFLAGS=--target=wasm32-wasi …`
		`-`
		`- The project-specific flags are also extracted to variables,`
		`- although at this point this is mainly for consistency sake.`
		`-`
		`- * Makefile: parametrize all binaries`
		`-`
		`- Similarly to previous change,`
		`- this allows for all binary paths to be easily adjusted.`
		`-`
		`- * Makefile: polish`
		`-`
		`- Tiny leftover nitpicks:`
		`-`
		- - Mark `optimize` and `clean` targets as phony
		- - Use separate target for `lib/` directory creation
		- - Use make's built-in `$(RM)` command
		`- ---`
		`- Makefile \| 39 ++++++++++++++++++++++++++++-----------`
		`- 1 file changed, 28 insertions(+), 11 deletions(-)`
		`-`
		`- diff --git a/Makefile b/Makefile`
		`- index 3098484..961b336 100755`
		`- --- a/Makefile`
		`- +++ b/Makefile`
		`- @@ -1,16 +1,33 @@`
		`- -lslib/lexer.wat: lib/lexer.wasm`
		`- - ../wabt/bin/wasm2wat lib/lexer.wasm -o lib/lexer.wat`
		`- +# These flags depend on the system and may be overridden`
		`- +WASM_CC := ../wasi-sdk-12.0/bin/clang`
		`- +WASM_CFLAGS := --sysroot=../wasi-sdk-12.0/share/wasi-sysroot`
		`- +WASM_LDFLAGS := -nostartfiles`
		`-`
		`- -lib/lexer.wasm: include-wasm/cjs-module-lexer.h src/lexer.c`
		`- - @mkdir -p lib`
		`- - ../wasi-sdk-12.0/bin/clang src/lexer.c -I include-wasm --sysroot=../wasi-sdk-12.0/share/wasi-sysroot -o lib/lexer.wasm -nostartfiles \`
		`- - -Wl,-z,stack-size=13312,--no-entry,--compress-relocations,--strip-all,--export=__heap_base,\`
		`- - --export=parseCJS,--export=sa,--export=e,--export=re,--export=es,--export=ee,--export=rre,--export=ree,--export=res,--export=ru,--export=us,--export=ue \`
		`- - -Wno-logical-op-parentheses -Wno-parentheses \`
		`- - -Oz`
		`- +WASM2WAT := ../wabt/bin/wasm2wat`
		`- +WASM_OPT := ../binaryen/bin/wasm-opt`
		`- +`
		`- +# These are project-specific and are expected to be kept intact`
		`- +WASM_EXTRA_CFLAGS := -I include-wasm/ -Wno-logical-op-parentheses -Wno-parentheses -Oz`
		`- +WASM_EXTRA_LDFLAGS := -Wl,-z,stack-size=13312,--no-entry,--compress-relocations,--strip-all`
		`- +WASM_EXTRA_LDFLAGS += -Wl,--export=__heap_base,--export=parseCJS,--export=sa`
		`- +WASM_EXTRA_LDFLAGS += -Wl,--export=e,--export=re,--export=es,--export=ee`
		`- +WASM_EXTRA_LDFLAGS += -Wl,--export=rre,--export=ree,--export=res,--export=ru,--export=us,--export=ue`
		`- +`
		`- +.PHONY: optimize clean`
		`- +`
		`- +lib/lexer.wat: lib/lexer.wasm`
		`- + $(WASM2WAT) lib/lexer.wasm -o lib/lexer.wat`
		`- +`
		`- +lib/lexer.wasm: include-wasm/cjs-module-lexer.h src/lexer.c \| lib/`
		`- + $(WASM_CC) $(WASM_CFLAGS) $(WASM_EXTRA_CFLAGS) \`
		`- + src/lexer.c -o lib/lexer.wasm \`
		`- + $(WASM_LDFLAGS) $(WASM_EXTRA_LDFLAGS)`
		`- +`
		`- +lib/:`
		`- + @mkdir -p $@`
		`-`
		`- optimize: lib/lexer.wasm`
		`- - ../binaryen/bin/wasm-opt -Oz lib/lexer.wasm -o lib/lexer.wasm`
		`- + $(WASM_OPT) -Oz lib/lexer.wasm -o lib/lexer.wasm`
		`-`
		`- clean:`
		`- - rm lib/*`
		`- + $(RM) lib/*`

nodejs-cjs-module-lexer.spec

file modified

+8 -7

		`@@ -3,7 +3,7 @@`

		`Name: nodejs-%{npm_name}`
		`Summary: A very fast lexer used to detect the named exports of a CommonJS module`
		`- Version: 1.2.3`
		`+ Version: 1.3.1`
		`Release: %autorelease`

		`License: MIT`
		`@@ -14,8 +14,7 @@`
		`Source2: %{npm_name}-%{version}-nm-dev.tgz`
		`Source3: %{npm_name}-%{version}-bundled-licenses.txt`

		`- # Adapt Makefile for easier rebuilds`
		`- Patch: 0001-parametrize-wasm-compilation-process.patch`
		`+ Patch0: https://github.com/nodejs/cjs-module-lexer/pull/94.patch`

		`# Binary files in this package are aimed at the wasm32-wasi "architecture".`
		`%global _binaries_in_noarch_packages_terminate_build 0`
		`@@ -29,7 +28,7 @@`
		`ExclusiveArch: %{ix86} x86_64 aarch64 ppc64le riscv64 noarch`
		`%endif`

		`- BuildRequires: clang lld make wasi-libc-devel`
		`+ BuildRequires: clang lld make wasi-libc-devel binaryen`
		`BuildRequires: nodejs-devel npm`
		`# for autosetup -S git_am`
		`BuildRequires: git-core`
		`@@ -47,14 +46,16 @@`
		`%build`
		`rm -rf %{prebuilt_blobs}`
		`tar -xzf %{S:2} && ln -rsf node_modules_dev node_modules`
		`+ mkdir dist`

		`%make_build -j1 \`
		`- WASM_CC=clang \`
		`+ WASM_CC=clang WASM_OPT=wasm-opt \`
		`WASM_CFLAGS='--target=wasm32-wasi --sysroot=/usr/wasm32-wasi' \`
		`WASM_LDFLAGS='-nostartfiles -nodefaultlibs -lc' \`
		`- clean lib/lexer.wasm`
		`+ clean optimize`

		`- npm --offline run build`
		`+ EXTERNAL_PATH="%{nodejs_sitelib}/%{npm_name}" npm --offline run build`
		`+ npm --offline pkg set 'files[]=lib/lexer.wasm'`
		`npm --offline pack`

		`%install`

sources

file modified

+5 -3

		`@@ -1,3 +1,5 @@`
		`- SHA512 (cjs-module-lexer-1.2.3.tar.gz) = dab77f606c0b0c12abf9e91994cd56b3492e45ecb6e615236abb948fa2a66146d25776bb895b4b25a1b3cb15168eb073b07ff405da47d3b2a1e62428171b4f76`
		`- SHA512 (cjs-module-lexer-1.2.3-nm-dev.tgz) = 44d91bfe4e782a47d0506ec5d0bf12877083b263222535019efd0e0d50614d6688edde17571025ddd9c8f1c2a428974bfb59ee9d9c97d1b7e20150785cfab3de`
		`- SHA512 (cjs-module-lexer-1.2.3-bundled-licenses.txt) = 3e591adb16e09b1c6fb75eb97a6ac51cd06e0aeae2bd5950a15902346e2c9626cb50203bfd0e15c7e8c98af8f24b81c1ce6617c359e7c0e7d0a6366457c844ba`
		`+ SHA512 (cjs-module-lexer-1.3.1.tar.gz) = cfd082aeb096bfa1ca9213571c072b6e0a13ba8ffdff3e12def6ea3f21a9ab6fcd4ad1ce342dff0a2a89f999b3bd1609e37aa11139b3deef39341b28cc856f4f`
		`+ SHA512 (cjs-module-lexer-1.3.1-nm-dev.tgz) = c6a3a780484636a9f93d04ac554ef5c5207f1c6cf896aa61662bcd705890ef73a3e910ed1c8e532b3c085bb6dbece6974507404fffabe167948a88f156713536`
		`+ SHA512 (cjs-module-lexer-1.3.1-nm-prod.tgz) = 1fea1128fdb65a4ba2fcd41dce0ca7b80eab224adf72508216f16790e0e0f8b9647b31662b0761b6c87e3a718e812577b9ed79f06dca1aa00c12d75cc639353e`
		`+ SHA512 (94.patch) = b9d6898dedf1d1124425c47107aa766d9d49037032031bfc57eb42b583e51f6aca801512ee966e8fdbad15bd931b63f8d9dd2dada5cd2accc954d03c69df7666`
		`+ SHA512 (cjs-module-lexer-1.3.1-bundled-licenses.txt) = 3e591adb16e09b1c6fb75eb97a6ac51cd06e0aeae2bd5950a15902346e2c9626cb50203bfd0e15c7e8c98af8f24b81c1ce6617c359e7c0e7d0a6366457c844ba`

mochaa commented 4 months ago

Align with https://src.fedoraproject.org/rpms/nodejs-undici/pull-request/3

rebased onto 58c36f1

4 months ago

rebased onto 3124970

4 months ago

jstanek commented 4 months ago

Similarly to the undici PR: Thanks for the proposal, but try to push the idea to upstream first and see what they think about it; I do not want to maintain it separately.

rebased onto 40d75fe

4 months ago

mochaa commented 4 months ago

Submitted to upstream: https://github.com/nodejs/cjs-module-lexer/pull/91

rebased onto 47a5991

a month ago

mochaa commented a month ago

Upstream merged this, but a release date is unclear.

jstanek commented 23 days ago

I saw, thanks for it! I tried to backport the final patch(es) to the current version, but messing with the log files + regenerating the dependencies after patching is a bit more complicated than I would like.

Are you fine with waiting for the release, whenever it might come?

jstanek commented 23 days ago

s/log/lock/… apparently some key combos are just hardwired to my fingers.

mochaa commented 17 days ago

The release came out as 1.3.0, I will update this PR accordingly

mochaa commented 17 days ago

~~waiting for https://github.com/nodejs/cjs-module-lexer/pull/93 to land~~
~~waiting for https://github.com/nodejs/cjs-module-lexer/pull/94 to land~~

Edited 15 days ago by mochaa

rebased onto be09892

17 days ago

mochaa commented 15 days ago

@jstanek I guess we are good to go (for now?)

jstanek commented 15 days ago

Well, kind of – I'm not thrilled about the changes being reverted upstream immediately after the version release. It does not fill me with optimism – if this breaks, I'll have no way of fixing this in the future without your help.

I'm willing to merge this even as proof-of-concept, provided you plan to follow up with upstream and getting "something that makes sense" (in their words) accepted and maintained there.

I hate to be that guy, but as stated above, I would have a hard time maintaining this myself if you decide enough was enough; that would also mean I would have to revert this in the future anyway.

mochaa commented 15 days ago

There is https://github.com/nodejs/cjs-module-lexer/pull/98 now - let's see wait for upstream's feedback.

mochaa commented 15 days ago

There is https://github.com/nodejs/cjs-module-lexer/pull/98 now - let's wait for upstream's feedback.