Wednesday, December 30, 2020

FETCH_PACKAGES, mk.conf and make variable substitution time

So the tech@ thread about make.1 improvements stems from an issue I have
with `make package' in devel/quirks and `FETCH_PACKAGES=-Dsnap' in
mk.conf(5):

$ cat /etc/mk.conf
SUDO = doas
FETCH_PACKAGES = -Dsnap
PORTS_PRIVSEP = Yes

$ cd /usr/ports/devel/quirks
$ make package
===> Looking for quirks-3.504.tgz in $PKG_PATH - found
cp: /usr/ports/packages/amd64/cache/quirks-3.504.tgz: No such file or directory
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2114 '/usr/ports/packages/amd64/no-arch/quirks-3.504.tgz': @cd /p/devel/quirks; g...)
*** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2605 '_internal-package': @case X${_DEPENDS_CACHE} in X) _DEPENDS_CACHE=$(doas -...)
*** Error 2 in /p/devel/quirks (/usr/ports/infrastructure/mk/bsd.port.mk:2584 'package': @lock=quirks-3.504; export _LOCKS_HELD=" quirks-3....)

Actual bsd.port.mk(5) line with failing cp(1) is 2129:

2114 # The real package
2115
2116 ${_pkg_cookie${_S}}:
2117 @${_PBUILD} install -d ${PACKAGE_REPOSITORY_MODE} ${_PKG_REPO${_S}} ${_TMP_REPO}
2118 . if ${FETCH_PACKAGES:L} != "no" && !defined(_TRIED_FETCHING_${_PACKAGE_COOKIE${_S}})
2119 @${_INSTALL_CACHE_REPO}
2120 @cd ${.CURDIR}; gotit=true; for p in ${_pkg${_S}}; do \
2121 if ! ${_PFETCH} ${MAKE} ${_CACHE_REPO}$$p; then \
2122 gotit=false; \
2123 break; \
2124 fi; \
2125 done; \
2126 if $$gotit; then \
2127 for p in ${_pkg${_S}}; do \
2128 s=${_CACHE_REPO}$$p; d=${_PKG_REPO${_S}}$$p; \
2129 ${_PBUILD} ln $$s $$d 2>/dev/null || ${_PBUILD} cp -p $$s $$d; \
2130 done; \
2131 else \
2132 exec ${MAKE} _TRIED_FETCHING_${_PACKAGE_COOKIE${_S}}=Yes _internal-package-only _FETCH_RECURSE_HELPER=No; \
2133 fi
2134 . else
2135 @${_MAKE} ${_PACKAGE_COOKIE_DEPS}

That's because the code ends up with `gotit=true' even though there's
no package, be it fetched or built...
(`make -d c' and `make -d v' plus lots of grepping can help
understanding such scenarios.)


Anyway, quirks is already special, so what about explicitly setting
FETCH_PACKAGES in the port's Makefile:


Index: Makefile
===================================================================
RCS file: /cvs/ports/devel/quirks/Makefile,v
retrieving revision 1.1115
diff -u -p -r1.1115 Makefile
--- Makefile 30 Dec 2020 08:44:38 -0000 1.1115
+++ Makefile 30 Dec 2020 20:18:32 -0000
@@ -23,3 +23,6 @@ do-install:
${INSTALL_DATA} ${FILESDIR}/Quirks/ghc.pm ${PERLDIR}/OpenBSD/Quirks/ghc.pm

.include <bsd.port.mk>
+
+# build from source regardless of mk.conf(5)
+FETCH_PACKAGES = No


This still fails, but more ugly and later in the `${_CACHE_REPO}${_p}'
(bsd.port.mk line 2101) which is called above via line 2121:

$ make package
===> Looking for quirks-3.504.tgz in $PKG_PATH - Can't find No
found
cp: /usr/ports/packages/amd64/cache/quirks-3.504.tgz: No such file or directory
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2114 '/usr/ports/packages/amd64/no-arch/quirks-3.504.tgz': @cd /p/devel/quirks; g...)
*** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2605 '_internal-package': @case X${_DEPENDS_CACHE} in X) _DEPENDS_CACHE=$(doas -...)
*** Error 2 in /p/devel/quirks (/usr/ports/infrastructure/mk/bsd.port.mk:2584 'package': @lock=quirks-3.504; export _LOCKS_HELD=" quirks-3....)

`FETCH_PACKAGES's value ends up as `pkg-name' argument to pkg_add(1)
("Can't find No") because bsd.port.mk(5) fails to handle this case.

This is because line

2118 . if ${FETCH_PACKAGES:L} != "no" && !defined(_TRIED_FETCHING_${_PACKAGE_COOKIE${_S}})

runs with the value of `FETCH_PACKAGES' at parse time, i.e. "-Dsnap"
from /etc/mk.conf, while line

2104 @if ${SETENV} [...] ${PKG_ADD} -I -x -n -q ${_PKG_ADD_FORCE} -r -D installed -D downgrade ${FETCH_PACKAGES} ${@F}; then \

runs with its value at execution time, i.e. "No" from Makefile.

The only way to reliably build a quirks package is by overwriting
variables on the command line as those are set in stone before Makefiles
are pared and cannot be canged by them:

$ make FETCH_PACKAGES=No package
[...]
Create /usr/ports/packages/amd64/no-arch/quirks-3.504.tgz
Creating package quirks-3.504
Link to /usr/ports/packages/amd64/all/quirks-3.504.tgz
Link to /usr/ports/packages/amd64/ftp/quirks-3.504.tgz


Now this raises questions (for me):

- What is the expected hierarchy for Makefile, bsd.port.mk(5) and
mk.conf(5)? I'd expect Makefile to always have precedence and
actually work as such.

- Is this something worth fixing? devel/quirks seems already special
and pretty much the only port that FETCH_PACKAGES does not makes sense
for anyway.

- Is there documentation I missed regarding such details? I cannot find
information on what to (not) define where.

No comments:

Post a Comment