devel/cargo: Support fetching crates as GitHub archive tarballs

Crates can come from various sources, we only support the registry which
is fine for most ports, but especially during development/porting it is
required to overwrite crates with patched versions.

Such crates often live as forks on GitHub and can be downloaded as usual
so let's support this.

I need this for an upcoming port (which is still in early development)
using feature branches of certain crates -- I followed the same process
in order to add OpenBSD support.

Neither existing nor new users of the cargo module have to do anything;
Cargo.lock `source' lines are parsed as before except that more types
are recognised.

Thanks to sthen for pointing at www/nginx wrt. MASTER_SITES handling.

Feedback? OK?

Index: cargo.port.mk
===================================================================
RCS file: /cvs/ports/devel/cargo/cargo.port.mk,v
retrieving revision 1.24
diff -u -p -r1.24 cargo.port.mk
--- cargo.port.mk 11 Sep 2021 07:13:13 -0000 1.24
+++ cargo.port.mk 10 Oct 2021 14:50:21 -0000
@@ -42,10 +42,14 @@ _MODCARGO_DIST_SUBDIR = ${MODCARGO_DIST_
.endif

# Use MASTER_SITES9 to grab crates by default.
-# Could be changed by setting MODCARGO_MASTER_SITESN.
MODCARGO_MASTER_SITESN ?= 9
MASTER_SITES${MODCARGO_MASTER_SITESN} ?= ${MASTER_SITES_CRATESIO}

+# Use MASTER_SITES0 to grab crates as GitHub tarballs by default.
+MODCARGO_MASTER_SITES_GITN ?= 8
+# XXX this assumes that all git crates come from the same site.
+MASTER_SITES${MODCARGO_MASTER_SITES_GITN} ?= https://github.com/
+
# per crates options
MODCARGO_CRATES_SQLITE3_BUNDLED ?= No

@@ -54,6 +58,12 @@ MODCARGO_CRATES_SQLITE3_BUNDLED ?= No
DISTFILES += ${_MODCARGO_DIST_SUBDIR}${_cratename}-${_cratever}.tar.gz{${_cratename}/${_cratever}/download}:${MODCARGO_MASTER_SITESN}
.endfor

+.for _cratename _cratever _crateurl in ${MODCARGO_CRATES_GIT}
+# strip leading master site for each crate, e.g. https://github.com/author/project/hash -> author/project/hash
+MODCARGO_CRATE_PATH_${_cratename} = ${_crateurl:${MASTER_SITES${MODCARGO_MASTER_SITES_GITN}}%=}
+DISTFILES += ${_MODCARGO_DIST_SUBDIR}${_cratename}-${_cratever}{${MODCARGO_CRATE_PATH_${_cratename}}}.tar.gz:${MODCARGO_MASTER_SITES_GITN}
+.endfor
+
# post-extract target for preparing crates directory.
# It will put all crates in the local crates directory.
MODCARGO_post-extract = \
@@ -352,7 +362,11 @@ modcargo-gen-crates: extract
@awk ' /^name = / { n=$$3; gsub("\"", "", n); } \
/^version = / { v=$$3; gsub("\"", "", v); } \
/^source = "registry\+https:\/\/github.com\/rust-lang\/crates\.io-index"/ \
- { print "MODCARGO_CRATES += " n " " v; }' \
+ { print "MODCARGO_CRATES += " n " " v; } \
+ /^source = .git\+https:\/\/github.com./ \
+ { s=$$3; gsub("\"", "", s); \
+ sub("git\\+", "", s); sub("(\\?.*)?#", "/archive/", s); \
+ print "MODCARGO_CRATES_GIT += " n " " v " " s; }' \
<${MODCARGO_CARGOTOML:toml=lock}

# modcargo-gen-crates-licenses will try to grab license information from downloaded crates.