On Wed, Sep 30, 2026 at 01:44:41PM +0100, Stuart Henderson wrote: > by_cipher has a bit of a problem because it uses unmodified b64-encoded > hashes as directory names, which can include / characters, so rather > than the intended xy/sha256-checksum/filename.tar.gz layout, there are > a bunch of xy/sha25/6-chec/ksum/filename.tar.gz. > > I'm looking at one machine which has 130k hashes so should have approx > 134k dirs counting the various xy/, but because of this, actually has > 220k dirs. So for the sake of namecache it might be advantageous to > replace / with _ or another char. > > More radical but if changing filenames anyway, we probably don't need > an extra level of dirs and filename.tar.gz either; for sha256 it might > as well just be xy/hash and avoid a huge number of dirs holding either > a single file, or a handful of links to a single file. (If sha256 > collisions are a concern there are a whole bunch of problems with the > open-source ecosystem). > > ...however, by_cipher is only used by ports infrastructure when fetching > files from ftp.openbsd.org etc, or when dpb fetches files. Specifically > when _building_, ports only looks in ${FULLDISTDIR}/file, not > ${DISTDIR}/by_ciphers. > > As all bulk builds (including for -stable and -current) share a distfiles > dir, any change in contents for a filename means we have to explicitly > rename the file (either with DISTFILES {url} syntax or DIST_SUBDIR) > anyway. > > So I'm not convinced by_cipher is doing anything useful...do we need to > keep it at all? > > In this day and age, where we have enough disk to keep stuff, we can probably get rid of it. No objection. You will probably need patches for the dpb part, at least ?... I have other things to clean up (long term) in dpb's fetch, as tb@ will know. :(
No comments:
Post a Comment