Also, fupes | duperemove --fdupes tries to re-deduplicate already deduplicated files, because fdupes has no understanding of btrfs native extents...

And duperemove just takes its output and goes "Ok", and then wastes a lot of IO doing nothing.

Like:

Queue entire file for dedupe: /mnt/btrfs/vm/m68k/chroot-0/var/cache/portage/.git/objects/pack/pack-075e6c8edaaee4b5824536ddb5795c58bcdcc4ec.idx
Queue entire file for dedupe: /mnt/btrfs/vm/m68k/chroot-0-snapshot-oct-2016/var/cache/portage/.git/objects/pack/pack-075e6c8edaaee4b5824536ddb5795c58bcdcc4ec.idx
Dedupe pass on 2 files completed

There is Zero way those files have changed since I made the snapshot.

a chain of fdupes | duperemove --fdupes still skips small files.

Wouldn't it just be better for duperemove to have a knob that controls what is considered a "small file" ?

Oh hai @chewi, fancy seeing you here.

Just reporting that under cross compile, the built perl still segfaults this way w/ perl 5.30.3 w/ cross 1.3.4 & GCC 10.

This caused a significant amount of "fun" for building a scratch VM for m68k as it means autoconf and friends like to fail :)

Fortunately, with enough brute force, natively building perl under emulation was achievable, and didn't suffer this problem.

Take note of https://archives.gentoo.org/gentoo-dev/message/2e9e67482f16d5c3833e324f50a3380f

Due to palemoon not being in ::gentoo, there's no pressure on gentoo devs to keep old pythons in tree to support it.

Subsequently, as soon as ::gentoo is able to remove python2.7, the overlay will cease to be usable what-so-ever.

I couldn't find anything on palemoon's side indicating they're cribbing the changes made in firefox to work with python3x and I'm not hugely motivated to sign up to a forum I'll likely never use.

Fortuanately there's still a few things that have to be done on gentoo's side, but this still means pressure must be put somewhere if we don't want this to be predictably broken.

@mathstuf I have duct taped together some utterly terrible code, that will mangle a new style Cargo.lock contents back into an old style lock file in "Cargo.lock".

Its presented as-is in the hope its useful to somebody.

It may be a bit messy to read given how I've had to implement parts of a statemachine and various parser grammar in-line, so ... good luck extending it.

Regex ninja skills may be required, but all the regex is relatively simple, but in ways that mean anything weird I didn't see in my own lock files may not be dealt with at all.

The logic is doable in rust, but I wanted to spend less than an hour making something work.

Enjoy? Feel free to hate the perl.

( I had initially embarked on hand-transcribing a lockfile in a text editor before I decided it would be faster and more reliable to script it poorly )

#!perl
use strict;
use warnings;

# This is very crude.
# 
# 1. Read ./Cargo.lock trying to extract crate name/version/checksum/source details from 'new' format
# 2. Read Cargo.lock a second time, writing it to Cargo.lock.new streamed, but
#    a. strip checksum = from [[ package ]] entries on the way through
#    b. rewrite dependency = [ sections using data collected on the first pass
# 3. Then, generate a [metadata] section from the data obtained in the first pass and emit to the new file.
#
# Its up to you to copy the new file back into place.

my $states = parse_meta("Cargo.lock");
rewrite_file("Cargo.lock","Cargo.lock.new", $states);
1;

sub fix_dep {
    my ( $depstring, $states ) = @_;
    if ( $depstring =~ /^([^ ]+)$/ ) {

        # single map
        my $package = $1;
        if ( not exists $states->{$package} ) {
            die "$package not found";
        }
        my ( $first, @garbage ) = values %{ $states->{$package} };
        if (@garbage) {
            die "Too many versions for $package";
        }
        return
            $package . " "
          . $first->{version} . " ("
          . $first->{source} . ")";
    }
    if ( $depstring =~ /^([^ ]+)[ ]+([^ ]+)$/ ) {
        my ( $package, $version ) = ( $1, $2 );
        if ( not exists $states->{$package} ) {
            die "$package not found";
        }
        if ( not exists $states->{$package}->{$version} ) {
            die "$package v$version not found";
        }
        return
            $package . " "
          . $version . " ("
          . $states->{$package}->{$version}->{source} . ")";
    }
    die "Unhandled depstring $depstring";
}

sub parse_meta {
    my ($filename) = @_;
    open my $fh, "<", $filename or die "Can't read $filename";

    my $states = {};

  lines: while ( my $line = <$fh> ) {
        next unless $line =~ /^\[\[package\]\]/;
        my $record;

      fields: while ( my $line = <$fh> ) {
            last fields if $line =~ /^\s*$/;
            if ( $line =~ /^([^ ]+) = "([^"]+)"/ ) {
                my ( $field, $value ) = ( $1, $2 );
                $record->{$field} = $value;
            }
        }
        if ( not exists $record->{name} ) {
            die "Didn't parse record name";
        }
        if ( not exists $record->{version} ) {
            die "Didn't parse record version for $record->{name}";
        }
        if ( not exists $record->{source} ) {
            warn
"Didn't parse record source for $record->{name} $record->{version}";
        }
        if ( not exists $record->{checksum} ) {
            warn
"Didn't parse record checksum for $record->{name} $record->{version}";
        }
        $states->{ $record->{name} }->{ $record->{version} } = $record;
    }
    return $states;
}

sub rewrite_file {
    my ( $src, $dest, $states ) = @_;
    open my $fh,  "<", $src  or die "Can't read $src";
    open my $wfh, ">", $dest or die "Cant write $dest";

  lines: while ( my $line = <$fh> ) {
        if ( $line !~ /^\[\[package\]\]/ ) {
            $wfh->print($line);
            next lines;
        }
        $wfh->print($line);
      fields: while ( my $line = <$fh> ) {
            if ( $line =~ /^\s*$/ ) {
                $wfh->print($line);
                last fields;
            }

            if ( $line =~ /^([^ ]+) = "/ ) {
                if ( "$1" ne "checksum" ) {
                    $wfh->print($line);
                    next fields;
                }
            }
            if ( $line =~ /^dependencies = \[/ ) {
                $wfh->print($line);
                dep: while ( my $dep = <$fh> ) {
                    if ( $dep =~ /^\]/ ) {
                        $wfh->print($dep);
                        next fields;
                    }
                    if ( $dep =~ /^(\s+")([^"]+)(".+\z)/ms ) {
                        my ( $lpad, $content, $rpad ) = ( $1, $2, $3 );
                        $wfh->print(
                            $lpad . fix_dep( $content, $states ) . $rpad );
                        next dep;
                    }
                    die "Unhandled $dep";
                }
            }
        }
    }
    # write a metadata section
    $wfh->print("\n[metadata]\n");
    for my $crate ( sort keys %{$states} ) {
      for my $version ( sort keys %{$states->{$crate}} ) {
        next unless exists $states->{$crate}->{$version}->{checksum};
        $wfh->printf(qq["checksum %s %s (%s)" = "%s"\n],
          $crate,
          $version,
          $states->{$crate}->{$version}->{source},
          $states->{$crate}->{$version}->{checksum});
      }
    }
}

1;

Fair enough, and I hope you're right :). I've just tended to preferred to be able to prove things via evidence, as opposed to constructing a conjecture that it should be ok, for I find that, however reasonable we are when making such a conjecture, humans always possess the risk of missing something, which only an evidential test will expose. :)

You could just leave the bug open, and then, close it when the problem is fixed. Closing it without fixing the problem doesn't really solve anything. :shrug:

It just makes it more likely that you'll not fix the bug.

Our CI already does feature flag check, and, IIRC, all the current futures's stable feature flags are additive (not only APIs but also implementations). So this PR shouldn't have a negative impact on the actually published crate.

That looks adequate for ensuring everything compiles, sure, but doesn't give many assurances of expected behaviour.

Just restricting this this way makes it harder to test things downstream in a comprehensive way. ( which is a relatively important thing to do, when you're a linux vendor )

Hhm, though I'd admit having the results themselves available in json would have its benefits for parsing simplicity, keeping the existing formats is also good ( better space utilization and more friendly to generic tooling )

For completeness I also tried sqlite 3.32.3, but it gave the same result. ( I'm not presently able to test other versions, nor downgrade glibc )

Able to repro with all Latest versions of each in the series 0.19.* - 0.25.*, 0.18 and under don't compile on this rust.

I have both:

• glibc 2.32
• sqlite 3.33.0

Installed to my system, in case any of that is relevant, ( but I really doubt it )

I've had a hard time nailing down what the problem is with this, but at very least, I've gotten something that reproduces the problem somewhat.

Doesn't seem to be anything in the code, and it does the same thing regardless whether you use a disk-backed database, or an in-memory based database.

But it only happens when you run a binary under memusage

# main.rs
use sqlite;

fn main() {
    let connection = sqlite::open(":memory:").unwrap();
    println!("Hello, world!");
}

# valgrind target/debug/sqlite-mfr

==18989== Memcheck, a memory error detector
==18989== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18989== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==18989== Command: target/debug/sqlite-mfr
==18989== 
Hello, world!
==18989== 
==18989== HEAP SUMMARY:
==18989==     in use at exit: 0 bytes in 0 blocks
==18989==   total heap usage: 298 allocs, 298 frees, 76,130 bytes allocated
==18989== 
==18989== All heap blocks were freed -- no leaks are possible
==18989== 
==18989== For lists of detected and suppressed errors, rerun with: -s
==18989== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

# memusage target/debug/sqlite-mfr
/usr/bin/memusage: line 253: 19481 Segmentation fault      LD_PRELOAD=/\$LIB/libmemusage.so "$@"

# memusage gdb --args target/debug/sqlite-mfr
# run
Program received signal SIGSEGV, Segmentation fault.
__memset_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:192
192		VMOVU	%VEC(0), -VEC_SIZE(%rdi,%rdx)
# bt full
#0  __memset_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:192
No locals.
#1  0x00007ffff7cebb66 in memset (__len=<optimized out>, __ch=0, __dest=0x5555555a7b30) at /usr/include/bits/string_fortified.h:71
No locals.
#2  rehash (new_size=4293843945, pH=0x5555555a4c98) at sqlite3.c:33016
        new_ht = 0x5555555a7b30
        elem = <optimized out>
        next_elem = <optimized out>
#3  sqlite3HashInsert (pH=pH@entry=0x5555555a4c98, pKey=0x5555555a7ac8 "fts5_expr", data=data@entry=0x5555555a7a80) at sqlite3.c:33145
        h = 0
        elem = <optimized out>
        new_elem = 0x5555555a7af0
#4  0x00007ffff7cf215f in sqlite3FindFunction (db=db@entry=0x5555555a4a60, zName=zName@entry=0x7ffff7dbb1e8 "fts5_expr", nArg=nArg@entry=-1, 
    enc=enc@entry=1 '\001', createFlag=createFlag@entry=1 '\001') at sqlite3.c:116005
        pOther = <optimized out>
        z = 0x5555555a7ad1 ""
        p = <optimized out>
        pBest = 0x5555555a7a80
        bestScore = <optimized out>
        h = <optimized out>
        nName = 9
#5  0x00007ffff7cfed97 in sqlite3CreateFunc (db=db@entry=0x5555555a4a60, zFunctionName=zFunctionName@entry=0x7ffff7dbb1e8 "fts5_expr", nArg=nArg@entry=-1, 
    enc=enc@entry=1, pUserData=pUserData@entry=0x5555555a7760, xSFunc=xSFunc@entry=0x7ffff7d46c82 <fts5ExprFunctionHr>, xStep=0x0, xFinal=0x0, xValue=0x0, 
    xInverse=0x0, pDestructor=0x0) at sqlite3.c:162920
        p = <optimized out>
        nName = <optimized out>
        extraFlags = 2097152
#6  0x00007ffff7d130b3 in createFunctionApi (db=db@entry=0x5555555a4a60, zFunc=0x7ffff7dbb1e8 "fts5_expr", nArg=nArg@entry=-1, enc=enc@entry=1, 
    p=p@entry=0x5555555a7760, xSFunc=0x7ffff7d46c82 <fts5ExprFunctionHr>, xStep=0x0, xFinal=0x0, xValue=0x0, xInverse=0x0, xDestroy=0x0) at sqlite3.c:162986
        rc = 1
        pArg = 0x0
#7  0x00007ffff7d130d6 in sqlite3_create_function (db=db@entry=0x5555555a4a60, zFunc=<optimized out>, nArg=nArg@entry=-1, enc=enc@entry=1, 
    p=p@entry=0x5555555a7760, xSFunc=<optimized out>, xStep=0x0, xFinal=0x0) at sqlite3.c:163014
No locals.
#8  0x00007ffff7d1709f in sqlite3Fts5ExprInit (db=0x5555555a4a60, pGlobal=0x5555555a7760) at sqlite3.c:215601
        p = 0x7fffffffca50
        aFunc = {{z = 0x7ffff7dbb1e8 "fts5_expr", x = 0x7ffff7d46c82 <fts5ExprFunctionHr>}, {z = 0x7ffff7dbb1f2 "fts5_expr_tcl", 
            x = 0x7ffff7d46c6f <fts5ExprFunctionTcl>}, {z = 0x7ffff7dbb200 "fts5_isalnum", x = 0x7ffff7cf3e19 <fts5ExprIsAlnum>}, {
            z = 0x7ffff7dbb20d "fts5_fold", x = 0x7ffff7cf3dad <fts5ExprFold>}}
        i = 0
        rc = 0
        pCtx = 0x5555555a7760
#9  fts5Init (db=0x5555555a4a60) at sqlite3.c:29244
        p = 0x5555555a7760
        rc = 0
        pGlobal = 0x5555555a7760
        fts5Mod = {iVersion = 3, xCreate = 0x7ffff7d97d8b <fts5CreateMethod>, xConnect = 0x7ffff7d97d68 <fts5ConnectMethod>, 
          xBestIndex = 0x7ffff7d34710 <fts5BestIndexMethod>, xDisconnect = 0x7ffff7d4828f <fts5DisconnectMethod>, 
          xDestroy = 0x7ffff7d8256e <fts5DestroyMethod>, xOpen = 0x7ffff7d8891d <fts5OpenMethod>, xClose = 0x7ffff7d498b7 <fts5CloseMethod>, 
          xFilter = 0x7ffff7d99f69 <fts5FilterMethod>, xNext = 0x7ffff7d999b5 <fts5NextMethod>, xEof = 0x7ffff7cda9a5 <fts5EofMethod>, 
          xColumn = 0x7ffff7d8a4ca <fts5ColumnMethod>, xRowid = 0x7ffff7ce8a3b <fts5RowidMethod>, xUpdate = 0x7ffff7d9c496 <fts5UpdateMethod>, 
          xBegin = 0x7ffff7d8890a <fts5BeginMethod>, xSync = 0x7ffff7d9bd23 <fts5SyncMethod>, xCommit = 0x7ffff7cda9c7 <fts5CommitMethod>, 
          xRollback = 0x7ffff7d46ce1 <fts5RollbackMethod>, xFindFunction = 0x7ffff7cdaa98 <fts5FindFunctionMethod>, 
          xRename = 0x7ffff7d9bc56 <fts5RenameMethod>, xSavepoint = 0x7ffff7d9bd55 <fts5SavepointMethod>, xRelease = 0x7ffff7d9bd69 <fts5ReleaseMethod>, 
          xRollbackTo = 0x7ffff7d46cc2 <fts5RollbackToMethod>, xShadowName = 0x7ffff7cdab11 <fts5ShadowName>}
        rc = <optimized out>
        pGlobal = <optimized out>
        p = <optimized out>
#10 sqlite3Fts5Init (db=0x5555555a4a60) at sqlite3.c:29310
No locals.
#11 0x00007ffff7da7ac6 in openDatabase (zFilename=0x5555555a4740 ":memory:", ppDb=0x7fffffffcbe8, flags=<optimized out>, zVfs=<optimized out>) at sqlite3.c:164379
        db = 0x5555555a4a60
        rc = 0
        isThreadsafe = <optimized out>
        zOpen = 0x5555555a47c4 ":memory:"
        zErrMsg = 0x0
        i = 1
#12 0x00007ffff7da7c49 in sqlite3_open_v2 (filename=<optimized out>, ppDb=<optimized out>, flags=<optimized out>, zVfs=<optimized out>) at sqlite3.c:164461
No locals.
#13 0x000055555555ae19 in sqlite::connection::Connection::open_with_flags (path=..., flags=...)
    at /home/kent/.cargo/registry/src/github.com-1ecc6299db9ec823/sqlite-0.25.3/src/connection.rs:31
        raw = 0x0
#14 0x000055555555b199 in sqlite::connection::Connection::open (path=...) at /home/kent/.cargo/registry/src/github.com-1ecc6299db9ec823/sqlite-0.25.3/src/connection.rs:24
No locals.
#15 0x000055555555a99a in sqlite::open (path=...) at /home/kent/.cargo/registry/src/github.com-1ecc6299db9ec823/sqlite-0.25.3/src/lib.rs:304
No locals.
#16 0x000055555555b5e0 in sqlite_mfr::main () at src/main.rs:4
No locals.

It could be something in glibc, it could be something in sqlite, but seems a more likely bet is something is "off" in FFI.

I had to rebuild both with debug symbols to make it clearer what's happening here :sweat_smile:

Also, to be clear, I'm not aiming to consume any filtered or sorted end-points, as those are mainly conveniences for people using the browser interface, I'll just be consuming the bulk files and implementing sorting/filtering client side, mostly because that's more flexible and powerful.

I've been musing over writing a general CLI interface to this website, but my main hurdle seems to be enumerating the end-points, as presently, my only option is :vomiting_face: parsing HTML :vomiting_face:

A good start would be a file in /results/ like "index.json", which could at least give a list of repos like:

[
   {     
         "name": "gentoo",
         "description": "Gentoo Main Repo",
         "repo_url": "https://github.com/gentoo/gentoo",
   },
   { "name": "kde", ... },
   ...
]

The data-format for each sub-layer though still has to be worked out. Just the "list of repos that are tracked" seems more likely to be an arbitrary thing than what checks are available, and thus, much harder to encode in software.

But ideally I want to be able to do something like:

# gqa_l9 repos
gentoo
kde
pentoo
rust
science
# gqa_l9 repos --verbose
gentoo - Gentoo Main Repo : https://github.com/gentoo/gentoo

I think ideally, this file could be constructed in such a way that the main html page is generated from it, which makes your ability to ensure consistency is a little better.

I've intentionally left out any hints of references to deeper, similarly useful data structures due to not really knowing how they should look yet, and the the initial goal here is to be able to ingest the content shown on the HTML page, without needing an HTML parser, and presently loading data from those directories is an exercise for the consumer, where the only inferrable data is the "name" element, which must map to the /results/REPONAME/ stuff.

Getopt::Long requires a double dash, there are no single letter dash options defined.

Ah, sorry, that's a fair thing, my fuckup. Sorry. Looks like I inherited some code to look after that was doing this wrong ( and patched so it wouldn't error at that, for reasons lost into the aether, though I suspect related to some generic flags being possibly passed somewhere that may not be mapped ).

HOPEFULLY I'll be able to get this mess cleaned up downstream.

( We don't have any gdlib-config anywhere whatsoever anymore, due to said deprecation )

perl also has the habit to add a wrong /usr/local/lib to the libpath.

usually this is straight foward to fix, at least for us, partly because our perl might be built without hints of that issue. usually the only thing that trips into doing this is silly code in ExtUtils:: stuff (and my god is there a lot of silly stuff in there, don't set LD=ld unless you want to be sorry ), or hard-coded logic in Build/Makefile.PL that thinks it makes sense.

Though the overall point remains that the present Makefile.PL is chaos incarnate, and if it could be structured in a more straight-foward way that is more amenable to making it obvious what needs patching if something needs patching, would go a long way, but I feel it might make more sense for somebody who understands the full weight of "what do we actually need to do here" to write it from scratch.

Particularly with CPAN stuff, the thing most people do is "stuff a thousand different ways of heuristically doing automagic based on whatever happens to be present to make the best chance at getting something working", which does make sense for CPAN, but for a vendor, its a nightmare, as we'd much rather an explicitly defined control, and if the explicitly defined controls are wrong, to simply hard error instead of trying something else, in order to demand the vendor work out what the "right" controls are to pass.

Just because something seems to be available on some path accessible on some filesystem, doesn't mean we ever actually want that to be used.

Yeah, just the downside really is without testing feature combos, you have no evidence features can be arbitrarily combined in a dependent.

So whether or not you have tests that break, you can still have code that breaks, no?

A workaround for "the tests take a long time" might be to set up a "cron" driven CI test, so they don't demand feature combination with PRs and soforth, but a reminder to keep on top of it still happens.

Ugh, I tried to wade in and "fix" this, but I don't think its even fixable, I'm trying to find a polite way to describe how terrible this is to work with, because it makes me alternate between self-abuse trying to fix it, and pondering if we can purge it from our tree entirely, only to bounce back to self abuse due to being unable to purge it :(.

But not sure such a polite option exists.

Obviously I'm at a disadvantage because I can't even understand what half the code is trying to do, because looking at 50% of the code makes you go "If that does that, why doesn't it actually do that", but keeping the myriad of codepaths in your head at once hurts.

https://github.com/Perl/perl5/blob/aa6cf46819b075024f8b18571474f2228eb31541/dist/Exporter/lib/Exporter.pm#L38

sh Configure -de '-Dprefix=/home/kent/perl5/perlbrew/perls/5.33.2-nossp-nopmc-nodot-strict' \
 '-Dcc=x86_64-pc-linux-gnu-gcc' \
 '-Dar=x86_64-pc-linux-gnu-ar' \
 '-Dnm=x86_64-pc-linux-gnu-nm' \
 '-Dcpp=x86_64-pc-linux-gnu-cpp' \
 '-Dranlib=x86_64-pc-linux-gnu-ranlib' \
 '-Ddefault_inc_excludes_dot' \
 '-Dusedefaultstrict' \
 '-Doptimize= -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native' \
 '-Dman1dir=none' \
 '-Dman3dir=none' \
 '-Dusedevel' \
 '-Accflags= -fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC' \
 '-Aldflags= -fstack-protector-strong -fno-stack-protector' \
 '-A'eval:scriptdir=/home/kent/perl5/perlbrew/perls/5.33.2-nossp-nopmc-nodot-strict/bin'' && \ 
 make -j1 && make install

x86_64-pc-linux-gnu-gcc -c -DPERL_CORE -fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -std=c89 -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement time64.c
x86_64-pc-linux-gnu-gcc -c -DPERL_CORE -fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -std=c89 -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement miniperlmain.c
x86_64-pc-linux-gnu-gcc -fstack-protector-strong -fno-stack-protector -L/usr/local/lib -o miniperl \
    opmini.o perlmini.o  gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o dquote.o time64.o  miniperlmain.o  -lpthread -lnsl -ldl -lm -lcrypt -lutil -lc 
./miniperl -w -Ilib -Idist/Exporter/lib -MExporter -e '<?>' || sh -c 'echo >&2 Failed to build miniperl.  Please run make minitest; exit 1'
Can't use string ("Exporter::EXPORT") as an ARRAY ref while "strict refs" in use at dist/Exporter/lib/Exporter.pm line 38.
BEGIN failed--compilation aborted.
Failed to build miniperl. Please run make minitest

( also, there isn't even anything called 'gdlib-config' anywhere on this system, so the notice about "using gdlib-config from PATH" is a total misdirect )

Just as a follow up, its not just ignoring --options, but other flags as well.

For instance, --lib_gd_path does literally nothing, and its default mechanism is also wrong.

perl Makefile.PL -lib_gd_path /usr/libbogus/ -options ANIMGIF
Configuring for libgd version 2.3.0.
Included Features:          GD_GIF GD_OPENPOLYGON GD_ZLIB GD_PNG GD_FREETYPE GD_FONTCONFIG GD_JPEG GD_XPM GD_TIFF GD_WEBP GD_UNCLOSEDPOLY GD_ANIMGIF GD_FTCIRCLE VERSION_33
GD library used from:       /usr/lib
gdlib-config used from the PATH
Warning: prerequisite Test::Fork 0.02 not found.
Generating a Unix-style Makefile
Writing Makefile for GD
Writing MYMETA.yml and MYMETA.json

"bogus" never turns up in Makefile.PL , though its obviously wrong, it doesn't do anything right when you pass an actually right path like "/usr/lib64/"

And the pkg-config stuff looks in the wrong places for config files on our architecture

qlist media-libs/gd | grep pc
# /usr/lib64/pkgconfig/gdlib.pc
# /usr/lib/pkgconfig/gdlib.pc

The one in /usr/lib is for the 32bit compat ... and subsequently, if it uses "/usr/lib/" in the link line, bad things happen:

LD_RUN_PATH="/usr/lib" cc  -shared  -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -L/usr/local/lib  GD.o  -o blib/arch/auto/GD/GD.so  \
   -L/usr/lib -L/usr/lib64 -lgd   \
  
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/libgd.so when searching for -lgd
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/libc.a when searching for -lc

This fortunately isn't a hard panic with GCC, but it is with clang. Because linking 64bit binaries to 32bit ones doesn't make any sense.

Some people set MAKEOPTS to include V=1, in order to coax autotools based makefiles into being more verbose.

But when that ENV var is used to direct perl ebuilds for Dzil things, well, it unintentionally uses "1" as the next version, and so breaks all code that isn't expecting that to happen.

Given its not the job of tests to tell people their system is weird, it just makes the scope "normal" ( that is, defaulting V to be unset internally )

Bug: https://bugs.gentoo.org/737308

Side tip: if you use the "copy permalink" on the "..." left-side widget on github, it gives you a nice link that embeds the source line, but immune to getting content changes by subsequent commits :)

https://github.com/ap/vim-buftabline/blob/728b85835dcf965c67bbedeac02aed660ff3e07a/plugin/buftabline.vim#L44

https://github.com/ap/vim-buftabline/blob/728b85835dcf965c67bbedeac02aed660ff3e07a/plugin/buftabline.vim#L160

I've stumbled down a rabbit hole of sorts.

I've been trying to find sources of Helvetica.afm (mostly just triggered by some obscure test), and in my searches for vendored things, htmldoc turned up.

The Helvetica.afm files in git raise a few questions for me, (some potentially with some legal stuff that we maybe should be handling downstream), particularly as it appears that Helvetica is considered "proprietary software" and requires a paid license from Lineotype: https://www.fonts.com/font/linotype/helvetica

• Where did you/how did you obtain these fonts?
• Is "Helvetica" actually Helvetica or some other font renamed? https://github.com/michaelrsweet/htmldoc/blob/b005d420deb1c928a93ee39299ed4760c96030ea/fonts/Helvetica.afm#L5-L7
• If the latter, what from
• If it is Helvetica, than how can it be "GPL" ? https://github.com/michaelrsweet/htmldoc/blob/b005d420deb1c928a93ee39299ed4760c96030ea/fonts/Helvetica.afm#L4
• Where is the "Copying" file for the fonts? ( https://github.com/michaelrsweet/htmldoc/tree/master/fonts ), I mean, one might assume its the same as the top level COPYING, but just the excerpted line doesn't clarify which GPL version its referring to, so it might be a different file that wasn't copied :/

And I don't see any good evidence that URW++ created a font called "Helvetica": https://github.com/michaelrsweet/htmldoc/blob/b005d420deb1c928a93ee39299ed4760c96030ea/fonts/Helvetica.afm#L14

But I do know "Nimbus Sans L" is a thing, and was a Helvetica-like font by URW++ that is GPLy, but maybe AGPL-3! :)

https://github.com/ArtifexSoftware/urw-base35-fonts/blob/master/fonts/COPYING

Apologies if this all requires long unactivated memory cells :)

Ideally, for META_MERGE you want to use the v2 spec, and then use test => { recommends => { } }.

That is if you're insisting on them being dependencies at all.

https://github.com/rurban/Filter/blob/e5360f4b14f62f14e9eedd35b30d84b49b99b12d/Makefile.PL#L69-L83

https://metacpan.org/source/RURBAN/Filter-1.60/META.json#L33-46

Given that these dependencies aren't used for anything other than tests, and authortests at that, making them runtime recommendations is completely terrible.

In general, its preferred that authortests aren't even shipped in a way that subject end-users to their sillyness, but declaring the dependencies properly is better than nothing.

This is also a bit of an issue with reviewing PR's using Github's interface, because the diff of a version bump doesn't show "what changed", but shows the whole ebuild as being "new".

Where what is useful to review is the diff between the "new" ebuild and the previous one.

If Github presented the "version bump" as a proper diff, the diff you'd have reviewed would have only been:

--- rex-1.12.1.ebuild	2020-08-08 04:41:11.892146874 +1200
+++ rex-1.12.2.ebuild	2020-09-09 19:00:12.275747373 +1200
@@ -30,6 +30,7 @@
 	dev-perl/Dist-Zilla
 	dev-perl/Dist-Zilla-Plugin-CheckExtraTests
 	dev-perl/Dist-Zilla-Plugin-ContributorsFile
+	dev-perl/Dist-Zilla-Plugin-Git
 	dev-perl/Dist-Zilla-Plugin-Git-Contributors
 	dev-perl/Dist-Zilla-Plugin-MakeMaker-Awesome
 	dev-perl/Dist-Zilla-Plugin-Meta-Contributors
@@ -87,6 +88,7 @@
 	test? (
 		virtual/perl-File-Temp
 		dev-perl/Test-Deep
+		dev-perl/Test-Output
 		>=dev-perl/Test-UseAllModules-0.150.0
 		virtual/perl-autodie
 	)
@@ -121,6 +123,7 @@
 		-e '/^Test::PerlTidy =/d' \
 		-e '/^Test::Pod =/d' \
 		-e '/^\[Test::CPAN::Changes\]/{N;d}' \
+		-e '/^\[OptionalFeature/,/^$/d' \
 		-e '/^\[Test::MinimumVersion\]/{N;d}' \
 		-i dist.ini || die "Can't patch dist.ini"
 }

( And this is why I like to manually summarize the inter-version diff when I submit my own PR's for others to merge )

I don't need them as such, just the formatting as-is enticed me into parsing it out and putting it in its own field :)

Having the actual dates is helpful, because sometimes the relative dates aren't so obvious, but logically, if you can calculate the age, then you can calculate the date from that, as long as you know what "now" is. ( Though obviously, "now" is not "now", but "when the data was last calculated" )

Yeah, seems better now. :+1:

I have a tiny little script that just does some pretty crude grepping of certain keywords that I've been pointing at every CPAN dist I vendorize, and its crudeness makes it very sensitive at detecting obscure outliers ^_^

Yeah, the thing with META.yml is somewhat expected, but was mostly presented as evidence that "perl" is the outlier.

Is it ok for you to add another 2 columns, something like that:

Yeah, my internal "map columns to fields" is flexible, designed in mind with the assumption keys may change or key orders might change.

 for my $line (@lines) {
        my %rec;
        @rec{
            qw(old_eapi old_age new_eapi new_age has_bugs cat_pn old_ebuild new_ebuild maintainers junk)
        } = split /\|/, $line;
        for (qw( old new )) {
            my $ebuild = delete $rec{ $_ . '_ebuild' };
            $ebuild =~ /\A(.*?)\(([^)]+)\)\z/;
            $rec{$_} = {
                eapi   => delete $rec{ $_ . '_eapi' },
                age    => delete $rec{ $_ . '_age' },
                ebuild => "$1",
                date   => "$2",
            };
            @rec{qw(cat pn)} = split q[/], $rec{cat_pn};

            if ( $rec{$_}->{ebuild} =~ /\A\Q$rec{pn}\E-(.*?)\z/ ) {
              $rec{$_}->{version} = "$1";
            }
        }
        $rec{maintainers} = [ split /:/, $rec{maintainers} ];
        $rec{age_gap} = $rec{old}->{age} - $rec{new}->{age};
        push @recs, \%rec;
    }

Though I guess this bug serves as a second vote for https://github.com/mm1ke/gentoo-scripts/issues/62, as if I'd been tracking "age of last change" instead of "age of addition to tree", the ages would have been logical :smile:

Ouch:

https://gentooqa.levelnine.at/results/gentoo/stats/EAC-STA-ebuild_cleanup_candidates/sort-by-maintainer/perl_at_g.o.txt

5|00078|7|00129|-|dev-perl/AutoXS-Header|AutoXS-Header-1.20.0-r1(2020-06-20)|AutoXS-Header-1.20.0-r2(2020-04-30)|[email protected]

Its correct that its a cleanup candidate, just the timestamp math is ... spooky.

I spotted it because I parse and reformat the line as follows:

  dev-perl/                           AutoXS-Header    1.20.0-r1 →    1.20.0-r2 ( 2020-06-20 → 2020-04-30 ) <   00078 →   00129 (diff:     -51)

The data is not exactly wrong ... its just really not what I expected, and I don't know what should be done about this, if at all.

I just sort of expected it would track as if the removal and revert of removal never happened.

( Obviously neither number is perfect on its own, but having a high-water and a low-water mark can greatly accelerate working out the bulk of the most important values )

Looking at the results, it looks like "FILE AGE(NV)" is based on the timestamp it first entered the tree in git.

However, for me that's not a useful number to analyze, as things that happened after that are more important when considering if its been in a working, usable state long enough to stabilize, or purge.

For instance,

5|01826|7|00121|-|dev-perl/CPAN-Changes|CPAN-Changes-0.400.2(2015-09-07)|CPAN-Changes-0.400.2-r1(2020-05-08)|[email protected]

This has indeed, been in tree, for 121 days.

But it was only stabilized for amd64 13 days ago, so I consider it still too premature to cleanup when there's no other pressing reasons.

I presume you're already poking into the history with git to work these out, so you could possibly collect the "last changed" commit timestamp as well while you're iterating to find that. (would be just the "lastest" commit that referred to that file instead of simply the first commit that referred to that file ).

Same logic applies to me when considering stabilization candidates, "added to tree 3 months ago" is not good enough for me to ponder stabilization if the last keyword was added yesterday.

Yeah, that's much better, all the remaining ones now look mostly like "I recently changed this, that's the only real reason its not stabilized", the others are probably due to open bugs ;)

I was wondering why so many were listed for perl@ that hadn't turned up yet on my radar, so I went looking.

List generated on 2020-09-05

1

5|01855|7|00130|*|dev-perl/Apache-SizeLimit|Apache-SizeLimit-0.970.0-r1(2015-08-08)|Apache-SizeLimit-0.970.0-r2(2020-04-28)|[email protected]

Keywords for dev-perl/Apache-SizeLimit:
           |                                                       s         |       |  
           |                                                       p         |       |  
           |                                                     s a         |       |  
           |                                                     p r         |       |  
           |                             a         a p           a c x x     |       |  
           |                             m   x x   r p           r 6 6 8     |       |  
           |                             d   6 8 a m c x p x x m c 4 4 6 x x |       |  
           |                             6   4 6 r 6 6 8 p 6 8 6 | | | | 6 8 |       |  
           |                             4 p | | m 4 4 6 c 4 6 8 s s s s 4 6 |       |  
           |                             | p c c | | | | | | | k o o o o | | |   u   |  
           | a   a     p s     a   r     l c y y l l l l m m m | l l l l w w |   n   |  
           | m   r h   p p   s l i i m m i | g g i i i i a a a m a a a a i i | e u s | r
           | d a m p p c a x 3 p a s 6 i n a w w n n n n c c c i r r r r n n | a s l | e
           | 6 r 6 p p 6 r 8 9 h 6 c 8 p u i i i u u u u o o o n i i i i n n | p e o | p
           | 4 m 4 a c 4 c 6 0 a 4 v k s x x n n x x x x s s s t s s s s t t | i d t | o
-----------+-----------------------------------------------------------------+-------+-------
0.970.0-r1 | + ~ o o + + o + o o o o o o o o o o o o o o o o o o o o o o o o | 5 o 0 | gentoo
0.970.0-r2 | ~ o o o o o o ~ o o o o o o o o o o o o o o o o o o o o o o o o | 7 o   | gentoo

Well, that's clearly nonsense, I wouldn't stabilize a thing until its fully keyworded, and then I can stabilize. I'm not sure if this is by design, but its very odd.

2

5|01855|7|00128|-|dev-perl/Authen-NTLM|Authen-NTLM-1.90.0-r1(2015-08-08)|Authen-NTLM-1.90.0-r2(2020-04-30)|[email protected]

Keywords for dev-perl/Authen-NTLM:
          |                                                       s         |       |  
          |                                                       p         |       |  
          |                                                     s a         |       |  
          |                                                     p r         |       |  
          |                             a         a p           a c x x     |       |  
          |                             m   x x   r p           r 6 6 8     |       |  
          |                             d   6 8 a m c x p x x m c 4 4 6 x x |       |  
          |                             6   4 6 r 6 6 8 p 6 8 6 | | | | 6 8 |       |  
          |                             4 p | | m 4 4 6 c 4 6 8 s s s s 4 6 |       |  
          |                             | p c c | | | | | | | k o o o o | | |   u   |  
          | a   a     p s     a   r     l c y y l l l l m m m | l l l l w w |   n   |  
          | m   r h   p p   s l i i m m i | g g i i i i a a a m a a a a i i | e u s | r
          | d a m p p c a x 3 p a s 6 i n a w w n n n n c c c i r r r r n n | a s l | e
          | 6 r 6 p p 6 r 8 9 h 6 c 8 p u i i i u u u u o o o n i i i i n n | p e o | p
          | 4 m 4 a c 4 c 6 0 a 4 v k s x x n n x x x x s s s t s s s s t t | i d t | o
----------+-----------------------------------------------------------------+-------+-------
1.90.0-r1 | + + o ~ + ~ + + ~ ~ ~ o o o ~ ~ o o o o o ~ ~ ~ ~ ~ ~ ~ ~ ~ o o | 5 # 0 | gentoo
1.90.0-r2 | + + ~ ~ + ~ + + ~ ~ ~ o o o ~ ~ o o o o o ~ ~ ~ ~ ~ ~ ~ ~ ~ o o | 7 o   | gentoo

Huh? That's already stabilized, what's it talking about?

3

5|01855|7|00064|-|dev-perl/capitalization|capitalization-0.30.0-r1(2015-08-08)|capitalization-0.30.0-r2(2020-07-03)|[email protected]

Keywords for dev-perl/capitalization:
          |                                                       s         |       |  
          |                                                       p         |       |  
          |                                                     s a         |       |  
          |                                                     p r         |       |  
          |                             a         a p           a c x x     |       |  
          |                             m   x x   r p           r 6 6 8     |       |  
          |                             d   6 8 a m c x p x x m c 4 4 6 x x |       |  
          |                             6   4 6 r 6 6 8 p 6 8 6 | | | | 6 8 |       |  
          |                             4 p | | m 4 4 6 c 4 6 8 s s s s 4 6 |       |  
          |                             | p c c | | | | | | | k o o o o | | |   u   |  
          | a   a     p s     a   r     l c y y l l l l m m m | l l l l w w |   n   |  
          | m   r h   p p   s l i i m m i | g g i i i i a a a m a a a a i i | e u s | r
          | d a m p p c a x 3 p a s 6 i n a w w n n n n c c c i r r r r n n | a s l | e
          | 6 r 6 p p 6 r 8 9 h 6 c 8 p u i i i u u u u o o o n i i i i n n | p e o | p
          | 4 m 4 a c 4 c 6 0 a 4 v k s x x n n x x x x s s s t s s s s t t | i d t | o
----------+-----------------------------------------------------------------+-------+-------
0.30.0-r1 | + o o + + o + + o ~ ~ o o o o o o o o o o o o o o o o o o o o o | 5 o 0 | gentoo
0.30.0-r2 | ~ o o ~ ~ o ~ ~ o ~ ~ o o o o o o o o o o o o o o o o o o o o o | 7 o   | gentoo

Ah, well, that one checks out, It just didn't get a stable request yet because last time I was filing them, it hadn't been sitting around long enough :p.

4

5|01855|7|00066|-|dev-perl/Class-MethodMaker|Class-MethodMaker-2.240.0(2015-08-08)|Class-MethodMaker-2.240.0-r2(2020-07-01)|[email protected]

Keywords for dev-perl/Class-MethodMaker:
           |                                                       s         |       |  
           |                                                       p         |       |  
           |                                                     s a         |       |  
           |                                                     p r         |       |  
           |                             a         a p           a c x x     |       |  
           |                             m   x x   r p           r 6 6 8     |       |  
           |                             d   6 8 a m c x p x x m c 4 4 6 x x |       |  
           |                             6   4 6 r 6 6 8 p 6 8 6 | | | | 6 8 |       |  
           |                             4 p | | m 4 4 6 c 4 6 8 s s s s 4 6 |       |  
           |                             | p c c | | | | | | | k o o o o | | |   u   |  
           | a   a     p s     a   r     l c y y l l l l m m m | l l l l w w |   n   |  
           | m   r h   p p   s l i i m m i | g g i i i i a a a m a a a a i i | e u s | r
           | d a m p p c a x 3 p a s 6 i n a w w n n n n c c c i r r r r n n | a s l | e
           | 6 r 6 p p 6 r 8 9 h 6 c 8 p u i i i u u u u o o o n i i i i n n | p e o | p
           | 4 m 4 a c 4 c 6 0 a 4 v k s x x n n x x x x s s s t s s s s t t | i d t | o
-----------+-----------------------------------------------------------------+-------+-------
2.240.0    | + + o + + + + + ~ ~ ~ o ~ ~ ~ ~ o o o o o ~ ~ ~ ~ o ~ o o ~ o o | 5 # 0 | gentoo
2.240.0-r2 | + + ~ + + + + + ~ ~ ~ o ~ ~ ~ ~ o o o o o ~ ~ ~ ~ o ~ o o ~ o o | 7 o   | gentoo

Huh? Again, that's stable, what's the deal?

I tried looking through the page: No luck I tried looking through the page source: No luck I tried looking at whois data: No luck

Though with even more persistence, I found it on like, page 8 of google search results for something.

But even then I wasn't sure, and had to ask somebody on a gentoo IRC channel.

It would be nice if there was some sort of "and here's the github for this page if you have issues/suggestions", or alternatively, at very minimum, some IRC aliases for gentoo dev's to harass.

( This criticism aside, its a very useful website :) )

This problem is easier to demonstrate if you check out 1.03 and see what it doesn't do:

# /usr/bin/perl Makefile.PL 
Checking for magic.h... yes
Checking for magic_open in -lmagic... yes
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for File::LibMagic
Writing MYMETA.yml and MYMETA.json
# make OPTIMIZE="-O3 -frecord-gcc-switches"
cp lib/File/LibMagic.pm blib/lib/File/LibMagic.pm
Running Mkbootstrap for LibMagic ()
chmod 644 "LibMagic.bs"
"/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- LibMagic.bs blib/arch/auto/File/LibMagic/LibMagic.bs 644
"/usr/bin/perl" "/usr/lib64/perl5/5.32/ExtUtils/xsubpp"  -typemap '/usr/lib64/perl5/5.32/ExtUtils/typemap'  LibMagic.xs > LibMagic.xsc
mv LibMagic.xsc LibMagic.c
x86_64-pc-linux-gnu-gcc -c  -I. -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -frecord-gcc-switches   -DVERSION=\"1.03\" -DXS_VERSION=\"1.03\" -fPIC "-I/usr/lib64/perl5/5.32/x86_64-linux/CORE"   LibMagic.c
rm -f blib/arch/auto/File/LibMagic/LibMagic.so
LD_RUN_PATH="/usr/lib64" x86_64-pc-linux-gnu-gcc  -shared -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -Wl,-O1  -Wl,--as-needed -fstack-protector-strong -fno-stack-protector  LibMagic.o  -o blib/arch/auto/File/LibMagic/LibMagic.so  \
   -lmagic   \
  
chmod 755 blib/arch/auto/File/LibMagic/LibMagic.so
Manifying 1 pod document
# make
"/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- LibMagic.bs blib/arch/auto/File/LibMagic/LibMagic.bs 644
Manifying 1 pod document

The key observation here is subsequent calls to make don't rebuild the .so components, and thus, the .so components retain the flags that were passed in the initial make call.

Now, compare with v1.22

# /usr/bin/perl Makefile.PL 
Checking for magic.h... Checking for cc... x86_64-pc-linux-gnu-gcc
yes
Checking for magic_open in -lmagic... yes
Checking for magic_version in -lmagic... yes
Checking for magic_setparam in -lmagic... yes
Checking for magic_getparam in -lmagic... yes
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for File::LibMagic
Writing MYMETA.yml and MYMETA.json
# make OPTIMIZE="-O3 -frecord-gcc-switches"
cp lib/File/LibMagic.pm blib/lib/File/LibMagic.pm
cp lib/File/LibMagic/Constants.pm blib/lib/File/LibMagic/Constants.pm
Running Mkbootstrap for LibMagic ()
chmod 644 "LibMagic.bs"
"/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- LibMagic.bs blib/arch/auto/File/LibMagic/LibMagic.bs 644
"/usr/bin/perl" "/usr/lib64/perl5/5.32/ExtUtils/xsubpp"  -typemap '/usr/lib64/perl5/5.32/ExtUtils/typemap'  lib/File/LibMagic.xs > lib/File/LibMagic.xsc
mv lib/File/LibMagic.xsc lib/File/LibMagic.c
x86_64-pc-linux-gnu-gcc -c  -I. -Ic -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -frecord-gcc-switches   -DVERSION=\"1.22\" -DXS_VERSION=\"1.22\" -fPIC "-I/usr/lib64/perl5/5.32/x86_64-linux/CORE"  -DHAVE_MAGIC_VERSION -DHAVE_MAGIC_SETPARAM -DHAVE_MAGIC_GETPARAM lib/File/LibMagic.c
rm -f blib/arch/auto/File/LibMagic/LibMagic.so
LD_RUN_PATH="/usr/lib64" x86_64-pc-linux-gnu-gcc  -shared -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector  LibMagic.o  -o blib/arch/auto/File/LibMagic/LibMagic.so  \
   -lmagic   \
  
chmod 755 blib/arch/auto/File/LibMagic/LibMagic.so
Manifying 2 pod documents
# make
"/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- LibMagic.bs blib/arch/auto/File/LibMagic/LibMagic.bs 644
x86_64-pc-linux-gnu-gcc -c  -I. -Ic -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector   -DVERSION=\"1.22\" -DXS_VERSION=\"1.22\" -fPIC "-I/usr/lib64/perl5/5.32/x86_64-linux/CORE"  -DHAVE_MAGIC_VERSION -DHAVE_MAGIC_SETPARAM -DHAVE_MAGIC_GETPARAM lib/File/LibMagic.c
rm -f blib/arch/auto/File/LibMagic/LibMagic.so
LD_RUN_PATH="/usr/lib64" x86_64-pc-linux-gnu-gcc  -shared -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector  LibMagic.o  -o blib/arch/auto/File/LibMagic/LibMagic.so  \
   -lmagic   \
  
chmod 755 blib/arch/auto/File/LibMagic/LibMagic.so
Manifying 2 pod documents
# make
"/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- LibMagic.bs blib/arch/auto/File/LibMagic/LibMagic.bs 644
x86_64-pc-linux-gnu-gcc -c  -I. -Ic -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector   -DVERSION=\"1.22\" -DXS_VERSION=\"1.22\" -fPIC "-I/usr/lib64/perl5/5.32/x86_64-linux/CORE"  -DHAVE_MAGIC_VERSION -DHAVE_MAGIC_SETPARAM -DHAVE_MAGIC_GETPARAM lib/File/LibMagic.c
rm -f blib/arch/auto/File/LibMagic/LibMagic.so
LD_RUN_PATH="/usr/lib64" x86_64-pc-linux-gnu-gcc  -shared -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector  LibMagic.o  -o blib/arch/auto/File/LibMagic/LibMagic.so  \
   -lmagic   \
  
chmod 755 blib/arch/auto/File/LibMagic/LibMagic.so
Manifying 2 pod documents

In its current incarnation, it seems whatever is done in Makefile.PL to faciliate this "have XS inside lib/" strategy, results in make cache invalidation fail entirely, and so, the target is recompiled EVERY TIME make is called.

And subsequently, it breaks when tooling applies to it that didn't expect this behaviour, because they expected the assets to not be unhelpfully recompiled without the required settings.

And its generally frowned upon to be recompiling your libraries during make test or make install.

A certain somebody says:

<___> The value for OBJECT is wrong
...
<___> It recompiles because lib/File/LibMagic\$(OBJ_EXT) never exists
...
<___> The whole setup is broken and wrong
<___> I don't recall any way to make .xs in lib/ work without using mohawks new stuff, and even then I recall running into some issues (but forgot details)

Ah, my mistake, not added in the PR, but added inline in the merge as the test.

https://github.com/houseabsolute/DateTime-Locale/commit/3cdc7ce49e07994324cab85633f1aec9f177d5e6

Failure of course introduced by #24

https://github.com/houseabsolute/DateTime-Locale/blob/d727eb79700f8996faaff92e58f7d8e1ae010aa5/t/nonexistent-locale.t#L24

Well, yeah, that explains things.

And a few cpan testers replications (all by SREZIC)

• http://www.cpantesters.org/cpan/report/7e6e9f12-e95e-11ea-9b31-bf311f24ea8f
• http://www.cpantesters.org/cpan/report/6814bb66-e95e-11ea-a553-9a311f24ea8f
• http://www.cpantesters.org/cpan/report/d8d3fb98-e961-11ea-9f4f-cb661f24ea8f
• http://www.cpantesters.org/cpan/report/1aad1e18-e95e-11ea-8e2e-732f1f24ea8f

===(     703;5  665/?  1/?  1/? )=======================================
# Failed test 'got an exception trying to read an unreadable file'
# at t/nonexistent-locale.t line 24.
# +--------------------------------+----+--------------------------------+
# | GOT                            | OP | CHECK                          |
# +--------------------------------+----+--------------------------------+
# | File '/var/tmp/portage/dev-per | =~ | (?^:No read permission)        |
# | l/DateTime-Locale-1.270.0/temp |    |                                |
# | /iPMqstUq4y/auto/share/dist/Da |    |                                |
# | teTime-Locale/unreadable.pl',  |    |                                |
# | no read permissions at /var/tm |    |                                |
# | p/portage/dev-perl/DateTime-Lo |    |                                |
# | cale-1.270.0/work/DateTime-Loc |    |                                |
# | ale-1.27/blib/lib/DateTime/Loc |    |                                |
# | ale/Data.pm line 6094.\n       |    |                                |
# +--------------------------------+----+--------------------------------+
# Seeded srand with seed '20200829' from local date.
t/nonexistent-locale.t ........... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 

Technically I think the other bug is the dupe, being newer, but not sure what standard conventions for this are on github

Out of curiosity, may I ask why you need to do --no-default-features when running tests on your side?

Firstly, dependents can specify this, so it makes sense to test this combination.

Secondly, without it, one can't really reliably do permutational testing to test each flag is usable on its own, nor test how flags interact.

Subsequently, this bug indicates a specific combination of features that don't work ( or at least, can't be tested to work ).

For more exhaustive testing, consider using cargo hack test --feature-powerset

And even if you mitigate the above by repressing errors on "file exists" in some way, you're still left with other race conditions due to directory cleanup happening.

Can't call method "desiredCompressionLevel" on an undefined value at /home/kent/.cpanm/work/1597995761.21949/Archive-Zip-1.68/blib/lib/Archive/Zip/Archive.pm line 296.
	Archive::Zip::Archive::addFile(Archive::Zip::Archive=HASH(0x55889ad33a38), "./testdir/oI5NYEN6JQ/22_deflated_dir.t", "testdir/oI5NYEN6JQ/22_deflated_dir.t") called at /home/kent/.cpanm/work/1597995761.21949/Archive-Zip-1.68/blib/lib/Archive/Zip/Archive.pm line 1135
	Archive::Zip::Archive::addTree(Archive::Zip::Archive=HASH(0x55889ad33a38), ".", "", CODE(0x55889bd7f490)) called at t/05_tree.t line 25
	main::makeZip(".", "", CODE(0x55889bd7f490)) called at t/05_tree.t line 32
	main::makeZipAndLookFor(".", "", CODE(0x55889bd7f490), "t/02_main.t") called at t/05_tree.t line 39
# Looks like your test exited with 2 before it could output anything.
t/05_tree.t ................... Dubious, test returned 2 (wstat 512, 0x200)                            

One of our testers hit this problem

make -j16 V=1 test TEST_VERBOSE=0 
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
File exists at t/common.pm line 36.
BEGIN failed--compilation aborted at t/common.pm line 38.
Compilation failed in require at t/02_main.t line 16.
BEGIN failed--compilation aborted at t/02_main.t line 16.

                                                                        

t/02_main.t ................... Dubious, test returned 17 (wstat 4352, 0x1100)
No subtests run 

===(       0;0  0/2   0/18   0/10  0/8   0/16   0/14   0/15   0/... )===
===(       1;0  1/2   0/18   0/10  0/8   0/16   0/14   0/15   0/... )===
===(       3;0  2/2   0/18   1/10  0/8   0/16   0/14   0/15   0/... )===
                                                                        

t/01_init.t ................... ok
...
Test Summary Report
-------------------
t/02_main.t                 (Wstat: 4352 Tests: 0 Failed: 0)
  Non-zero exit status: 17
  Parse errors: No plan found in TAP output
Files=28, Tests=392,  1 wallclock secs ( 0.12 usr  0.02 sys +  5.06 cusr  1.47 csys =  6.67 CPU)

And it seems to indicate there's a rare problem in this line if you get the timing just perfect: https://github.com/redhotpenguin/perl-Archive-Zip/blob/340cd7ea81ecd0b55ca4a2bfd2331c2787cc77f7/t/common.pm#L35-L38

I presume this error happens because the directory didn't exist when 2 tests started at the same second, and sheer luck made one of them execute mkdir between the time they both returned false for -d, and before the other process executed mkdir, leading one case of mkdir to error due to already existing.

On our test system, there is no f77 or gfortran in path, so all the heuristics become faulty. ( Its explicitly set up this way to mimic more exotic systems that don't have a GNU-standard toolchain, and to smoke out potential issues with cross compiling )

For me, the "right" gfortran is "x86_64-pc-linux-gnu-gfortran", but you shouldn't be trying to use that hardcoded string as a heuristic either.

As this becomes important for cross-compiling, its important that the consumer has the ability to stipulate their cross target, as auto-detection there can't be imagined to work.

But presently, this means even non-cross workflows result in inability to test.

make -j3 --load-average=4 test TEST_VERBOSE=0 
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
ExtUtils::F77: Using system=Linux compiler=GNU
ExtUtils::F77: Unable to guess and/or validate system/compiler configuration
ExtUtils::F77: Will try system=Generic Compiler=G77
ExtUtils::F77: Well that didn't appear to validate. Well I will try it anyway.
      ExtUtils::F77: There does not appear to be any configuration info about
      ExtUtils::F77: names with trailing underscores for system Generic. Will assume
      ExtUtils::F77: F77 names have trailing underscores.
      ExtUtils::F77: There does not appear to be any configuration info about
      ExtUtils::F77: the F77 compiler name. Will assume 'f77'.
ExtUtils::F77: Compiler: f77
      ExtUtils::F77: There does not appear to be any configuration info about
      ExtUtils::F77: the options for the F77 compiler. Will assume none
      ExtUtils::F77: necessary.
ExtUtils::F77: Cflags: 
t/require.t .. 1/3 Compiling the test Fortran program...
Can't exec "f77": No such file or directory at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 571.
Executing the test program...
Can't exec "/var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/temp/testf7761_exe": No such file or directory at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 573.
Use of uninitialized value in string ne at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 573.
Test of Fortran Compiler FAILED. 
Do not know how to compile Fortran on your system

#   Failed test 'testcompiler method returns 1'
#   at t/require.t line 11.
#          got: '0'
#     expected: '1'

#   Failed test 'runtime libs found'
#   at t/require.t line 13.
#          got: undef
#     expected: '1'
# Method: runtime, undef
# Method: trail_, 1
# Method: compiler, f77
# Method: cflags, 
# Compiler: 
# Looks like you failed 2 tests of 3.
t/require.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/3 subtests 

Test Summary Report
-------------------
t/require.t (Wstat: 512 Tests: 3 Failed: 2)
  Failed tests:  2-3
  Non-zero exit status: 2
Files=1, Tests=3,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.09 cusr  0.04 csys =  0.17 CPU)
Result: FAIL

I attempted to fix it myself, but the logic in this is too contorted for my patience.

I got as far as

From 3fafdbb0f288c373ee2c0781ac0717fc41f51495 Mon Sep 17 00:00:00 2001
From: Kent Fredric <[email protected]>
Date: Thu, 20 Aug 2020 14:15:15 +1200
Subject: Respect F77 set in env

As all the autodiscovery magic is wrong
---
 F77.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/F77.pm b/F77.pm
index dd2e949..34bbf35 100644
--- a/F77.pm
+++ b/F77.pm
@@ -62,6 +62,9 @@ else {
    $gfortran = 'gfortran';
    $fallback_compiler = 'G77';
 }
+$gcc = $ENV{CC} if $ENV{CC};
+$gfortran = $ENV{F77} if $ENV{F77};
+$fallback_compiler = $ENV{F77} if $ENV{F77};
 
 ############## End of Win32 Specific ##############
 
@@ -236,7 +239,7 @@ $F77config{Solaris}{DEFAULT} = 'F77';
 
 $F77config{Generic}{GNU}{Trail_} = 1;
 $F77config{Generic}{GNU}{Cflags} = ' ';        # <---need this space!
-$F77config{Generic}{GNU}{Link}   = link_gnufortran_compiler('gfortran', 'g77', 'g95', 'fort77');
+$F77config{Generic}{GNU}{Link}   = link_gnufortran_compiler("$gfortran", 'g77', 'g95', 'fort77');
 $F77config{Generic}{GNU}{Compiler} = find_in_path("$gfortran", 'g77',  'g95','fort77');
 
 $F77config{Generic}{DEFAULT}     = 'GNU';
-- 

Which seems to have gotten some headway, but its still far from any good.

make -j3 --load-average=4 test TEST_VERBOSE=0 
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
Found compiler x86_64-pc-linux-gnu-gfortran
ExtUtils::F77: x86_64-pc-linux-gnu-gfortran version 10.1.0
t/require.t .. 1/3 
#   Failed test 'use ExtUtils::F77;'
#   at t/require.t line 9.
#     Tried to use 'ExtUtils::F77'.
#     Error:  Can't use an undefined value as an ARRAY reference at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 652.
# Compilation failed in require at t/require.t line 9.
# BEGIN failed--compilation aborted at t/require.t line 9.
Compiling the test Fortran program...
Can't exec "/var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/temp/testf7761.f": Permission denied at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 574.
Executing the test program...
Can't exec "/var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/temp/testf7761_exe": No such file or directory at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 576.
Use of uninitialized value in string ne at /var/tmp/portage/dev-perl/ExtUtils-F77-1.240.0/work/ExtUtils-F77-1.24/blib/lib/ExtUtils/F77.pm line 576.
Test of Fortran Compiler FAILED. 
Do not know how to compile Fortran on your system

#   Failed test 'testcompiler method returns 1'
#   at t/require.t line 11.
#          got: '0'
#     expected: '1'

#   Failed test 'runtime libs found'
#   at t/require.t line 13.
#          got: '0'
#     expected: '1'
# Method: runtime, -LSNAFU -lwontwork
# Method: trail_, 1
# Method: compiler, 
# Method: cflags, 
# Compiler: undef
# Looks like you failed 3 tests of 3.
t/require.t .. Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/3 subtests 

Test Summary Report
-------------------
t/require.t (Wstat: 768 Tests: 3 Failed: 3)
  Failed tests:  1-3
  Non-zero exit status: 3
Files=1, Tests=3,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.09 cusr  0.04 csys =  0.17 CPU)
Result: FAIL
Failed 1/1 test programs. 3/3 subtests failed.

These failures started occurring in 0.15, 0.14 is the newest version tests pass for me on.

/me blames @mohawk2

I have a setup wherein, there is no g++ anywhere in $PATH, nor any of the things this module looks for.

Perl itself has all relevant CC and stuff passed to it during its configure/compile, but of course, perl itself doesn't have any CXX stuff in %Config.

Under this configuration, nothing I seem to be able to do fixes the problem, explicitly setting CXX in ENV doesn't help at all, and the number of places in the code that hardcode the CXX to be "g++" is ridiculous.

>>> Test phase: dev-perl/ExtUtils-CppGuess-0.210.0
make -j3 --load-average=4 test TEST_VERBOSE=0 
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-report-prereqs.t .. # 
# Versions for all modules listed in MYMETA.json (including optional ones):
# 
# === Configure Requires ===
# 
#     Module              Want Have Where                 Howbig
#     ------------------- ---- ---- --------------------- ------
#     ExtUtils::MakeMaker  any 7.44 /usr/lib64/perl5/5.32 109673
# 
# === Build Requires ===
# 
#     Module              Want Have Where                 Howbig
#     ------------------- ---- ---- --------------------- ------
#     ExtUtils::MakeMaker  any 7.44 /usr/lib64/perl5/5.32 109673
# 
# === Test Requires ===
# 
#     Module                  Want     Have Where                              Howbig
#     ------------------- -------- -------- ---------------------------------- ------
#     Capture::Tiny            any     0.48 /usr/lib64/perl5/vendor_perl/5.32   29656
#     Cwd                      any     3.78 /usr/lib64/perl5/5.32/x86_64-linux  21942
#     Data::Dumper             any    2.174 /usr/lib64/perl5/5.32/x86_64-linux  45832
#     ExtUtils::CBuilder  0.280231 0.280234 /usr/lib64/perl5/5.32                8883
#     ExtUtils::MakeMaker      any     7.44 /usr/lib64/perl5/5.32              109673
#     ExtUtils::Manifest       any     1.72 /usr/lib64/perl5/5.32               23471
#     Fatal                    any     2.32 /usr/lib64/perl5/5.32               59021
#     Module::Build            any   0.4224 /usr/lib64/perl5/vendor_perl/5.32   35680
#     Test::More              0.88 1.302175 /usr/lib64/perl5/5.32               52697
# 
# === Runtime Requires ===
# 
#     Module            Want   Have Where                              Howbig
#     ----------------- ---- ------ ---------------------------------- ------
#     Capture::Tiny      any   0.48 /usr/lib64/perl5/vendor_perl/5.32   29656
#     ExtUtils::ParseXS 3.35   3.40 /usr/lib64/perl5/5.32               66867
#     File::Basename     any   2.85 /usr/lib64/perl5/5.32               11194
#     File::Spec         any   3.78 /usr/lib64/perl5/5.32/x86_64-linux  10571
#     File::Temp         any 0.2309 /usr/lib64/perl5/vendor_perl/5.32  118255
# 
t/00-report-prereqs.t .. ok   
t/001_load.t ........... 1/? # EUMM:{
#   'CC' => 'g++',
#   'CCFLAGS' => ' -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xc++',
#   'dynamic_lib' => {
#     'OTHERLDFLAGS' => ' -lstdc++'
#   }
# }
# ---
# MB:{
#   'config' => {
#     'cc' => 'g++'
#   },
#   'extra_compiler_flags' => ' -xc++',
#   'extra_linker_flags' => ' -lstdc++'
# }
# ---
# Config:{
#   'byacc' => 'byacc',
#   'cc' => 'x86_64-pc-linux-gnu-gcc',
#   'cccdlflags' => '-fPIC',
#   'ccdlflags' => '-Wl,-E',
#   'ccflags' => '-fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
#   'ccflags_nolargefiles' => '-fwrapv -fno-strict-aliasing -pipe ',
#   'ccflags_uselargefiles' => '-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
#   'ccname' => 'gcc',
#   'ccstdflags' => ' -std=c89',
#   'ccsymbols' => '',
#   'ccversion' => '',
#   'ccwarnflags' => ' -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement',
#   'cppccsymbols' => '',
#   'd_PRIEUldbl' => 'define',
#   'd_PRIFUldbl' => 'define',
#   'd_PRIGUldbl' => 'define',
#   'd_PRIeldbl' => 'define',
#   'd_PRIfldbl' => 'define',
#   'd_PRIgldbl' => 'define',
#   'd_SCNfldbl' => 'define',
#   'd_accept4' => 'define',
#   'd_access' => 'define',
#   'd_accessx' => undef,
#   'd_eaccess' => 'define',
#   'd_ldbl_dig' => 'define',
#   'd_ldexpl' => 'define',
#   'd_locconv' => 'define',
#   'd_old_pthread_create_joinable' => undef,
#   'd_oldpthreads' => undef,
#   'd_oldsock' => undef,
#   'd_pthread_yield' => undef,
#   'd_sched_yield' => 'define',
#   'd_setlocale_accepts_any_locale_name' => undef,
#   'd_strtold' => 'define',
#   'd_strtold_l' => 'define',
#   'd_telldir' => 'define',
#   'd_telldirproto' => 'define',
#   'gccansipedantic' => '',
#   'gccosandvers' => '',
#   'gccversion' => '10.1.0',
#   'i_sysaccess' => undef,
#   'ld' => 'x86_64-pc-linux-gnu-gcc',
#   'ld_can_script' => 'define',
#   'lddlflags' => '-shared -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector',
#   'ldflags' => '-Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector',
#   'ldflags_nolargefiles' => '-Wl,-O1 -Wl,--as-needed -fstack-protector-strong -fno-stack-protector',
#   'ldflags_uselargefiles' => '',
#   'ldlibpthname' => 'LD_LIBRARY_PATH',
#   'old_pthread_create_joinable' => '',
#   'sPRIEUldbl' => '"LE"',
#   'sPRIFUldbl' => '"LF"',
#   'sPRIGUldbl' => '"LG"',
#   'sPRIeldbl' => '"Le"',
#   'sPRIfldbl' => '"Lf"',
#   'sPRIgldbl' => '"Lg"',
#   'sSCNfldbl' => '"Lf"',
#   'sched_yield' => 'sched_yield()',
#   'yacc' => 'yacc',
#   'yaccflags' => ''
# }
# Method: is_sunstudio = 0
# Method: is_msvc = undef
# Method: is_gcc = 1
# Method: is_clang = 0
# Method: compiler_command = 'g++ -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xc++'
# Method: linker_flags = '-lstdc++'
Can't exec "g++": No such file or directory at /var/tmp/portage/dev-perl/ExtUtils-CppGuess-0.210.0/work/ExtUtils-CppGuess-0.21/blib/lib/ExtUtils/CppGuess.pm line 500.
# Method: iostream_fname = 'iostream.h'
# Method: cpp_flavor_defs = '
# //#define __INLINE_CPP_STANDARD_HEADERS 1
# //#define __INLINE_CPP_NAMESPACE_STD 1
# 
# '
t/001_load.t ........... ok   
t/002_icpp.t ........... ok   
t/010_module_build.t ... # Module::Build version: 0.4224
# ExtUtils::CBuilder version: 0.280234
t/010_module_build.t ... 1/? 
#   Failed test 'build with Module::Build'
#   at t/010_module_build.t line 19.
# Build.PL output
# ========================================
# Can't exec "g++": No such file or directory at ../../blib/lib/ExtUtils/CppGuess.pm line 500.
# Created MYMETA.yml and MYMETA.json
# Creating new 'Build' script for 'CppGuessTest' version '0.01'
# ========================================
# Build output
# ========================================
# Building CppGuessTest
# x86_64-pc-linux-gnu-gcc -I/usr/lib64/perl5/5.32/x86_64-linux/CORE -DVERSION="0.01" -DXS_VERSION="0.01" -fPIC -xc++ -DINCLUDE_DOT=1 -c -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -pipe -frecord-gcc-switches -march=native -mtune=native -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector -o lib/CppGuessTest.o lib/CppGuessTest.c
# lib/CppGuessTest.xs:15:14: error: ‘string’ in namespace ‘std’ does not name a type
#    15 | typedef std::string std__string;
#       |              ^~~~~~
# lib/CppGuessTest.xs:11:1: note: ‘std::string’ is defined in header ‘<string>’; did you forget to ‘#include <string>’?
#    10 | #include <string.h>
#   +++ |+#include <string>
#    11 | #else
# lib/CppGuessTest.xs:84:6: error: ‘string’ in namespace ‘std’ does not name a type
#    84 | std::string useless_test( const std::string& a, const std::string& b )
#       |      ^~~~~~
# lib/CppGuessTest.xs:84:1: note: ‘std::string’ is defined in header ‘<string>’; did you forget to ‘#include <string>’?
#    84 | std::string useless_test( const std::string& a, const std::string& b )
#       | ^~~
# lib/CppGuessTest.c: In function ‘void XS_CppGuessTest_useless_test(CV*)’:
# lib/CppGuessTest.c:270:2: error: ‘std__string’ was not declared in this scope; did you mean ‘scan_vstring’?
#   270 |  std__string RETVAL;
#       |  ^~~~~~~~~~~
#       |  scan_vstring
# lib/CppGuessTest.c:271:13: error: expected ‘;’ before ‘a’
#   271 |  std__string    a = std::string( SvPV_nolen( ST(0) ), SvCUR( ST(0) ) )
#       |             ^   ~
#       |             ;
# lib/CppGuessTest.c:273:13: error: expected ‘;’ before ‘b’
#   273 |  std__string    b = std::string( SvPV_nolen( ST(1) ), SvCUR( ST(1) ) )
#       |             ^   ~
#       |             ;
# lib/CppGuessTest.c:276:2: error: ‘RETVAL’ was not declared in this scope; did you mean ‘GETVAL’?
#   276 |  RETVAL = useless_test(a, b);
#       |  ^~~~~~
#       |  GETVAL
# lib/CppGuessTest.c:276:24: error: ‘a’ was not declared in this scope; did you mean ‘ax’?
#   276 |  RETVAL = useless_test(a, b);
#       |                        ^
#       |                        ax
# lib/CppGuessTest.c:276:27: error: ‘b’ was not declared in this scope
#   276 |  RETVAL = useless_test(a, b);
#       |                           ^
# lib/CppGuessTest.c:276:11: error: ‘useless_test’ was not declared in this scope
#   276 |  RETVAL = useless_test(a, b);
#       |           ^~~~~~~~~~~~
# error building lib/CppGuessTest.o from 'lib/CppGuessTest.c' at /usr/lib64/perl5/5.32/ExtUtils/CBuilder/Base.pm line 185.
# ========================================
# Looks like you failed 1 test of 1.
t/010_module_build.t ... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 
t/011_makemaker.t ...... 1/? 
#   Failed test 'build with ExtUtils::MakeMaker'
#   at t/011_makemaker.t line 11.
# Makefile.PL output
# ========================================
# Can't exec "g++": No such file or directory at ../../blib/lib/ExtUtils/CppGuess.pm line 500.
# Checking if your kit is complete...
# Looks good
# Generating a Unix-style Makefile
# Writing Makefile for CppGuessTest
# Writing MYMETA.yml and MYMETA.json
# ========================================
# make output
# ========================================
# make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
# make[1]: Entering directory '/var/tmp/portage/dev-perl/ExtUtils-CppGuess-0.210.0/work/ExtUtils-CppGuess-0.21/t/makemaker'
# cp lib/CppGuessTest.pm blib/lib/CppGuessTest.pm
# Running Mkbootstrap for CppGuessTest ()
# chmod 644 "CppGuessTest.bs"
# "/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- CppGuessTest.bs blib/arch/auto/CppGuessTest/CppGuessTest.bs 644
# "/usr/bin/perl" "/usr/lib64/perl5/5.32/ExtUtils/xsubpp"  -typemap '/usr/lib64/perl5/5.32/ExtUtils/typemap' -typemap '/var/tmp/portage/dev-perl/ExtUtils-CppGuess-0.210.0/work/ExtUtils-CppGuess-0.21/t/makemaker/typemap'  CppGuessTest.xs > CppGuessTest.xsc
# mv CppGuessTest.xsc CppGuessTest.c
# g++ -c   -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xc++ -DINCLUDE_DOT=1 -O2 -pipe -mtune=native -march=native -fstack-protector-strong -fno-stack-protector   -DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC "-I/usr/lib64/perl5/5.32/x86_64-linux/CORE"   CppGuessTest.c
# /bin/sh: g++: command not found
# make[1]: *** [Makefile:334: CppGuessTest.o] Error 127
# make[1]: Leaving directory '/var/tmp/portage/dev-perl/ExtUtils-CppGuess-0.210.0/work/ExtUtils-CppGuess-0.21/t/makemaker'
# ========================================
# Looks like you failed 1 test of 1.
t/011_makemaker.t ...... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 

Test Summary Report
-------------------
t/010_module_build.t (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
t/011_makemaker.t    (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=5, Tests=14,  4 wallclock secs ( 0.05 usr  0.02 sys +  2.71 cusr  1.20 csys =  3.98 CPU)
Result: FAIL
Failed 2/5 test programs. 2/14 subtests failed.
make: *** [Makefile:869: test_dynamic] Error 1

^ everything here is full of wrong.

( I'm explicitly setting CXX to x86_64-pc-linux-gnu-g++ in ENV, which should be enough, but it clearly is not )

I'm just gonna point out, but this was removed without:

• a deprecation cycle (so there's no way CPAN would expose this missing in PREREQ_PM)
• a deprecation notice in Module::Metadata (so there's no way one could even ask if it was about to be removed from the tool who's job it is to advertise that )

# corelist CGI

Data for 2020-07-17
CGI was first released with perl 5.004, deprecated (will be CPAN-only) in v5.19.7 and removed from v5.21.0
# corelist Pod::Parser

Data for 2020-07-17
Pod::Parser was first released with perl v5.6.0 and removed from v5.31.1

The horse is bolted on that already, and we only noticed when we hit the first package that had this problem.

But there should be a system or reminder to prevent this happening in future.

Annotated ebuild better with comments, and merged ( ** keyworded still, because I'm about to test what an upgrade does, lucky me )

https://github.com/briandfoy/dumbbench/blob/490bac1eac2e8f4cfcd832649523f8517c7ef151/LICENSE#L1-L7

https://github.com/briandfoy/dumbbench/blob/490bac1eac2e8f4cfcd832649523f8517c7ef151/Makefile.PL#L52

https://github.com/briandfoy/dumbbench/blob/490bac1eac2e8f4cfcd832649523f8517c7ef151/lib/Dumbbench.pm#L538-L540

( And in fact, all modules site the same perl5- license blurb )

Thus, it seems the error is the LICENSE file itself having the wrong content.

Thanks :smile:

What I'm considering as an alternative approach is to let Travis CI publish the build results of each successfully tested commits of the master branch to somewhere, e.g. in a build branch or in another rex-builds repository.

If you go that way, I'd encourage you to find a way of doing that outside of dzil (ie: some bash glue that calls dzil build and stashes the result itself somewhere), otherwise you'll grossly complicate the workflow for general contribution/development.

( I say this having used such a thing myself, and grown unfond of how it does this )

https://github.com/miyagawa/Dist-Zilla-Plugin-LicenseFromModule/blob/3be26293c75b31654d7d19cffdc71e40eed984fa/t/license-nonexistent.t#L3

https://github.com/miyagawa/Dist-Zilla-Plugin-LicenseFromModule/blob/3be26293c75b31654d7d19cffdc71e40eed984fa/META.json#L46-L49

https://github.com/miyagawa/Dist-Zilla-Plugin-LicenseFromModule/blob/3be26293c75b31654d7d19cffdc71e40eed984fa/cpanfile#L6-L8

Hey, sorry to duplicate a bug, https://rt.cpan.org/Ticket/Display.html?id=82815

We currently ship Data-Diver in Gentoo, and due to lack of declared/stated license, its pending removal.

I don't demand a new CPAN release, even though it would be appreciated.

But if you have the time to informally state a license you would be happy to have Data::Diver used under, as an interim solution, we, and at least one user, would appreciate not having to remove it :smile:

This repo was discovered after trying to search the internet for you, worried that perhaps you'd joined team forever-stable-in-dirt, and we were rather relived to stumble onto a github account with recent commits :smile:

Then why are we putting so much effort into doing it? :) Sadly, there seems to be no existing projects/modules that would provide functionality for dynamic or conditional configuration for Dist::Zilla.

While saying that its not generally done, Gentoo is special here. What's also not generally done by vendors is to even provide a way to deploy entirely unchecked code directly from git as part of your build chain ;)

Most vendors who would even anticipate such a thing only have the option of doing it as a snapshot, where they'd run the deploy code against a specific commit, and then ship a hard built copy of that commit with a patch-revision to their users.

Wouldn't it be more straightforward then to directly provide a Gentoo-specific dist.ini as contrib/vendor/gentoo/dist.ini, instead of ~250 new lines of a custom solution which patches out 11 non-empty lines from the vanilla dist.ini? Or perhaps even just the patch that the Gentoo ebuild may apply?

Mostly the problems there are:

• The ini file gets out-of-sync, and so you double effort to keep gentoo simply working when you add new changes that are otherwise unproblematic
• The general nature of patches aren't well suited to the sorts of mangling this code does, and the patches have a high risk of simply failing to apply ( and you'll get bitten perhaps by the specific mechanism gentoo uses to apply patches if you don't format the patch differently when you update it )

That's all this code really papers over, it implements "patch" in a somewhat-syntax-aware and platform-agnostic way, so the patcher only has to indicate the blocks of syntax they need to get rid of.

And that way, if gentoo needs different mechanisms than is already in git, its easier to patch the dist.ini patching code, than to ship a patch for dist.ini itself. ( Patches being "add/remove X line" are more resilient than patches that "augment X line" )

I honestly don't expect to see other vendors using any of this, but it should make their lives easier if they do. (eg: they just add their own patch script that utilizes the existing provided tool libraries, instead of reworking dist.ini in a more fragile way)

A lot of this is very specific to the "build sources from live git" work flow, which I should point out, is generally not done with Perl, nor encouraged in Perl.

Its not so much oriented at "package vendors" so much, but to provide you with an "out", to work around various problems that may crop up between releases while the code is still in -9999 versions.

Vendors don't need any help doing this, but doing it this way is intended to give you more control over the deployment process when people build from git sources, without necessitating you send PR's to fix the ebuild (and this avoids the proliferation of git-specific patch logic leaking into the non-git ebuilds)

So it sits right up against the vendor/upstream split, just sitting slightly on your side of the split, instead of the vendor side of the split.

Putting this code anywhere else would be a net negative, because its coupled specifically to the Rex build process, and it would also be a premature generalization, ( as nothing else needs this at present ).

And it is by very nature "hackish", in that, the approaches used are fine because we know exactly what code its working with, where a generalized approach would need to accommodate a broader rage of variability, and thus, increase its complexity.

As per the changes entry in https://github.com/rjbs/Dist-Zilla/commit/ff42620c2f0badbbb2e5042e5b5662286030239a

There should be no more dependency on Class::Load

However:

https://metacpan.org/source/RJBS/Dist-Zilla-6.015/META.json#L54

And this looks due to

https://github.com/rjbs/Dist-Zilla/blob/64dd98f9639427d89956ea547f33dca0ef2efe10/dist.ini#L40

Just pointing out for future reference, that these changes necessitate fixing the -9999 ebuild, and the ebuild for 1.12.0

I'm just gonna do the work because its not substantial.

https://metacpan.org/diff/file?target=FERKI/Rex-1.12.0/&source=FERKI%2FRex-1.11.0#META.json

The importantest change is the File-ShareDir-Install injection, which automatically turned up when you added:

https://github.com/RexOps/Rex/commit/e826f4ef7cbe56eee94baf7d346c978cbfb824a0#diff-85e47ac07ac9d6416168a97e33fa969a

Due to @Basic including [ShareDir], which [MakeMaker] reacts to automatically https://metacpan.org/release/Dist-Zilla/source/lib/Dist/Zilla/Plugin/MakeMaker.pm#L153-176

https://metacpan.org/release/Dist-Zilla/source/lib/Dist/Zilla/Plugin/MakeMaker.pm#L126-131

But its easy to overlook this problem unless you regularly do system depcleans such that "build time only deps" get pruned, as File::ShareDir::Install is a common dependency, but 100% of the things that depend on it depend on it only at build time.

This diff is enough to get it working for Gentoo's build tooling, but it needs to be better for general CPAN usage (eg: using more $Config, something better than regex for compiler type detection, etc)

--- a/t/lib/Helper.pm
+++ b/t/lib/Helper.pm
@@ -46,12 +46,13 @@ sub create_testlib {
     my $cc = $Config{cc};
     my $gccv = $Config{gccversion};
     my $rv =
-        $cc eq 'gcc'    ? _gcc_lib( $libname )  :
-        $cc eq 'cc'     ? _gcc_lib( $libname )  :
-        $cc eq 'cl'     ? _cl_lib( $libname )   :
+        $cc =~ /gcc\z/    ? _gcc_lib( $libname )  :
+        $cc =~ /cc\z/     ? _gcc_lib( $libname )  :
+        $cc =~ /cl\z/     ? _cl_lib( $libname )   :
         $gccv           ? _gcc_lib( $libname )  :
                           undef         ;
 
+
     chdir $orig_wd;
     return $rv ? canonpath($tempdir) : undef;
 }
@@ -59,8 +60,8 @@ sub create_testlib {
 sub _gcc_lib {
     my ($libname) = @_;
     my $cc = find_compiler() or return;
-    my $ar = find_binary('ar') or return;
-    my $ranlib = find_binary('ranlib') or return;
+    my $ar = find_binary($ENV{AR} ? $ENV{AR} : 'ar') or return;
+    my $ranlib = find_binary($ENV{RANLIB} ? $ENV{RANLIB} : 'ranlib') or return;
     my $ccflags = $Config{ccflags};
 
     _quiet_system("$cc $ccflags -c ${libname}.c") and return;

I have a system where:

which ar gcc cc ranlib
which: no ar in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/lib/llvm/10/bin:/usr/lib/llvm/9/bin)
which: no gcc in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/lib/llvm/10/bin:/usr/lib/llvm/9/bin)
which: no cc in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/lib/llvm/10/bin:/usr/lib/llvm/9/bin)
which: no ranlib in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/lib/llvm/10/bin:/usr/lib/llvm/9/bin)

And the correct values for each are passed either by ENV, or by implication due to having perl compiled explicitly for this:

/usr/bin//perl -V:config_arg\\d+ ... config_arg13='-Darchname=x86_64-linux'; config_arg14='-Dcc=x86_64-pc-linux-gnu-gcc'; config_arg15='-Dar=x86_64-pc-linux-gnu-ar'; config_arg16='-Dnm=x86_64-pc-linux-gnu-nm'; config_arg17='-Dcpp=x86_64-pc-linux-gnu-gcc -E'; config_arg18='-Dranlib=x86_64-pc-linux-gnu-ranlib'; ...

Most of perl is just fine with this configuration, but the way its done in Devel-CheckLib's tests is very sub-par

For starters, Helper::create_testlib is unable to handle a compiler named anything other than a very short list.

And Helper::_gcc_lib also fails due to trying to run find_binary('ar'), which doesn't exist ( and provides no way to override the ar ), and if I could get it past that issue, the same problem would occur with ranlib.

The consequence is somewhat beneficial in that it doesn't cause a complete testing catastrophe, but the downside is none of the compilation tests get run, which makes it harder to assure Devel-CheckLib works as intended on the target system.

I get the overall impression the idea was to license this under the same terms as perl, but the way its written in some of the documentation suggests a much weaker / more restricted license set.

These two indicate only Artistic or GPL ( Which may imply GPL-1 only )

https://github.com/PadreIDE/Debug-Client/blob/01b843745390b82d253756600dd2482213e13b4f/README#L53-L55 https://github.com/PadreIDE/Debug-Client/blob/01b843745390b82d253756600dd2482213e13b4f/README.md#L55-L57

While this indicates the same as perl ( Artistic or GPL-1-or-any-newer-version )

https://github.com/PadreIDE/Debug-Client/blob/01b843745390b82d253756600dd2482213e13b4f/lib/Debug/Client.pm#L1116-L1118

I think the best option here is just to fix the readme, either to use the same phrasing as mentioned in the .pm file, or using the same phrasing as used at https://dev.perl.org/licenses/

 It is free software; you can redistribute it and/or modify it under the terms of either:

a) the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or

b) the "Artistic License". 

( the important aspect here is the explicit qualification of GPL-1-or-newer as opposed to the ambiguous "GPL" )

Just pointing this out as this is a class of mistakes that can happen in dzil conversion.

Formerly it was dual licensed "under the same terms as Perl itself", and now its "artistic-2 only".

Naturally if you have the copyright holders ack, this is fine, but otherwise this might be bad.

I mock keyworded:

dev-perl/XML-LibXML-2.20.100 

For

~amd64-linux ~x86-linux

And despite one dependency ( dev-perl/Alien-Build ) not having those keywords,

pkgcheck scan

Gives it a bill of clean health

# pkgcheck scan -v
target repo: '/usr/local/gentoo'
Running 4 tests
dev-perl/XML-LibXML
  DroppedKeywords: version 2.20.100: m68k-mint, ppc-aix, ppc-macos, sparc-solaris, sparc64-solaris, x64-macos, x64-solaris, x86-macos, x86-solaris

And this is with

;~/.config/pkgcheck/pkgcheck.conf
[DEFAULT]
repo = /usr/local/gentoo
arches = alpha,amd64,arm64,hppa,ia64,m68k,mips,ppc,ppc64,riscv,s390,sparc,x86,ppc-aix,amd64-linux,arm-linux,arm64-linux,ppc64-linux,x86-linux,ppc-macos,x86-macos,x64-macos,m68k-mint,sparc-solaris,sparc64-solaris,x64-solaris,x86-solaris,x64-winnt,x86-winnt,x64-cygwin,x86-cygwin
source-arches = alpha,amd64,arm64,hppa,ia64,m68k,mips,ppc,ppc64,riscv,s390,sparc,x86,ppc-aix,amd64-linux,arm-linux,arm64-linux,ppc64-linux,x86-linux,ppc-macos,x86-macos,x64-macos,m68k-mint,sparc-solaris,sparc64-solaris,x64-solaris,x86-solaris,x64-winnt,x86-winnt,x64-cygwin,x86-cygwin
reporter = FancyReporter
checks = all
glsa-dir = /usr/portage/metadata/glsa/
keywords = error,warning,info
scopes = git,profiles,eclass,repo,cat,pkg,ver
profiles = all

[gentoo]
keywords = -UnstableOnly,-PotentialStable

By contrast, repoman full -d -e y complains

NumberOf KEYWORDS.dropped 1
KEYWORDS.dropped dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: m68k-mint ppc-aix ppc-macos sparc-solaris sparc64-solaris x64-macos x64-solaris x86-macos x86-solaris
NumberOf dependency.badindev 1
dependency.badindev dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~amd64-linux(default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+) ['dev-perl/Alien-Libxml2']
NumberOf dependency.badinexp 7
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~amd64-linux(default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~amd64-linux(default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~x86-linux(default/linux/x86/17.0/prefix/kernel-2.6.16+) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~x86-linux(default/linux/x86/17.0/prefix/kernel-2.6.32+) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~x86-linux(default/linux/x86/17.0/prefix/kernel-3.2+) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~amd64-linux(prefix/linux/amd64) ['dev-perl/Alien-Libxml2']
dependency.badinexp dev-perl/XML-LibXML/XML-LibXML-2.20.100.ebuild: DEPEND: ~x86-linux(prefix/linux/x86) ['dev-perl/Alien-Libxml2']