(For context, it will probably be useful to read Code blue and Code blue, part two. Otherwise, none of this is going to make a whole lot of sense. Also, unless you’re a Perl programmer, you can probably safely skip this one…)
I decided to go ahead and reply publicly, since this information may also be useful to other people. Also, I’ve submitted a talk proposal for the Pittsburgh Perl Workshop based on the idea of letting people who aren’t super connected to the community know about some of the improvements of the last several years (because I think you’re far from alone in your “WTF is all this shit anyway” reaction), and writing this reply is sort of research for that.
Some parts of your rant I’m just going to bypass – in particular, the stuff about deep dependency chains. This is something that the people maintaining these important distributions think about quite a lot (or at least the ones I hang out with do). I think the trade-offs they’re considering and the trade-offs you’re considering are not very well aligned – you’re looking at installation difficulties; they’re looking at not having to maintain as much code, or at organizing code across modules to maximize code reuse. For better or worse, deep dependency chains and the “I had to install half of CPAN” problem is just a fact of life in the Perl world these days. (IME, it’s generally a fact of life for any higher-level language, but that’s a different topic entirely.)
I know you said that saying “use a package manager” was missing the
point of your rant, but … geez, use a package manager already. It
will make your life better. You want to use
cpanm, which you get by
App::cpanminus. (The ‘minus’ in the name is because it
has a much smaller set of ambitions than the native
cpan client, and
especially the other alternative client, CPAN-Plus.) One of the
nicest things about
cpanm is how much less verbose it is than
cpan. For your particular use case, I think if you used
captured the output into a log file, you’d be able to easily edit that
log file down into the set of notes you want showing exact versions of
what got installed (the output is regular enough that you could even
automate that step). Here’s part of the output of running
% cpanm LWP --> Working on LWP Fetching http://www.cpan.org/authors/id/E/ET/ETHER/libwww-perl-6.13.tar.gz ... OK Configuring libwww-perl-6.13 ... OK ==> Found dependencies: HTTP::Date, File::Listing, Encode::Locale, HTTP::Daemon, HTTP::Cookies, WWW::RobotRules, HTTP::Request, Net::HTTP, HTML::Entities, HTTP::Response, LWP::MediaTypes, URI, URI::Escape, HTTP::Request::Common, HTTP::Status, HTTP::Negotiate, HTML::HeadParser --> Working on HTTP::Date Fetching http://www.cpan.org/authors/id/G/GA/GAAS/HTTP-Date-6.02.tar.gz ... OK Configuring HTTP-Date-6.02 ... OK Building and testing HTTP-Date-6.02 ... OK Successfully installed HTTP-Date-6.02 --> Working on File::Listing Fetching http://www.cpan.org/authors/id/G/GA/GAAS/File-Listing-6.04.tar.gz ... OK Configuring File-Listing-6.04 ... OK Building and testing File-Listing-6.04 ... OK Successfully installed File-Listing-6.04 --> Working on Encode::Locale Fetching http://www.cpan.org/authors/id/G/GA/GAAS/Encode-Locale-1.05.tar.gz ... OK Configuring Encode-Locale-1.05 ... OK Building and testing Encode-Locale-1.05 ... OK Successfully installed Encode-Locale-1.05 --> Working on HTTP::Daemon Fetching http://www.cpan.org/authors/id/G/GA/GAAS/HTTP-Daemon-6.01.tar.gz ... OK Configuring HTTP-Daemon-6.01 ... OK ==> Found dependencies: HTTP::Request, HTTP::Status, HTTP::Response, LWP::MediaTypes --> Working on HTTP::Request Fetching http://www.cpan.org/authors/id/E/ET/ETHER/HTTP-Message-6.10.tar.gz ... OK Configuring HTTP-Message-6.10 ... OK ==> Found dependencies: LWP::MediaTypes, URI, IO::HTML ...
So you see what depends on what. You see what versions of what gets installed. You don’t have to download everything by hand, extract it, and run the build yourself. This will make your life better.
Now, to respond to your list of “things I don’t know”…
- MetaCPAN is an alternative interface to CPAN. It uses the exact same underlying data as search.cpan.org, but I think it looks nicer, and it provides a lot of additional information in a much more accessible fashion. Particularly useful are seeing how many people have ‘++’-ed a module (which is akin to starring a Github repo or faving a tweet), which gives you some idea about the consensus around module quality, which can be useful when you’re trying to decide between two libraries that do the same thing. MetaCPAN also links to bug trackers, code repos, and shows upstream and downstream dependencies for a module. It also happens to be where the ‘!cpan’ DuckDuckGo search shortcut goes to, which is what I use for searching CPAN these days.
- I recommended Cpanel::JSON::XS as an alternative to JSON::XS. I did this before I realized you weren’t using JSON::XS directly yourself. Given that you’re only getting it as a dependency of JSON::Util, you don’t have a lot of alternatives. I am sort of interested in what you’re using out of JSON::Util, because, to me, the latter looks like a very thin wrapper around JSON::XS and IO::All – basically, almost exactly the sort of thing you ranted about earlier, that would be trivial to do yourself.
- The module is called ‘Cpanel::JSON::XS’ because it is a fork of ‘JSON::XS’ that was created by somebody working for Cpanel after they decided that the module was too important for their business for them to rely on the asshole maintainer of JSON::XS. It was something that was originally only intended for use inside Cpanel (hence the name), but ended up getting released to the public because other people were considering a public fork (because, again, asshole maintainer). So, basically, the name is an unfortunate accident of history.
- All JSON::XS does is serialize Perl data structures into JSON, and deserialize JSON back into Perl data structures. That’s it. The ‘XS’ in the name is because it’s written using something called ‘XS’, which is a way of calling C code from Perl. What that really means is, it’s fast – much faster than JSON::PP (where ‘PP’ stands for ‘pure Perl’, which means you can build it without a C compiler, because it’s just Perl).
- All the JSON::* modules have exactly the same API (and there’s even a module that will use the fast XS one if possible and the slow PP one if not, or error out if neither of them are available), so you could update JSON::Util to use Cpanel::JSON::XS and everything would – well, should – still work just fine. That said, as I said above, I bet you could trivially replace JSON::Util with a couple of your own lines of code that use Cpanel::JSON::XS. Alternatively, I found the JSON::Util Github repo, so you could send in a patch switching that over to a different JSON library. (I may just do that myself…) (Update: I did it myself.)
It may surprise you that I agree with you about treating CPAN modules as black boxes. I pretty much treat them as black boxes too – with one major difference, which is that when something in one of them breaks, then I open up the box and try to figure out what went wrong, and either send in a patch or at least a bug report. And, yes, I do know quite a lot of these people, to one degree or another, and I assure you: they’re not above your pay grade. They are, generally, just like you: people who are trying to solve problems. Some of them are raging assholes, but most of them are reasonably nice and do appreciate hearing about things that are broken or poorly documented or whatever – just like you appreciate hearing about those things from your users.
My reasons for doing this have less to do with knowing the authors of the code, and more out of a sense of obligation. Someone (often, many someones) have given me the gift of their labor, in the form of code I don’t have to write or maintain. Letting them know when it doesn’t work, and trying to fix it if I can, or at least wrap the failure up in the form of a bug report or maybe a failing automated test case, seems like the least I can do to return the favor.
Finally, as far as HTTP::FuckingSimple, you may want to investigate HTTP::Tiny or even HTTP::Thin::UserAgent (although mind the alpha API warnings on that last one).
Hopefully this was at least somewhat helpful for you. Sorry it took me so long to respond.