
Intermodal: A new command-line BitTorrent metainfo utility - rodarmor
https://rodarmor.com/blog/intermodal
======
armitron
A suggestion, since I see this more and more especially in communities that
consist of (relatively) young people.

Don't use discord. Even if you don't agree with the spyware characterization,
there are very good social reasons to choose something else (e.g. IRC/Freenode
or Matrix or preferably both). I don't use or advise people use discord for
anything, but I don't really have any strong opinions if gamers or
youtubers/"social influencers" end up going that route (because to them it's a
means to an end for their monetization strategies). But it's a totally
different situation for open technical communities.

Going with discord you are basically isolating yourself and the community you
are trying to form from the older generations. People who grew up on the
Internet in the 90s and 2000s and for whom IRC is bread & butter. These are
also the folks that have accumulated a lot of experience and wisdom and that
you really, really, don't want to be apart from. Especially if you don't have
time to waste in mistakes that others have done and learned from before you.

These "wise veterans" are not going to come to you, in discord, but you can go
after them by signaling through setting up shop in the places I mentioned.

~~~
rodarmor
I think those are fair points.

Personally, I have found Matrix to be _extremely_ buggy, and much less usable
than Discord. And, I now have an IRC setup that works for me using IRCCloud,
but that took a while and I pay $5 per month for it.

Also, although Discord somewhat isolates one from older generations, IRC or
Matrix somewhat isolates one from a different group, and I think it would be
hard to argue that one was more important than the other. I've already gotten
a bunch of early use, along with a lot of feedback, from users on Discord.

But, I do use IRC, so I should definitely consider having a link to
#intermodal on Freenode in the readme. I sort of worry about fragmenting the
chat, and having to check a lot of chat clients tho :/

~~~
rodarmor
I just registered #intermodal on freenode. I'll also mention that in the
readme.

------
atomashpolskiy
Man, this looks slick! And I loved your rant regarding collecting vs
tagging/sorting, definitely can relate. I miss my CDs :)

It's a pity that most users are with dynamic IP and behind a NAT. This makes
sharing directly with someone via BT impractical, so people still use
email/messengers for that.

~~~
the8472
Dynamic IPs are not a problem for torrents. As for NAT: Many routers support
PCP or UPnP IGD which lets the torrent client open the port. µTP can also
exploit from simple UDP hole punching to allow incoming connections if the NAT
is full cone (alas, iptables' MASQUERADE is not, despite IETF
recommendations).

So whether it works depends on more circumstances

~~~
atomashpolskiy
What you say is correct, but note that I was talking about direct connection
to a specific person.

BT is great for "giving out to the world", but if you want to send a file to a
friend, using an email, a cloud drive link or Skype/whatever is more
practical. Intermediary storage is required for asynchronous transmission, and
even if both you and your friend are online, you still need to take an extra
steps of creating the metainfo file and sending it or a magnet link first and
then wait for the DHT announce and lookup procedures to execute, which can
take quite some time.

~~~
StavrosK
Even using public trackers (in addition to DHT) I haven't had any luck trying
to send friends my torrents. They just never discover my seed, even though I'm
conectable. It's very odd.

~~~
armitron
If you are connectable you should set up your own tracker. Even the old
python-based bittorrent command line utilities will work for this and it's a
piece of cake to get going.

------
the8472
Have you run this against a large collection of torrent files? There are many
edge cases. E.g. many metadata handlers barf on very large torrents such as
the TLMC or danbooru2019. Others have issues with non-utf8 filenames since
while it is strongly recommended it's not required (since unix doesn't require
it either).

~~~
rodarmor
I just downloaded the danbooru2019 metainfo files, and used `imdl torrent
show` to view them, which worked. (Those are some big torrents!)

I don't support non-unicode filenames, which is pretty unfortunate. I have a
hunch that it would be a little painful to add, and that time might be better
spent on other features, but I could be convinced otherwise if there was a
strong need!

------
ghostpepper
As one of the few who still maintains a personal music collection I applaud
the amount of thinking that has gone into the question of how to make doing so
as frictionless for nontechnical users as Spotify.

I like the idea of a common metadata format but I wonder what the next step is
after that. What problems will this solve that can't be solved by ID3 tags for
MP3s, and what is the benefit of having a single metadata container for music
and films?

~~~
rodarmor
I think the main benefit would be how much it would simplify software.
Software that wants to do something useful with user-downloaded content
doesn't have very good options.

To do more than just display a list of files, and maybe try to render a
preview based on the file extension, software has to implement support for
reading lots of different kinds of files and parsing lots of different kinds
of metadata in those files. (MP3, MKV, PNG, JPG, OGG, etc).

And then, because the data contained in that metadata is only semi-structured,
and varies between content types, it has to do a lot of error-prone heuristics
to do anything useful with that metadata. Like, for example, figure out that
these files are the tracks of an album, and this file is the cover art.

This is definitely possible! But it's very hard, and even then has a long tail
of issues.

If a publisher or packager could add structured metadata with a constant
format and filename to a release, then all the downstream software and
services could mechanically consume that one source of metadata, which would
simplify everything enormously.

------
jsilence
It would be nice if it would have a .gitignore like ignore file. Or does it?

~~~
rodarmor
It tries to ignore common junk files, and it can be configured to ignore
things when invoking it with `--glob !PATTERN`, but there isn't anything like
a `.gitignore` file at the moment.

Were you thinking of a `.gitignore` file that would be global? Like
`~/.config/imdl/ignore`, or on a per-torrent basis?

I'd like to add a config in the future, and adding files to ignore to the
config could be useful.

~~~
jsilence
I was thinking on a per torrent basis. The .gitignore format is well known and
there are a lot of goot templates on github et al, so using them would be a
very convenient way to share github managed directories via your tool.

~~~
rodarmor
That's a good idea. There's already a `--glob` option, which allows selecting
what to include with globs. It's the inverse of `.gitignore`, so `--glob foo`
includes foo, and `--glob !foo` ignores foo.

I think that the glob library I'm using implements the same syntax as
`.gitignore`, so it would be easy to add an `--ignore-file PATH` option.

I opened an issue to track this:
[https://github.com/casey/intermodal/issues/378](https://github.com/casey/intermodal/issues/378)

------
pzmarzly
A bit offtopic, but does anyone know how’s the adoption of BEP39 and BEP46
standards (for mutable torrents)?

~~~
steeve
libtorrent implements it, so most torrent clients should have it

~~~
rodarmor
Do clients actually expose that functionality? I haven't seen those BEPs being
used in the wild, so I'm very curious of features using those BEPs are
actually available to users in existing clients.

~~~
steeve
They don't need to enable it, it's enabled. Yes, you can put mutable data in
the DHT used by all libtorrent programs (qBittorrent for instance).

~~~
rodarmor
I might be misunderstanding, but I would expect that both BEP 39 and BEP 46
would have to be exposed to the end user in the UI, and I don't see that in
qBittorrent.

For BEP 39, "Updating Torrents Via Feed URL", added torrents with
`info.update-url` would hit a feed to check for updates. Does qBittorrent
actually do this?

For BEP 46, "Updating Torrents Via DHT Mutable Items", qBitorrent would have
to allow adding torrents via magnet link with `?xs=urn:btpk:PUBKEY`. I tried
this, and it didn't work. Is there some other way to do this?

