Could you elaborate on what you mean by "for a time they were the only reliable way a site could remember a visitor between requests"?
Isn't this still the dominant/primary way websites add state to a stateless protocol? What other way is there for managing se? Is there something that has supplanted cookies for "remembering" or managing sessions?
The first request to a protected page will produce an authentication prompt. Subsequent requests to the same site will automatically send the same set of credentials (in every browser I'm familiar with. This part of the spec seems to be optional ).
Using HTTP Basic Authentication, the server can track the user across different pages. All other state can be maintained on the server side, keyed to the user.
One way to handle logout (without closing the browser) is to have a logout link with a destination of "https://bad_username:firstname.lastname@example.org". I believe this causes the browser to forget the original (valid) credentials and attempt authentication with the invalid credentials. This will fail, and produce a new login prompt. Then you have to close the prompt, and close the subsequent "401" page.
So yeah, it is awkward.
In the 90's and early 00's I used to see the session token in the URL of every request.
For example, instead of:
post_with_session_cookie("/auth/api/<et cetera>", ...)
$.post("/auth/<token>/api/<et cetera>", ...)
You can mitigate some of these problems by changing the token on every request, but now your security problem is only a (massive) usability problem.
None of this is the default for any major web framework, which is probably why this style of authentication completely disappeared in the mid 2000's when people stopped rolling their own backends from stratch.
Aren't they complimentary instead of mutual exclusive?