Hacker News new | past | comments | ask | show | jobs | submit login
Curl is just the hobby (haxx.se)
219 points by TangerineDream 9 months ago | hide | past | favorite | 86 comments



Off topic but I find it wild that SF has API ads on bus shelters. Is that unusual/right in a dev hotspot, or have I been well underestimating just how intrinsic the tech culture is there? (I’m an Australian, so I guess my question applies to the US as much as SF)


https://nitter.poast.org/JosephPolitano/status/1683476652276...

I guess it just makes sense sometimes to do weird targeted public ads in specific areas?


The same thing stroke me, I assumed it is specific to an area where there are many software engineers.

I think people mostly don't know about Curl and PUT requests, it probably would be a huge waste of money to advertize like this in the wild. Not that I don't think ads are a huge waste in general.


hi, Stytch team member here who worked on the PUT request ad from Daniel's post -- feel free to AMA

TL;DR on 'why pay to put a curl request on an ad' is what you all have already said -- (1) unique concentration of tech in SF; and (2) to specifically engage software engineers. More on each...

(1) We wouldn't have run this ad in Sydney, or New York, or LA. SF definitely has an uniquely tech-oriented culture, and in particular has lots of startups in our ideal customer persona (ICP) at Stytch - in this case, engineers building B2B SaaS apps.

But in addition to the people who live in SF, even more software engineers visit the city for conferences, or events, or to fundraise. For example, while our ads are up over the next month, SF will host Stripe Sessions and POSTCon (two entire conferences focused around APIs), plus RSA (security focused).

(2) And although even with that, only a small segment of the pop will understand the ads, those people might be intrigued enough to actually look at them - and our ultimate goal is to get more engineers to look at our code & our docs. Another perk is that engineers can't use ad blockers if the ad is on a bus shelter :)

So that was a bit of the thinking for us - on why SF & why a PUT request on a billboard. We're also making the physical ads into an anchor for a 'marketing moment' for Stytch -- so pair offline ads with digital marketing, as well. So if we're successful, maybe you'll see more on that, soon.


When in an Uber from the airport in San Francisco, I remember seeing a giant billboard for Twilio. I had a similar reaction.


I saw ads recruiting devs for a bank or something on the metro here in paris. It was a weird thing that caught me off guard but they kinda make sense since a lot of working people use public transport here. Maybe it's the same reasoning here?


I think it's a Bay Area thing.

Google's {First 10-digit Prime in Consecutive Digits of e}.com ad is a classic in this genre (https://www.hanshq.net/eprime.html)


Twilio famously had billboards that just said “ask your developer” in SF


It's very much a SF = tech thing.


Sydney Airport will frequently have some hyper-specific ads around it. This happens pretty much everywhere.

I occasionally see tech/API ads on the London Underground. You often see ads for all sorts of niches that would probably be smaller than developer and developer-adjacent people.


I've heard that, sometimes, buying ads in a prominent public place like this can be the best way to influence one or two crucial executives that you're trying to target. It can be worth the cost just for the one or two impressions.


I've travelled to SF (from Poland) and this has struck me too, together with lawyer ads.


> lawyer ads

I take it you haven't had the pleasure of driving thru Philadelphia. Some say you can even see skyscrapers behind the sea of lawyer ads.


And Florida. Most popular billboard ad is a lawyer ad.

I’m glad WA banned them.


Nope, just NYC and California.

This one got stuck in my mind: https://www.sfgate.com/local/article/anh-phoong-iconic-billb...


I remember seeing billboards for XML+EJB in Seattle in 2002... Found that unusual at the time, as a visitor to the USA.


NYC has Mullvad VPN ads in the subway.


Same in LA! I saw an entire train plastered with a mullvad ad on the outside. The ad didn't even say "Mullvad", it just had the logo and some "free the Internet" slogans. Truly bizarre.


Depends by state. Florida has lawyer ads. “Have you argued with a woman” ads.


There's a mall in Santiago de Chile that has ads for Copilot.


> I would personally perhaps protest against the use of PUT for POSTing JSON, but nobody asked me.

Just wondering, if the resource ID is known, I thought PUT is the recommended method? We usually use POST for creating a new resource for which you don't know an ID yet.


According to RFC 9110:

- POST: Perform resource-specific processing on the request content.

- PUT: Replace all current representations of the target resource with the request content.

I interpret this as the only immediate side-effect of PUT is supposed to be replacing the target resource with the request content. Everything else is POST, but that does not mean that we can't use POST for everything. Thus, JSON via PUT is not inherently odd at all. Calling a random API using PUT with JSON arguments that executes some code other than replacing a resource, would be odd.

I do think, though, that PUT may very well implicitly create a resource, if the name of that resource is the argument to the put. That is just something that's rather odd, as, often, the server has authority over the names of new resources.

I find PUT particularly helpful, as, given these constraints, I assume that PUT is idempotent; POST is not.


You should've added the patch specs too, for completions sake (it's to modify an object, not replace it - leaving unset attributes the same as before the operation)


Auth requests are for a new session ID, not a new user ID.

It's a (C)reate in CRUD, not an (U)pdate.

POST - Create, GET - Retrieve, PUT - Update, DELETE - Delete


The difference between PUT and POST is weather the command is idempotent or not.

Some APIs us PUT for creating items and let the clients specify the ID. If the IDs are UUIDs, the risk of creating duplicates is so slim that it's neglectable, and if that should happen, it's easy for the client to call again with a different ID. The real advantage of this approach is that if the API call times out, the client can just PUT again without having to fear creating a duplicate record. Recovering from a create with POST that has timed out can be pretty difficult because it can happen that the record was created just fine, but the internet connection died before the OK response reached the client.


Aren't you basically describing that you want idempotency?

The RFC 9110 (and also the old 2616) clearly state PUT is idempotent while POST isn't.

9.3.4 PUT

"The fundamental difference between the POST and PUT methods is highlighted by the different intent for the enclosed representation. The target resource in a POST request is intended to handle the enclosed representation according to the resource's own semantics, whereas the enclosed representation in a PUT request is defined as replacing the state of the target resource. Hence, the intent of PUT is idempotent and visible to intermediaries, even though the exact effect is only known by the origin server."

9.2.2 Idempotent Methods

"Idempotent methods are distinguished because the request can be repeated automatically if a communication failure occurs before the client is able to read the server's response. For example, if a client sends a PUT request and the underlying connection is closed before any response is received, then the client can establish a new connection and retry the idempotent request. It knows that repeating the request will have the same intended effect, even if the original request succeeded, though the response might differ."


PUT can be used for Create too, if the user ID is generated by the client


According to the RFC's PUT can be used to create a resource as long as the request has a clear target. That doesn't necessarily mean an ID, but just some kind of identifier (composite key, hash of content, url, etc.). When a create instead of update happens the server should respond with a 201 CREATED, which should indicate the location or identifier of the created resource.


how can you leave out PATCH?

POST - Create, GET - Retrieve, PUT - Upsert, DELETE - Delete, PATCH - Update


It's not part of RFC 9110, so why should it be included?


Because this comment chain doesn't mention that RFC. That's in a different one.


Fair enough. I lost track of the correct comment chain there.


The protocol could be:

CREATE /xxx

READ /zzz

UPDATE /yyy

DELETE /abcd


Isn't it already supported by REST? CREATE: POST READ: GET (or QUERY if body is needed) UPDATE: PUT for full update, PATCH for partial update DELETE: DELETE


Should have SET, because CREATE implies error if anything already exists, but SET would override it


Why not create an RFC and start the process? :)


Loving the nerd sniping here haha.

Join us at Stytch (https://stytch.com/careers) if you have a strong opinion so you can decide whether this should be a POST or PUT :)


I also wondered, did some research and look at some examples and came to the opposite conclusion for my API.


> Just wondering, if the resource ID is known, I thought PUT is the recommended method?

Recommended for what purpose?

The set of recommendations I'm familiar with is:

    GET:   requests without side effects
    POST:  requests with side effects
    PUT:   never use
    other: never use
POST gets special treatment from browsers for various security risks. Otherwise, methods don't differ. You can use PUT as part of an effort to feel like you and your server have a secret code, but it's fundamentally the same thing as doing `GET /path/to/resource/put`.


PUT has a big advantage in that it is mandated to be idempotent - it is safe to be repeated if it fails for whatever reason.

Meanwhile, POST is generally not considered safe to repeat in case of failure, because the client/proxy does not know where exactly the failure happened and if thus the processing has been already done or not.


Yes, that's why you use POST for everything with effects.

But if this is really your concern, you should also be using PUT to create new database records. In principle, those requests are safer to repeat than updates are, since the database will ignore repeated inserts and apply repeated updates. (Though as far as I can tell this only matters if your update adjusts a value rather than setting it outright.)


> Yes, that's why you use POST for everything with effects.

I don't get it. Isn't an idempotent effect still an effect?

> you should also be using PUT to create new database records.

If the client is free to define the identity, then I do. But if the resource gets e.g. an ID from Postgres sequence, then I need to use POST, because repeated call would create duplicates.


I don't think this is quite accurate, browsers won't treat PUT requests as safe the same way it does a get request.


PUT's can have request bodies, GETs can not.


Incorrect; GET requests are free to have bodies.

See the note here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GE...


It's still a rather bad idea. An older version of HTTP spec said that GET request body SHOULD be ignored. Proxies implementing that older version of spec could just drop the body ... You might also encounter problems with many tools not supporting it simply because the authors were not aware of it or perhaps considered it useless.


If you want a GET-with-body that actually works, use QUERY. The draft is expired, but at least it's just a non-standard method (which is fine!), not actively broken like GET-with-body.

https://datatracker.ietf.org/doc/draft-ietf-httpbis-safe-met...


Also see the first row in the table on that page.


> while curl is my hobby, I also work on curl as a full-time job. It is a business and I serve and communicate with many customers on a daily basis.

While it's not my business, I still wonder how this business looks like. Is curl sponsored? If so, is this a common thing for foundational tools alike?


If you look around, you'll see the sponsors page: https://curl.se/sponsors.html

Some sponsorship is through opencollective, which includes transparent information about donations: https://opencollective.com/curl#category-BUDGET

So opencollective has it at $280k USD since sometime in 2018, not bad, that's like half of what the average Googler makes in 1 year (except spread over 5 years).

You can also see there hadn't been many big single donations in 2020 from this post: https://daniel.haxx.se/blog/2020/01/03/curl-receives-10k-usd...


They also received €195,000 from Sovereign Tech Fund in 2022: https://www.sovereigntechfund.de/tech/curl


From Daniel's contact page:

> Just don’t send me private emails about the open source projects I participate in – unless you want to pay for commercial and private support!

So it's support basically, I suppose.


So who do I email when I'm angry at my car's screen and get lost at the "open source licenses" screen?


RMS I would presume.


He probably still works for WolfSSL [1], who I think lets him work on Curl full time.

[1] https://en.wikipedia.org/wiki/Daniel_Stenberg



Imagine having a hobby project, installed on billions of devices.

MBA: they must be making 10B$/year?

Engineer: no, this is just a hobby


A bit unfair as the main reason it is installed on billions of devices is that it is FOSS.

It's like giving to charity and then being aggravated that it was used to generate lots of money, you can't retroactively change your mind. Although they can always stop updating curl if it is that bad for them.


> A bit unfair as the main reason it is installed on billions of devices is that it is FOSS.

You are right, but I wanted to show current mindset. Redis, ElasticSearch, Mongo, HashiCorp stack were also FOSS and landscape is changing. Same could have happened to cURL


We're living at the bottom of a gravity well, so we don't pay for air.

Once upon a time, even economists thought that automation could make food, shelter, and routine health care too cheap to meter; in such a world would we then be interacting with each other —as the XIX aristocracy did?— chiefly via shared interests and hobbies?

Lagniappe: https://cepr.org/voxeu/columns/downton-abbey-effect-british-...


If a man knows how to save the world from imminent asteroid impact, how much should he be paid?

If he does it without pay but asks afterwards for pay commensurate with delivered value, how much should he be paid?

Who pays it?


At least he should not get angry comments (as mentioned in the article) about how saving the world is the job of NASA, and he had no business saving the world as an individual


Does that mean Daniel Stenberg can't use Llama3?


Shouldn't there be a comma after "business"? I read it wrong the first time.


Just in case you're not familiar with the phrase - https://www.collinsdictionary.com/dictionary/english/have-no...


No. That would not be correct, grammatically speaking.


It would be grammatically correct, with different semantics.


> https://daniel.haxx.se/blog/2024/04/22/curl-is-just-the-hobb... has a security policy called HTTP Strict Transport Security (HSTS), which means that Firefox can only connect to it securely. You can’t add an exception to visit this site.

That's aggravating. What's the workaround?

edit0: Chrome made more progress, then collided with Vodafone's shitty approximation to infrastructure which is known unsolvable.

edit1: Magic: https://web.archive.org/web/20240422091821/https://daniel.ha...

We are in trouble when some organisation manages to kill archive.org


> > https://... has a security policy called HTTP Strict Transport Security (HSTS), which means that Firefox can only connect to it securely. You can’t add an exception to visit this site.

> That's aggravating. What's the workaround?

Are you aware that you’re possibly being impacted by some kind of MITM attack?


Sounds like their ISP, Vodafone, is MITM-ing them. It's well-reported over the past many years that they do this, though whether its malicious or incompetent is unknown.


I thought this was fairly common?


It is and it's a best practice. I've never heard anyone complain about this.

Most security scanning software will ding any site that doesn't use HSTS


You don't see why Firefox refusing to connect would be annoying? I don't care whether the blog about curl is encrypted in transit or not and I do care about a forced change to chrome to see the content.


I can visit it in Firefox 123 just fine. Tests like [1] say the site works on everything from Firefox 31 to Firefox 73.

You're probably the target of a MITM attack. Or you've done something weird, like taking a job with an employer that MITMs your web traffic then refusing to install their MITM certificates.

[1] https://www.ssllabs.com/ssltest/analyze.html?d=daniel.haxx.s...


I once was talking to somebody who was a brother in law staying on the couch sort of deal.

They were complaining that their battery life on their phone just got decimated recently and kept dying over and over. I believe I was helping to troubleshoot, so I had them turn on airplane mode, he flipped it and complaining of something else annoying happening and saying oh yeah my phone airplane mode doesn't work, I still get internet. I was totally baffled, it was all very weird to me.

A little bit of time later that person got busted as a part of a big local drug bust. I'm assuming that's how they tap phones.


I'm definitely the victim of Vodafone screwing with the connection. They want me to prove my identity by giving them a card number, despite already having that because I pay for the SIM connection, but both their website and their mobile app are so poorly implemented that it's not actually possible to meet that inherently meaningless request.

It seems Firefox notices this and refuses to contact the site, and Chrome notices this and lets me override, but generally I don't see this failure mode. I wonder what is significant about this particular website.

I unsportingly separate work hardware from personal, no idea if my employer's likely MITM nonsense would have the same behaviour.

Learned something today, albeit with details missing. Oh and Vodafone employee if you're reading this? None of your tech works for shit.


Firefox connects to https://daniel.haxx.se/blog/2024/04/22/curl-is-just-the-hobb... fine. Are you connecting to a http:// URL instead? If yes, why?


But it's working just fine in my Firefox, so it sounds more like there's something wrong on your end by either security software or on the network level.


Common misconception, but Https / TLS provides a combination of gaurantees, and the one cannot work without the other:

Encrypted transit but you might be talking with the hacker on the other end == worthless.

And with plaintext transit you cannot prove integrity during transit AND also not prove talking with the proper endpoint.

In short: Browser really is warning you that something is fishy. Don’t shoot the messenger.


I think his complaint is that HSTS also prevents the user from overriding it and Firefox is complying, which I agree is a bit annoying.


Complaining about no overrides is complaining about not being able to ignore quite serious symptoms.

Firefox makes you fix the root problem.


Bingo. Primarily because I don't really mind if reading this post is compromised, but at least partly because I hadn't thought through the implications of vodafone intercepting traffic.


Can I be cheeky and ask what VF service and in what country?

I've been using their CityFibre backed broadband in the UK for about 3 years and really can't complain - £32/month for 900mbit synchronous (which mostly actually lives up) and 4g backup.

I had to call them to remove the "child safe" filters when I first got it, but that's been the case for every ISP or phone provider I've used in the last 20 years. I also run my own DNS which might help, as I've seen others complain about that, but other than those two it's been impressively ignorable.


The 4G mobile service, UK. Chosen based on mast placement. That's the filter that complained under Chrome, Firefox didn't get that far. DNS by Google, nominally, though I wouldn't be shocked if that wasn't configured successfully.


In Chrome you can type `thisisunsafe` to bypass HSTS.


vodafone has been blocking my site daniel.haxx.se for several years.

Discussed on this very site for example back in 2022: https://news.ycombinator.com/item?id=31248250




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: