Hacker News new | past | comments | ask | show | jobs | submit login
Self-hosting forms, the sane way (karelvo.com)
127 points by xz18r 6 months ago | hide | past | favorite | 92 comments



I'll dare to say I like it!

N8n seems to have a pretty fine gui for configuring little pipelines, sort of alike node-red. If the author wanted to embellish & enhance what they have there's a variety of other connectors & processors they could easily snap into place. It's easy to glance at a pipeline and see what the general shape is. This high level world feels much more normative & clear than scratching together "simple" php scripts.

Ditto for sending data into nocodb. An Airtable spreadsheet/database like system, with a good gui, with form submissions being fed into a spreadsheet: it's again nicely high level. It integrates with other documents or reports, if you want. It's easy to access from the web. It's a very slick very user friendly solution that still brings a ton of power. Another huge win for a high level system.

I too had an initial WTF reaction, are you serious reaction. But it wasn't that hard to find some empathy when I tried. I didn't have to work that hard to appreciate what the post is going for, to envision what the actual usage/configuration looked like, and to see there is a pretty neat high level set of guis here that are used to program a very flexible small little pipeline. And I can see how each piece is extremely malleable by end users. That freedom to rework & reshape this system freely is really neat.

There may be good tailor made solutions that we can agree to dub as "simpler" for form handling, but the composability & flexibility of this end-user driven solution is super neat & super compelling to me. These tools are extremely generic & could be used for all manners of tasks, and that is enormously compelling, to good general systems that we can use to tackle all manners of tasks. This is a cool pick of tools to bring together.


THIS!

And the authors thoughts about why are well spelt out. Not to mention that this seems infinitely more flexible than what some other people are recommending.

Like theirs an entire thread of people somehow acting like dumping forms to a mailto: handler that the client then has to send via a hopefully configured mail client is somehow a realistic and reliable option


This is supposed to be the sane way? Certainly not! You guys know that you can use "mailto:" as form action, yes? No backend stuff needed.


Aside from having to have something to parse out the submission as the response isn’t that human readable, I think the biggest problem is that users need a mail client and requires them to hit send. This disorients people so even if they have a mail client, you end up with people not hitting submit.


There’s also the bigger issue your directly exposing an email address to web scrapers like it’s not the 90s using mailto forms is a shocking take as acceptable


This isn’t really a concern for me. I’ve had my gmail exposed to web scrapers for decades without making me regret it.

For this purpose though it’s a non-issue as I also have a contact email published on my site so people can email me. And I would create a separate mailbox just for the form.

I’m not sure why people are concerned about their email being scraped as it’s comical that any email address isn’t already on a million spam lists.


exposing mail addresses on the web is fine as long as you have semi-decent spam filters.

obsfucating addreses won't work much longer anyway https://news.ycombinator.com/item?id=38150096


I think you can register GMail and Outlook as mailto: handlers, but I've certainly never tried it.


You can. It is actually relatively hard to do though unless you are extremely motivated.

Where you have to find a setting in mac / windows as well as configure your browser (chrome) for it, by using an obscure icon in the address bar etc.. and then you can have some apps fighting for you to change the setting. And then it depends on which browser profile is currently active. It is pretty messy to say the least


You can, but many people do not do this.


I have a few qualms with this app:

1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.

2. It doesn't actually replace a USB drive. Most people I know e-mail files to themselves or host them somewhere online to be able to perform presentations, but they still carry a USB drive in case there are connectivity problems. This does not solve the connectivity issue.

3. It does not seem very "viral" or income-generating. I know this is premature at this point, but without charging users for the service, is it reasonable to expect to make money off of this?


Classic comment and perfectly captures the vibe.

I don't understand why people don't understand why making users do this weird shit (and yes, mailto: is weird although not as weird as SVN/CVS vs Dropbox) isn't going to work.


For today's lucky ten thousand, the grandparent comment is about dropbox

https://news.ycombinator.com/item?id=9224


And the "lucky ten thousand" is a reference to https://xkcd.com/1053/


but mailto is done on the client side. I am not sure everyone has a local mailto handler these days.


It is surprisingly rare. I remember working at Google even on documents targeted towards engineers many people were confused by a mailto.


If you do this, recognise that you'll have a lot of desktop users fail out because they don't have an email client set up properly.

And even when email sends, it's hard to guarantee delivery. I'd sooner set up and host an API than trust email to work in a business setting.


> You guys know that you can use "mailto:" as form action, yes?

Author mentions "a form with file upload capability".


I would guess that mailto will be great for deliverability. Since the user has already emailed you before your emails are more likely to go through to them and not get filtered as spam or promotion.

Anyone have any data / observations on this?


I had no idea that you could use the mailto: URL for a form action.


Same. How would that work? What would be the end result (email body)?


It passes all form fields in URL encoded format in the body (example, name=Billy+Cravens&state=TX)


Sounds like a really bad UX

I think if my mom was trying to submit a form, and it opened her email client with a body consisting of URL encoded data she’d probably just close the email client thinking that something went wrong. Then she’d try again and the same thing would happen again. Then she might call me, and I’d probably tell her to just forget about it and try to call them on the phone instead or give up and try another company instead.


> with a body consisting of URL encoded data

The e-mail client decodes the URL encoded data. So you actually see plain text. The encoding is only done for the purpose of passing the data from the browser to the e-mail client.


I created a form with a dropdown and a some other inputs.

The result when using enctype=application/x-www-form-urlencoded and method=post in the form html is that the body that is shown in my email client is URL encoded.

They have a different enc type that you could use to specifically make it plain text. That one is not recommended because then you're gonna have a bad time parsing out the fields that were submitted from the form.


And for reference, here is what the mail body looks like with enctype=text/plain and method=post when it is opened in iCloud mail ready to send

    cat=services
    btext=adsfasdfsdafsdf afsdfas asd fa sdf as dfs
    subscribe-newsletter-weekly-yes=yes
Other email clients might create different looking body for text/plain enc type.


One variant that seemed interesting was method=get with enctype=application/x-www-form-urlencoded

In this case the values from the form get added as headers in the email so they are not directly visible to the user

I thought that I could still add user-visible subject and body by adding ?subject=foo&body=bar to the mailto: url

For example I could then have the subject say "Web form submission", and have the body of the mail contain a description that tells the user to send the email and that the data they filled into the form will be sent along with the email.

Even that is not great UX imo, but could still be interesting.

However from my testing with Brave web browser and Apple Mail, the subject and body are not filled in for the user in this case.


Looks like it can also be plaintext encoded[1], so something like:

    name=Billy%20Cravens
    state=TX
[1]: https://html.spec.whatwg.org/multipage/form-control-infrastr...


Email body is in the “body” form field, subject line in “subject”, destination in “email”.

https://www.w3docs.com/snippets/html/how-to-create-mailto-fo...


Those parameters need to be passed in the mailto: URL, not the form, if the FORM is a POST

mailto:example@gmail.com?subject=About+your+extended+warranty


You see that in the "email" forms of for example most "contact" sites. Like, for example, here on HN, in the right end of the site's footer (on desktop), by clicking "Contact" (but this isn't a form, just a "mailto:..." link).


this was something that was more commonly used in the late 90s/early/2000s, an early internet feature, but still works to this day.

there are some niceties that have been added or maybe they were always possible - you can add a subject and message body, possibly cc etc.

i used it just last year to make an easy contact form for contacting local municipalities from a single website for my wife's NP


Sadly the best way to use this stopped working years ago. I vaguely recall in some browsers (maybe IE6 or earlier?) it actually send the submission to email directly without opening the user's email program at all.

Having to send an email with the fields prepopulated feels rather archaic by comparison, and leaves me using form scripts as a rule now.


automatically expose my email to any random marketer with a single button click? I can't imagine why anyone ever thought this would be a good idea.


but that means exposing an email address in the page source code


I don’t consider that a risk as running a web site likely already has some contact email.

I can set up infinite emails on my $30/year cpanel host so I just create a new mailbox for the form and forward it wherever I like.


Which you are legally required to do anyways in some parts of the world.


There’s a difference between a random contact adddress and one that your using for data processing and lead handling


interesting! where is this required?


Germany probably? Impressum aka imprint.


> Impressum aka imprint.

What does that mean?


> An Impressum is a statement of ownership and authorship for online and print media. An Impressum helps combat spam and disinformation by holding creators responsible for their content. An Impressum is legally required for commercial sites operating in Germany, Austria, and Switzerland.

https://termly.io/resources/articles/impressum/


If I hit "submit" on a form and I saw it start to open a new Gmail tab in my browser, I'm going to close the new Gmail tab before it even has time to finish loading. (Or same if I saw it opening Mail.app.)

I'd just assume the site was trying to trigger some kind of spam e-mail or something.

The idea that I'd fill out a form on a site, then submitting it would open my mail program, and I'd then have to hit send there, and then close my mail tab/window (not to mention exposing my e-mail address to the site when maybe I wouldn't want to), is some of the worst UX I've ever heard of.


I have a Pavlovian annoyance response to noticing that I have inadvertently clicked a mailto link, because back in ~2005 firefox would try to start Evolution. I usually only noticed the click because of the sound of my spinning disk thrashing to try to lift into memory hundreds of MB of dependencies from their rust platter slumber. Evolution generally didn't even load enough to so much as show its splash screen before I found a terminal and killed the process tree.


Craigslist does this exact thing. They give you a custom email address to email, and then you click their link and it pops open gmail.


except it's actually labeled "gmail", so it is totally expected. Unlike the random form on unknown website.


I believe the last time I've sent an e-mail was in July 2017, when I was finishing my Master degree thesis, and I was glad I'd probably never have to do it again. Please don't ruin my dream?


that email from 2017 will still be in that sent folder, waiting for you, readable and accessible on all possible platforms and form factors, when all the latest owners of the slacks, teams, whatsapps and telegrams of the world ratshit onto their users into oblivion. Ask the ex-twitterati.


Genuinely curious: what is so bad about writing an email? Do you really prefer/expect that every interaction with someone online is better to be had via an app or automated form?


Easily yes. Especially when you interact with companies the email is just a shitty gateway to their actual CRM/Ticketing Software.

Ignoring the general shittyness of email itself being plaintext or bastardized html that's destroyed the moment someone replies -- Different reply and quoting styles, emails |||||||| of every previous email in the thread. A haphazard mix of fonts, font sizes depending on the client, obnoxious signatures on every message. No one understands threads where threads in chat are immediately groked.

Ignoring all that. Unsolicited communication mediums can go die in the hell from whence they came. All communication that allows someone to message me without asking, where new identities can be minted like candy so they're impossible to block permanently. Awful. My inbox is just for password resets and spam now. Same with SMS, it's the messaging of last resort.

Being able to close your DMs to just actual humans you want to talk to is goated. Email, SMS, and my mailbox are just junk drawers ever since the marketing people got ahold of them.


While a good rant is always appreciated, I don't see how forcing people to install an app or having an online form (which will very probably ask for your email anyway) is any better. And to avoid abuse, email masking services work quite well.

It's just funny that with Communick I have a whole Discourse site setup because I was anticipating people weary of giving out email addresses, but in the end the majority of my customers just prefer to solve issues by email.

One could dream of a world where XMPP is relevant and that most clients support its HTML submission capabilities, but this is also not the timeline we're in.


What.


Well, Spivak in the sibling reply summarized the reasons perfectly.


Much simpler solutions exist like https://formsubmit.co/

And for others who use the static hosts' free tiers for hobby projects, Cloudflare provides form submissions to your static pages, netliffy forms is quite generous too https://www.netlify.com/platform/core/forms/


Except.. as the first half of the page says he’s trying to avoid relying on third party services and to self host it


Yes, they too mention that service but for work purposes. I just find it odd that this is now considered a sane way for hobby projects.

I would rather have something like this for a client who wants complete control of their data.


Netlify forms looks like it could be great, but the pricing is awful: 100 per site /month ($19+ when exceeded)

Cloudflare form plugin sets up a worker/handler, which is cheap and easy. It does require coding though, unlike the formsubmit.co link.

https://developers.cloudflare.com/pages/functions/plugins/st...


That static form plugin sends submissions to Cloudflare KV Storage, I only recently started experimenting with Cloudflare offerings so haven't tried KV yet but does it get populated in the Cloudflare dashboard? If not, then that would be another thing to build.

And about Netlify. If you are getting more than 100 submissions/month on your hobby project, I think it might be time to consider some better serious solutions ;)


> that was secure and wouldn’t give me a headache, so number 3 was off as well.

Is having a backend controller that securely writes to a DB when a url is posted to that difficult in PHP, that this 'sane' way is preferred? Isn't it the most basic of CRUD setups?

I can understand doing this because n8n has a quick way to send emails (at least that's what I assume based on this article), but I really don't understand how this over engineered solution is supposed to be the sane way.


Security in php is a headache.

Many crud apps which separate the frontend and backend have form validation and sanitization on the frontend and backend (partly by virtue of converting raw input into escaped json strings), but IIRC isn’t as straightforward in php.


Im not sure what’s so hard about doing it in PHP. Can I not just get all the data from the GET/POST-Data-Assocarray, get the form fields I want, and put them into a prepared statement to save them to the DB? What’s the vulnerability here? Maybe add an CSRF Token for extra security and I think you’re done, or am I missing something?


You are not. Its same security any other stack would do.

This “PHP security is terrible” is mostly because people remember stories from 20 year ago (when none of the cool stacks didnt even exist) and things like Worpdress the most targeted cms/framework in the world.

I have suspicion that if Vercel/next powered 60% of web than its security reputation wouldnt be great either.

PHP is flawed but so are all the other stacks. PHP is old but that doesnt mean its not being updated or up to date. If anything its boring.


Security in vanilla php using old tools is a headache.

I use Symfony and using the form component (https://symfony.com/doc/current/components/form.html) you can achieve much of what is needed.

If you use the framework as well (which is very modular nowadays) you also have security built-in (https://symfony.com/doc/current/forms.html).

But probably not as fast as a "quick and loose" approach if you don't know Symfony yet, but extendible and secure (if you do know Symfony, it might be faster than the vanilla php approach, because you can avoid much of the "generic" code, the validators, the error handling, avoid SQL and XSS injection).


> Security in php is a headache.

It really isn't.


what are the possible security problems in this case (grab all form data and stuff to database)?

The only one I can think of is SQL injection, but that's trivial to fix with placeholders.


This is so overengineered. Just a simple web hosting and php script will do it


I write PHP forms for fun but there's a very valid point the default of this is extremely manual for every form you want to build. I really like the idea of at minimum using a database that creates tables and columns as needed for a form sent at it.

At the office we have less proficient users who want to make web forms, but self-hosting the data is important to us.


Why? If you need more than five forms, invent a small DSL that consumes something like a five column CSV (form element type, label, id, something, something) and craps out some PHP and SQL for you. Maybe make the layout boilerplate configurable through a bit of simple templating.


Yeah but this is using 3 different backend services just to automate self hosted forms.

Pretty sure you can do this without 3 different services.


That's entirely fair. I don't think I'd end up using the same setup as the author, but it definitely planted some ideas.


I mean one a db.. you’d want that regardless and the others the processor.. and ones the form… you’d need those 3 in some variety anyway and technically you could drop the db if you just want it dumped to email id imagine

This is literally a blog recommending to use n8n as your processor… that’s basically it lol he’s just adding ways that can be extended too like noco and metrics


Maybe the article should be titled "Self-hosting forms, the n8n way" then, because if I was working with a dev who did this I would definitely question their sanity.

But as an article about a cool way to utilize n8n, this is fair, and perhaps even elegant.


The author clearly states he is not dev/sysadmin and just playing around for hobby - I think it's totally fine.


Everything you're describing could still be done in PHP (or another language)


Yeah, but nowadays with GPT new forms can be generated fairly quickly.


If you want your forms submissions in a spreadsheet, it's also possible using only Google Apps Script: https://github.com/levinunnink/html-form-to-google-sheet


How does validation work with the approach that the author advocates for?

Is this something "n8n" does? I've gone to learn more about it, and it describes itself as an "AI-native workflow automation" tool. What the f** is that meant to actually mean?


Or just use PHP and done.


+1 not sure how OP proposal for a hobby website form is "sane". You can solve this with a simple PHP script


It seems like an extremely convoluted way to receive what will pretty much exclusively be automated spam submissions since I’m not seeing any protection methods mentioned.

Attacks on any form on the open web have gotten absurdly bad in recent years — hope the author is using something like Cloudflare + captcha.


I understand your concern, but what about an open-source form builder that gives you control over what’s on your site?

https://news.ycombinator.com/item?id=39895960


On one of my sites, I needed to install a form with file upload capability.

Soo ... the file upload is happening via JSON webhook? Seems like this was defined as the scope and then file upload was just never mentioned again.


For those that actually want a SaaS type tool for this and don’t want to use Jotform, which is utterly horrible, I recommend Fillout, which has been a joy to use and is seamlessly integrated with a bunch of services like AirTable and Dropbox and so on.

Even if you do want to eventually build your own it’s ridiculously fast as a prototyping tool, can pre-fetch data and use conditional logic and accept URL parameters and all that out of the box.


Makes me miss the bad old days of Perl scripts and cgi-bin directories... It was, at least, simple.


A fully self hosted solution that for me is good enough and more easy is a WordPress site with the plugin WPforms (free version). No third party services used.

Update: not sure if the WPforms free version supports file uploads as the OP needs.


This is imho wrong approach. Wordpress gives you extremely flexible possibilities but its hard to maintain. Like you really need to keep it updated or you will get hacked (every wp security hole gets hyper targeted). That means trust plugin authors they will keep up to date. WP is suited for smaller/mid businesses that keep it maintained but its not hands off.

Coding php/symfony form by hand (if you can do it) might be security through obscurity but realistically it will probably just work for a decade without issues or maintenance.


Is there really no good open source form backend? That doesn't sound right.


Formbricks can do what Formspree does but open source see here: https://formbricks.com/vs-formspree


You could use Drupal and the very versatile Webform module: https://www.drupal.org/project/webform


I use https://www.formtools.org (php) but I wish there were more self-hosted options.


There are quite a few, at the top of my head: getinput.co, quillforms.com, heyform.net snoopforms.com


Jesus, I can't believe I'm saying this in 2024 but just use a php script and an sqlite file.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: