

Show HN: Check out my (sort of) weekend project... - davesmylie

Check out my (sort-of) weekend project . . .<p>And by weekend project, I mean something that I thought sounded like a small idea that I could throw together over one or two weekends, but ended up taking over six months. This is a bit of a long submission, but after lurking on  HN for the last few years, I thought this might be a chance to contribute something back.
I’m just going to briefly discuss why my “weekend project” took so long, and how I got where I am.<p>The url is http://dudmail.com - it's a disposable email site. Similar in intention/scope to the many existing ones out there, but hopefully differentiated a bit based on how long we keep the mail (two weeks), and by accepting attachments and optionally forwarding the email onto a personnel email address. (And of course, by being mine =)<p>After being inspired by all of you that seem to be actually getting ideas off the ground, over the last couple of years I had tried (and failed) to get two or three other application ideas to succeed. One reason they failed was that my ideas were far too ambitious for me to pull off. Their scope was so large that I could never finish them off, and I never felt they were ready to be released.  It took a year or so (or more) before I was ready to give up on each idea, so this was a huge waste of time for me in terms of not failing often/early. Identifying this large scope issue as a failing,  I thought that a disposable email site was such a simple concept with a single well defined focus (ie email) that I should be able to knock something together quickly and actually get something released...<p>Why did it take so long? There were a few reasons... The guts of the site were actually thrown together pretty quickly  - not much longer than my naive ‘weekend or two estimate’.  The bits that let me down were:<p>- learning how to render a million different types of email sanely (still in progress).<p>- trying to learn how to bend postfix to my will (achieved)<p>- learning how to make a site design that didn’t suck (sort of achieved)<p>- trying to work out why my application keep falling over (problem avoided)<p>- fear of actually releasing it / considering it finished.<p>The site was created using Ruby on Rails - it has a really simple database schema and it was all pretty straight forward. The biggest problem was rendering email - this turned out to be far more complicated than I expected. Plain text email is simple, but then you have to handle html mail, multi-part mail (text and html parts!), base64 encoded mail, mail in any one of a million different character sets, email with attachments etc etc. For html mail, you need to be able to render it within the css hierarchy on your page, without impacting adversely how the email was meant to be display. For emails in a odd unicode character set, you need to ensure you’re handling and storing it correctly for every step of the process, or the end result will not be pretty.
Rendering email correctly was one of the hardest parts of this site, and I’m still finding types of mail that I’ve not yet encountered and haven’t handled yet.<p>Postfix was a real problem as well - I’d never really dealt with it before, and what I was asking it to do was pretty far out of the ordinary. I wanted any email, coming to any email address, on any subdomain, on any valid domain to be forwarded to my Rails instance for processing. This was strange enough that fine folks in #postfix were hesitant to help me because of the 'overwhelming amount of spam' I could expect to receive. I was expecting 99.999% of emails to be spam, so this was cool.  In the end after many many hours, I was able to make it do what I wanted using virtual_alias_maps and some very generous regular expressions. Any email coming in gets forwarded to an alias, which then pumps it directly into the rails webserver as a POST request (to avoid firing up a rails instance for each email)<p>The next issue was that of design - I’ve never been competent in this area. Initially I started off with a free template I found, but I quickly grew dissatisfied with how tacky it looked and at the same time, a number of excellent articles on designing for hackers flowed past on HN. (Not the book, but I have ordered that, and it’ll take a few weeks yet to get here, so won’t be any use until my next project!). One of the articles was http://www.attackofdesign.com/improving-a-design-step-by-step-part-2/ which helped a lot. It again took a few weekends of effort, but I got there eventually. (You’ll see some strong similarities with that article in regards to the colors I chose). In the end, I’m not 100% satisfied with the design, but it’s not completely rubbish, and I don’t think anyone would care as much as I do anyway =)<p>After having getting the app working, and the design mostly sorted, I posted a few links on some app directories to try and get some beta users. This was pretty successful, and I got a few users - not many, but enough to test that things were working. The problem was that things were not working, or at least not for long. After a few hours or days, the site would slow down, and then finally fall over and I’d have to manually start it back up. The logs would have some errors about running out of memory, leading me to  assume I had a memory leak. I went through all the code with a ruby gem designed to track down memory leaks (bleakhouse), and found nothing. I switched from Apache+Passenger to nginx+unicorn and back again. I changed from Ruby 1.8 to 1.9 and even tried Ruby Enterprise Edition. I removed other apps I was hosting on the same machine, and killed all non-essential services. I tried periodical restarts of apache and postfix via cron at increasingly small intervals. None of this helped at all. I tried running monitoring software to restart the servers if things fell over (specifically monit) and this still didn’t help. It actually seemed to make thing worse as the memory wasn’t there for it to restart the services and then monit would get all messed up. In the end . . . on a hunch, I changed VPS providers (to rimuhosting.com), and the problems instantly went completely away.  Memory usage now stays well below 30% on average (about 280 meg), and I’ve never seen it go above 50%. I’ve not once had the application fall over in the two months it’s been on rimuhosting so I’m pretty happy I’ve got that situation resolved. It was frustrating though the amount of time I wasted trying to diagnose an issue that turned out to be out of my control. I am not naming the original VPS provider here. Seems it was most likely an issue on their end, but it could just as easily been something I fixed when I reset every thing up on the new provider. (And of course, having to re-install, reconfigure and test the entire rails, ruby,  capistrano, postfix stack soaked up another weekend)<p>Finally, having made the site stable, I posted out a whole bunch more links, and got a whole bunch more traffic and by and large it’s handled it. Not huge amounts, but far more than anything else I’ve done. I get about 300 hits a day over 3000 page views and about 2000 emails coming in. There’s 600+ registered users and the number is climbing reasonably fast (for me at least).  I’ve managed to get to page two of google results for “disposable email” and hopefully will manage to get a little higher yet.<p>Even now though I am hesitant to consider it finished - there’s a couple of bugs caused by a race condition that I can’t seem to track down, and I’ve just found a new email type that I’m going to have to deal with (emails with inline cid images that I think come from outlook).  However . . . six months down the track, I’ve actually managed to release something that doesn’t suck.<p>Check it out if you’d like - http:\\dudmail.com - I’d love to hear any feedback, criticisms or questions =)
======
mike-cardwell
You're displaying HTML parts inline. This is both dangerous and unreliable. A
_much_ neater solution would be to take the HTML and convert it to a PDF using
wkhtmltopdf:

<https://code.google.com/p/wkhtmltopdf/>

Then displaying the PDF to the end user. You can then generate a PNG or
something from the PDF using ImageMagick, although the PDF retains the anchor
tags and the PNG wouldn't...

The way you're doing it at the moment, there are _loads_ of ways of somebody
malicious abusing the site.

~~~
davesmylie
Hmmm. I did actually look at using wkhtmltopdf for another project so I'm sort
of familiar with it at least.

I do definitely see the concerns here, but I've not been able to think of a
good way of handling them. I'm not sure how user friendly a pdf download would
be. Chromium at least doesn't display them inline, so you'd have to view the
email, get prompted to save it to disk, and then open it up using a pdf
viewer...

Do you think stripping out all the JS tags from the email before rendering it
would be a safe enough solution?

~~~
mike-cardwell
There are so many different ways of executing script. Just stripping out
script tags isn't really enough. Here's another example which I've just tested
with your service and works:

<img src="#" onerror="alert('XSS')">

It's also possible to use SVG images with embedded javascript.

You could generate a PNG from the PDF and display that inline in the page. You
could then have a "(PDF Version with Clickable links)" link next to it?

~~~
davesmylie
That sounds more usable. I'll have a look into implementing that.

Thanks heaps =)

------
JoachimSchipper
Mailinator has a blog, at <http://mailinator.blogspot.com> \- you should
definitely read everything on there. Some things I recall reading:

\- Mailinator is designed and written from the ground up to be efficient by
someone who seems pretty good (Paul Tyma; on the implementation side, it's
completely custom SMTP-to-web software keeping e-mail in compressed form in
memory; on the design side, it "sheds load" far more aggressively than mail
servers that are intended to be reliable, necessary for the constant DoS
attacks);

\- Mailinator has lots of imitators, but they tend to find out that the cost
of running servers is higher than the ad revenue (see "efficient and cheap",
mailinator processes a ridiculous amount of e-mail and recently added its
second server _because getting that much bandwidth to a single server was more
expensive_ );

\- As I understand, this is a hobby for Paul; he's not exactly getting rich
(well, not from Mailinator anyway; he's on LinkedIn if you want an idea.)

I don't mean to discourage you, it sounds like a nice project. Just don't
expect to get rich off it, or to be done tomorrow...

~~~
davesmylie
Yeah, I know what you mean.

I'm not too concerned about costs/add revenue at the moment. The server is
relatively cheap, and used for other apps anyway.

I haven't even yet bothered to put up ads on the page - because I know the
revenue off them is going to be pretty close to zero, and it means going back
and trying to fit the ads on the page in way that doesn't look terrible.

My main goal for this was to actually start a project, finish it and actually
get some users. I've managed to achieve this (in a small way) at least.

It was pretty discouraging to fail to get anywhere with the last few projects
I attempted, so at the very least getting to this stage with dudmail.com
should carry over into some motivation for the next attempt.

(And I'm sure whatever my next project is, it will be the one that makes me
rich and famous =)

------
arkitaip
So all !sub-domain emails are publicly accessible? I actually like this idea,
sorta like a very lo-fi mailing list.

Anyways, this is very impressive to me. I've only dealt with email servers in
the most superficial way and they've always struck me as being tedious to set
up and manage. Creating a service that's based on email is pretty much a
herculean enterprise in my book.

~~~
davesmylie
Yup. Pretty much. You do need to know the inbox name they were originally sent
to. This is to prevent someone from just wgetting all the emails from id=1
thru id=99999.

(Security thru obscurity I guess =)

Thanks for the feedback =) I actually thought this would be a lot easier when
I started before I started digging into it. I guess this is probably thru of a
lot of things - they look easy enough until you have to actually start doing
them!

------
mike-cardwell
Your hosting provider has an affiliate program, you link to them on your about
page, but you're not taking advantage of their affiliate program. Check it
out:

<http://rimuhosting.com/affiliate/about>

~~~
davesmylie
Cool. I didn't know this! Will definitely check that out.

------
mike-cardwell
Your signup emails are being sent from an IP address listed on the Barracuda
Reputation Block List (BRBL) - <http://www.barracudacentral.org/rbl>

This is a pretty widely used blacklist

~~~
davesmylie
Yeah, I've had issues with this from the start. (And being listed negatively
on sites like <http://www.mywot.com>)

I'm not quite sure what's causing it - suspect maybe email bounces for
undeliverable mails could have something to do with it.

I'll have to look into how I get myself off this blacklist though . . . I'm
guessing it may one of those things that it's easier to get yourself onto,
that it is to get yourself off of!

I do find it kind of ironic that a site that's about trying to prevent spam,
gets itself blacklisted for spam =)

~~~
mike-cardwell
One thing worth considering in future is to have a separate IP address for
sending out signup emails so it's not subject to the same reputation problems.
If you grow, and people use your forwarding feature, they _will_ start
clicking the "Report as Spam" feature on forwarded email, in things like
GMail.

~~~
davesmylie
Yeah, that sounds like a good idea. I have no idea how I'd be able to
implement that from with in the same rails application though. (But I'm sure
it's possible!)

I can see I have a lot of homework to do =)

------
mike-cardwell
Can you please clarify why password lengths are limited to 20 characters when
registering.... Are you storing the plain text password rather than hashing
it? How are you storing passwords?

~~~
davesmylie
I'm using the devise plugin for rails which seemed to be the most popular
choice at the time. (<https://github.com/plataformatec/devise>)

I've pretty much just gone for the default settings, which I'm guessing must
include a limit on the length of the password.

The password is definitely hashed in the database though.

(Thanks for the all feedback you are giving too btw =)

