Hacker News new | comments | ask | show | jobs | submit login
RethinkDigg.com (rethinkdigg.com)
159 points by samkottler on July 20, 2012 | hide | past | web | favorite | 110 comments

>What they didn’t mention is that we’re rebuilding it from scratch. In six weeks.

Famous last words.


One measly article does not a law constitute.

There ARE counter-examples of successful systems re-built from scratch.

The rewrite might have killed Netscape, for example, but without the rewrite from scratch not only would have Netscape died but there wouldn't be a Firefox or Mozilla Foundation now.

(And I'd argue that it wasn't the rewrite that killed Netscape: the rewrite was necessary. What killed Netscape, and would have killed it even without the rewrite, was that the then all-mighty Microsoft decided to get into the internet game for real and build a good-enough browser. It's ironic that this good enough browser was IE6, which we know view as the worst impediment to web progress).

Also, didn't Frederic Brooks say "prepare to throw one away"?

Brooks has repudiated that claim, though. See for example the first question at: http://www.computerworld.com/s/article/350008/The_Grill_Fred...

Also, if we're going to link Joel articles from the decade before last, let me add this one which is merely ten years old, and elaborates the ideas from the previous: http://www.joelonsoftware.com/articles/fog0000000348.html

All that said, I think there are things to be learned and gained from rewrites.

To extend on this, a later article talks about how the system is written on an "in-house language" ( http://www.joelonsoftware.com/items/2006/09/01.html ) called wasabi.

> FogBugz is written in Wasabi, a very advanced, functional-programming dialect of Basic with closures and lambdas and Rails-like active records that can be compiled down to VBScript, JavaScript, PHP4 or PHP5. Wasabi is a private, in-house language written by one of our best developers that is optimized specifically for developing FogBugz; the Wasabi compiler itself is written in C#.

So to avoid starting from scratch they introduced a new language/compiler? Hrm... I question the scalability of this solution, what if every company decided to do this instead of biting the bullet and doing a rewrite?

Advanced, functional programming and basic...

Not phrases that usually go together.

Yea, what killed Netscape was not the rewrite, but IMO the cancellation of Mariner:


And BTW, on IE6, the lack of competition resulting from the killing of Netscape was why MS can get away with stagnating IE for 5 years in the first place.

Look at history: The Digg v4 rewrite was catastrophic. Not in that it didn't work (it was quite elegant engineering wise e.g. Cassandra-powered Digg buttons) but in that it took a really long time and when it was finally ready it was already behind.

Actually, when it was first released, it simply didn't work. Very few pages actually loaded, and there were errors all over the place.

> It's ironic that this good enough browser was IE6, which we know view as the worst impediment to web progress.

On the contrary, if any browser killed Netscape, then it would seem to be IE5. (Scary, eh?)

No worries, if they run out of time, there's always https://github.com/reddit/

Funny because the only "real" difference between reddit and digg is the fact that reddit has an unlimited amount of categories. That and people seem(ed) more interested in the reddit categories than the comparable digg ones.

Reddit allows adult content. That makes a big difference.

I'm pretty sure /r/gonewild is one of the top 25 subreddits.

The Digg algorithm was much easier to game than Reddit's.

I totally forgot about the big "gaming digg" scene. Damn its been a long time since I've used digg regularly.

Bury bridgades and Pedobear ascii art are parts of digg we've all partially forgotten about I bet.

With digg, there are descriptions for each link (besides the title). Might seem like a minor thing, but I still miss it on reddit.

I remember that reddit was very hard to set up. Is it any easier nowadays?

You run a single script which ends with "Congratulations! reddit is now installed"

Assuming everything works as planned.


It never, ever does.

Wasn't rebuilding it from scratch and not being able to go back what caused Digg to fall in the first place?

It's kind of rare that you see a company actually make that mistake now. But I guess Digg did.

I'm interested and curious - anyone know of any other articles on this topic?

I feel like I've come across a couple "bad" implementations of some concept (as I see it anyways), and thought the best solution would be to start over and redo it.

Are there scenarios where starting from scratch is a good idea? How do you know?

(As an aside, I don't think starting from scratch will revive Digg. I think its time has passed...)

That article is probably the most frequently cited piece of bad engineering advice in the world. Bad and unneeded advice, since most corporate environments are already extremely hostile to any kind of rewrites.

Heck, there are companies that still run DOS "servers". Engineering black holes. They suck developer time in and bend all infrastructure around themselves.

Yep often all the accrued "bug fixes" accumulated in the complicated solution are "fixes" for "bugs" like "Jim didn't know that the language standard library already had a solution for this, so he made it himself, poorly" or "Pam,Bob,and Dave each had to come back and fix Jim's naive solution by coding specific logic for the corner case they each ran into".

When the new hire Steve shows up and deletes the whole thing and uses the time-tested model of the wheel, he is not a villain, and anything to suggest otherwise is poor management philosophy that will drive Steves away, leaving you with a bad project, bad management and bad engineers (AKA 99% of every project ever). Even if by doing so Steve introduces bugs in the two-wrongs-make-a-right consuming code (which is simply a bug) and isn't polite enough to discover this on his own. The flexibility and resources to make systematic improvements like these which combat cruft and maintain or repair feature velocity are why QA - automated or otherwise - exists.

Honestly though, I don't think the article really means to cover such a scenario. There are many times that engineers decide to replace rather than understand a system which is complex-for-a-reason, this is written specifically with that in mind.

I regularly see this as I use/develop against/contribute to Drupal - a marvelously complex-for-a-reason(usually) piece of engineering that happens to be written in PHP - a language with a community that still celebrates simplicity-at-whatever-cost. "Nevermind that the theme/cache/localization/form/whathaveyou system accounts for what I need, it's more than what I need and I don't trust code more complex than I (or any single person with a single use case for that matter) could have written."

He doesn't make that clear, but it's the only sense I can make of the article given his degree of experience.

Presumably they're not rebuilding the same product!

So I doubt that applies at all.

And a Digg-type site is not that hard to build. It's light-years away from the complexity of a web browser, for instance.

Easy to build, hard to scale (although far easier than when Digg was first built). Also getting the voting algorithm right, assuming they're going to continue with that model, takes a lot of refinement.

But yeah, overall I agree. Why would they rebuild the same product? That's already failed.

Most rewrites described in that article were also intended to be substantial revisions rather than a functionally identical version with "new" code.

The project I'm working on now is at a point where it's either spend a HUGE amount of time refactoring and fixing everything or just start over. We can't just continue adding on features with the state the code is in.

The question is, which will be faster?

Right now, it's written to run on an old OS, with old versions of old languages. We absolutely HAVE to update this stuff as it's just stupid to continue programming in the dark ages (no code completion, no jump to definition, printf debugging).

But is it better to refactor the code and rewrite it slowly, or just scrap it, lose all of the old terrible design decisions, and move on.

You ask the question "Why do you think you'd make the design decisions correctly this time around?" Well, we didn't write the first version. It was given to a vendor that totally botched the job and it's our responsibility to fix it.

"We absolutely HAVE to update this stuff as it's just stupid to continue programming in the dark ages"

There might also be security issues. I've found that's sometimes a good rationale - if I've got code that will only work in PHP 4.1, and it's gotta work for the next 5 years, probably rewriting ('wholesale refactoring') is a much more prudent choice, even though it's "a complete rewrite". There will be no security patches for the underlying language - building on a platform with known and 'never-will-be-fixed' security holes isn't a good long-term strategy.

My experience says that for real legacy code (deployed and actually used) it is almost always better to slice it up and re-write and deploy sections of it incrementally. Big-bang transitions with a "point of no return" are just so expensive and risky.

Yes, you lose "conceptual integrity" and your UX may be inconsistent in the transitional time, but those are small prices to pay.

IMO, an even worse one is the MS OS/2 2.0 fiasco where MS abandoned it and the MS/IBM JDA after the SDK with a beta of it was already sent to developers. To be honest, the aforementioned JDA was not particularly good, but IBM ended up beating Chicago (Win95) by three years with OS/2 2.0, and during that gap MS used unethical tactics that was a lot worse than the JDA to attack OS/2.

What are you building from scratch.

What exactly do you think you're building?

I take "from scratch" to mean "we are totally rethinking everything!

Look - lets not b fucking morons. The ONLY thing bought was digg.com the name which has a PR/publicity pull.

You will fail - digg has sucked and will suck and even the cult of personality around kevin rose evaporated, like milk, immediately after joining Google.

The fact is that the word digg and kevin rose have represented only one thing - the desire to extract fame and fortune from outdated and poor thought out ideas.

I wish you only failure.

You're fucked.

Ironically, it was "rethinking digg" and "rebuilding it from scratch" that killed it in the first place.

Yeah, that was my first thought too.

The second thought was that ground-up rewrites are almost never a good idea to begin with. You end up throwing out whatever's good about the old property along with all the bad, and risk turning off the members of whatever user base you have left (who presumably stuck around because they saw something they liked in the old version).

I suppose the population of Digg users is probably small enough at this point that the new proprietors aren't worried too much about whether they stay or go, though...

>The second thought was that ground-up rewrites are almost never a good idea to begin with.

For a complex system, like an OS or a browser.

This is a bloody social news site, as trivial an application as it gets.

One man can write a half-decent one in a week.

And for ramping it up, even the scaling roadblocks and tricks are old news by now to the web building community in general.

> This is a bloody social news site, as trivial an application as it gets.

I'm gonna have to disagree with you there. It's a fairly complex transactional system.

> One man can write a half-decent one in a week.

No, one man can write something that resembles a social news site in a week, but building a good one that scales takes a bit longer. :)

> And for ramping it up, even the scaling roadblocks and tricks are old news by now to the web building community in general.

That one I'll agree with. Their initial scaling problems should be fairly well understood.

> No, one man can write something that resembles a social news site in a week, but building a good one that scales takes a bit longer. :)

That totally depends on your available toolset and experience.

> That totally depends on your available toolset and experience.

That's either ignorant or boastful (or both). Any decent programmer will admit that they can't build a good and scalable anything in a week from scratch.

If you count your "toolset" as an entire framework that does what you want already, then sure, maybe it's possible. Otherwise you're just being foolish.

Not really though, everyone knew that V4 was a terrible idea and they simply went forward without caring much for the community. I mean what kind of marketing research could you do to prove that using promoted stories is a good way to build a community?

"Promoted Stories" myth needs to end. It wasn't the case, there was a severe bug whereby a Regular Expression only matched RSS content. The Regular Expression acted as a gateway into the Popular Algorithm. I worked at Digg and I fixed that bug.

It wasn't noticed before launch because we echoed the v3 popular stories into the beta version of v4.

Digg was never paid for stories hitting the frontpage. And for all the flack it gets for this myth, it should have been.

Well the fact rests, the community generally believed that there was a ton of promoted stories because, you guys never told anyone otherwise or did a poor job communicating.

And im certain promoted stories did exist, I remember them clearly marked as that.

Digg ads existed that were placed in the feed at slot 3 and 14 which were introduced in v3: http://about.digg.com/blog/ads-you-can-digg%E2%80%A6or-bury

They were ads that were pieces of content. Digg was in front of this trend with promoted tweets / stories.

If you don't mind me asking, how good a job do you think John N. did? Did the buck stop with him on this engineering project?

You can find a research firm to tell you anything you want. They may have to torture the numbers more in some scenarios than in others, but there's no shortage of people willing to take your money in exchange for telling you what you already want to hear.

True true, but as we can see, that doesn't mean it'll help them succeed.


They're not off to a good start. I just spent 10 minutes filling out the text boxes on page two, and it just lost all of that info.

I hope the site works better than the survey.

I filled out the first survey then stopped at page two. I'm too lazy to fill in all those boxes. I hope they didn't lose all that "valuable" info they got from me on page 1.

Survey monkey keeps the answers from page 1

It seems to be a SurveyMonkey problem.

Really? Just tested again and it worked fine. Sorry about that. @ me on twitter @jrlevine if you're still willing to give feedback.

I just filled it out again and it worked. I think I got hosed because you changed the survey while I was taking it. :)

hah, my bad! thanks again for doing it.

Am I the only one without a smartphone? I'm still using some old, beaten up Samsung crap which allows me to talk and sms a little. I spend my whole life in front of a computer, I am still always connected.

Clearly, they are taking the "mobile" approach, it just doesn't suit me very well

It's not just your phone. I have an iPhone and an iPad (well...and another convertable netbook "tablet" thing that has been sitting in a project box for over a year), and I don't really "do" mobile.

Mobile news? Are people that attention deficient?

My favorite time to read news is my commute. I don't get your point.

Me too. Hopefully they will use a font big enough to see while I'm driving.

Yeah, I am not going to waste 30 minutes of commute time staring at the floor or out the window.

For the vast, vast majority of the country, a commute involves piloting a car.

Please don't use your phone while doing this.

I bike. Commuting sounds awful.

Just because people keep up with the news a different way than you do doesnt mean they have A.D.D.

Mobile news is as old as newspapers... this is just a different format. :)

Me as well, no mobile phone at all actually. But I just made decision to buy a WIFI only smartphone (meaning an Ipod Touch or Galaxy player) since I am connected 12 out my 16 waking hours. The rest of the time (driving, reading, relaxing) it's probably best I stay disconnected.

I'm the same. I have a motorolla razr from 2006 on an $11/month pay as you go. I've always got my laptop with me.

Their question about "how do you get your news" is very significantly missing Reddit or Hacker News as an answer.

wow, good call. late night oversight. edited the survey.

That section should have radio buttons for the rows, not checkboxes.

I wouldn't lump HN, Reddit and "Other" together since Reddit is widely read with impressive traffic. It is a significant news site.

google reader and tons of blogs and sites

Please lose the Facebook/Twitter popup. Get rid of a lot of that ad space and social media buttons. "News" should not be a major focus. Programming, gaming, science, and the arts would be a good start.

Stop quoting Silicon Valley insiders and relegate them to a corner of the site if you must place any emphasis on "tech". They did a lot to ruin Digg with the endless self-promotion. Acknowledge Reddit like they never did and take inspiration from subreddits, but make them easier to discover. Reddit has yet to do this and it is a noticeable deficiency on their part.

So what'd they buy? No team. Scrapped the old code. 500k for the name, and the ability to get a percentage of their old users to at least take a look when they launch?

They'll also get a modest press bump when they roll out the "new Digg," like AVOS did when they rolled out the "new Delicious." There'll be press interest in the potential revivial of a once-huge property that there wouldn't necessarily be if they launched under a completely new brand.

I dunno if that's worth $500K, $500K can buy a fair bit of PR, but it's something.

Yep, (they paid a lot more than that) and it's probably worth it imo. Half the battle with web 2.0 sites is getting enough users to get the ball rolling. Digg already has that, and all they have to do is "reinvent" themselves with a great design and a focus on some set of users that aren't being well served by other community websites. Personally I'd try to target groups that reddit alienates.

> Personally I'd try to target groups that reddit alienates.

Who are these groups? I'm (re)starting development of my own social news site, and I'm looking for a userbase to cater to.

You could try creating a conservative-bent social news site (as suggested here in 2008: http://mattmaroon.com/2008/10/11/conservative-social-news/). As the article mentions, it can be harder to do so since the Internet as a whole leans left, but as it continues to become more mainstream someone could maybe make it work.

There are plenty of conservative-leaning subreddits, and it's easy to create your own.

The Anime weeaboo japenese 13 year old female furries/biochemical engineers/lawyers.

I believe that market is already well covered (https://www.4chan.org/).

> Anime weeaboo japenese 13 year old female furries

Gaia Online - (http://www.gaiaonline.com/)


Again, betaworks paid 500K for domain, code (which they're throwing away), and data (and I'm not optimistic that they'll keep anything but the user accounts). What's left of the team went to WaPo, and LinkedIn bought some bullshit feature patents.

500K for an initial glut of 4million unique dailies?

Obviously that number will drop off real quick, but still.

The brand is worth a fair amount, beta works launching a new news site won't get much attention but "DIGG REBUILT AND RELAUNCHED" could get some international press and at least convince old users to have a look at the new site.

It's not that much of a loss for Betaworks considering if they do it right they'll start building a huge company if not they've lost $500k and may make a fair amount of that back from the traffic.

So they have less than 2 weeks left, sounds like they waited till now to post this to insure they can hit that deadline. How much can they really change of the outlook now, based on the survey results?

This reminds me of del.icio.us and the new team, that rewrote it from scratch. It is very similar - old product that shone in the past but was out of focus in recent years.

The rewrite didn't work right; old users were angry and new users didn't appear.

But maybe this will be better, who knows.

This happened twice. Once at Yahoo, by engineers who really didn't want to understand product decisions, and then again at the new place, by people who didn't have access to the original product decisions.

The first comment on the site when I saw it earlier today was "run reddit in an iframe."

Redirects to Reddit.com... clever. In my opinion, Reddit is kind of fun in small doses, but the groupthink and mob mentality over there is unbearable.

If any site needs a "rethink" it's Reddit. It's like a garden that's overgrown with weeds.

Unsubscribe from the defaults and your experience will be much improved.

I wonder if the name isn't tarnished enough to hold, whatever comes out next, back from being truly great?

It just seems there's enough baggage for people to dismiss the new digg offhand if it isn't named something else. What do you guys think?

This has been submitted into digg itself: http://digg.com/news/technology/digg_rebuilding_from_scratch...

They should just put up Kevin Rose's original php version.

A SURVEY. Oh this is definitely going to work.

I think we're good on digg. It had it's run, arguably way overhyped. Let it go. :)

I humbly submit that anything built in 6 weeks is going to be a disaster

Start small and iterate? If built by knowledgeable engineers, I could see it being decent. It won't be awesome at version 1.0, but what ever is?

It won't be "done" in six weeks, but "done" is a very antiquated idea.

Because its a software problem and not a community problem. Right.

Rebuilding? Why not take the working code and refine, refine, refine, continuously, so that each week or two the site is subtly better, less buggy, more usable?

Rebuilding is a lot more stimulating. I had an engineer quit on me when I convinced the team to do incremental updates instead of rolling forward with an over-engineered rebuild that would just trade one set of problems for a different set of problems.

We have no idea what the code looks like. Maybe some knowledgeable engineers took a look at it and figured it would be better in the long run to just design and build something new themselves, instead of inheriting someone else's hairy, warty code. (I am just offering ideas here, I don't actually know how hairy the code is or how many warts it has.)

Kudos that they're the new team is getting user feedback. One thing that would be sad to loose is the API - it was one of the first back in 2007.

Six weeks!? A well designed static website takes longer than that.

Be prepared to be amazed. I'm pretty skeptical of what's going to result as well.

Domain gentrification...

Hmm we already have the new Digg. It's called "Hacker News".

Good luck, chaps

"Maybe just put reddit in an iframe?" haha

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact