
Go read: fast Google Reader clone in AngularJS with Go on App Engine - mjibson
http://mattjibson.com/blog/2013/06/26/go-read-open-source-google-reader-clone/
======
davidjohnstone
The funny thing about feed readers is that we actually only care about the UI,
not the underlying feed fetching and parsing and whatever else happens on the
server.

So, when I made my feed reader — this has got to be the most popular pastime
of the last few months around here — I ended up taking the lazy approach and
made a UI that sat on top of the NewsBlur API. Thus, I now have a NewsBlur
account, all the messy date parsing and everything else gets done on
NewsBlur's servers, and I access it through my own much prettier interface
that works just how I want it to work
([http://www.altfeedreader.com](http://www.altfeedreader.com)). For anybody
still thinking about making a feed reader, I definitely recommend this
approach.

~~~
bonaldi
It's not really funny — because when you actually do care about the underlying
feed parsing, you end up searching in vain for a replacement among readers
that are, as you say, lazy prettification on top of bad fetch/parse engines.

Google Reader had become amazing at getting updates virtually as soon as they
were published. I've been comparing how well the competitors do, and some of
them are more than 30 minutes later than Reader at finding and showing
stories.

~~~
simcop2387
This is likely because Google had such a large userbase that they were able to
update more people's feeds at the same time than a lot of other places like
newsblur and feedly can right now.

I also imagine it's because google was requesting the feeds more often than
others, or leveraging the feedburner system to know when they were updated and
other such things.

~~~
sergiosgc
Google Reader implemented PuSH:Pubsubhubbub (worst protocol name ever!). Most
blog platforms ping using PuSH, so the update needs no polling and is thus
almost instantaneous.

~~~
edavis
If anyone is looking to play around with PuSH,
[http://superfeedr.com/](http://superfeedr.com/) has been a joy to work with.
There's also a reference implementation available at
[https://pubsubhubbub.appspot.com/](https://pubsubhubbub.appspot.com/).

It handles all the "dirty" parts of feed reading (checking for new items,
normalizing fields) and adds near instant updates for sites that ping hubs.

When I finally get around to writing my own feed reader, I definitely plan on
using it.

~~~
newman314
Does anyone know which of the new replacement readers, feebly etc. support
pubsubhubbub?

~~~
vmialik
This is a question I have been wanting an answer myself for a while.
Especially I am interested if Feedly, Newsblur, Inoreader, is using PUSH
method

------
arms
Thank you for open sourcing this. I've been itching to dive into some
production ready Go code, and I've got a lot of interest in Angular.

~~~
drhayes9
I second this. I'm especially interested in how other people model their data
for App Engine's store.

Did you have any difficulties there?

~~~
mjibson
Yes. The data modeling was by far the hardest part of the app. If you look in
the repo you'll see my first attempt was using a separate index table, which
was convenient, but expensive. The current data model is quite nice in that
the number of users doesn't matter - we don't keep track of who subscribes to
which feed. But because of that, when getting the unread story list, we must
do 1+N queries (where N is the number of new feeds). However, since we're
using go and the datastore, all that work can be done in parallel, so it's not
too slow.

~~~
Arnor
> However, since we're using go and the datastore, all that work can be done
> in parallel, so it's not too slow.

I thought we couldn't run routines in parellel yet. Per:

'The Go runtime environment for App Engine provides full support for
goroutines, but not for parallel execution: goroutines are scheduled onto a
single operating system thread. This single-thread restriction may be lifted
in future versions. Multiple requests may be handled concurrently by a given
instance.' [1]

[1]
[https://developers.google.com/appengine/docs/go/](https://developers.google.com/appengine/docs/go/)

~~~
mjibson
That description is misleading. All it means is that one go routine is
executing at the same time between all requests. Up to 10 requests can be
processing on any go instance. However, most of the time the go routines are
just waiting for data to come back. Hence, we can fire up a bunch of go
routines to do some work. It's the parallel vs concurrent stuff that Rob Pike
has spoke about before. go app engine isn't parallel, but it is concurrent.

------
webwanderings
Looks good and simple. But I am still sticking with InoReader. Even though
Feed readers should be simple but they should still provide enough bells and
whistles for the people to adjust settings (look and feel) according to their
own taste and not the taste of the developer. Below I see a comment which
suggest that you only prefer to use Google to sign in. Well, most people won't
agree and they'd rather you provide your own sign on system. In any case, the
same logic works for UI adjustments etc. For all these reasons, InoReader by
far is the closest you can get to Google Reader replacement.

PS: I have only recently discovered InoReader and I have no stake in it but I
have been playing with many RSS readers as I am a long time RSS user of GR.

~~~
jdewald
Login-wise I feel exact opposite... I became quickly annoyed when I need to
create yet another login. I prefer seeing "Login with Google"

~~~
logic
[http://www.mozilla.org/en-US/persona/](http://www.mozilla.org/en-US/persona/)

------
bsaul
Not a Google Reader user (i used iGoogle until they discontinued it on
smartphones...), but it's really a great site, congrats. I'll probably go get
a look at the source since i'm an angular and app engine user (and not really
serious with go...yet).

One small question : is there a reason why you need to go to the top right
menu to add a new stream, instead of just a "+" at the bottom of the left pane
list of feed ? Is that what google reader required ?

~~~
mjibson
You can just press 'a'. And it's because I didn't have enough time to make it
better, and that's the first place it went.

------
mtgx
I actually like it, even though it seems to have fewer bells and whistles than
Feedly, but I think it's even a bit faster, which is surprising, since most of
the others readers I've tried have been slower than feedly.

------
liyanage
I tried it, looks nice and works well.

One feature I miss from Google Reader is the ability to hide feeds that have
no unread items from the sidebar completely.

~~~
mjibson
I like that idea. I've made an issue for it:

[https://github.com/mjibson/goread/issues/9](https://github.com/mjibson/goread/issues/9)

~~~
onli
Just wanted to suggest the same.

Anotherwish of mine would be to not set a font size. It's all a bit too small
for me. Sure, I can set a user-style, but it's easier if the site behaves
right by itself.

One other thing: In the list of entries, the missing footer below an article
is a bit irritating right now. Maybe it is just new, but I think it might be
helpful to add a margin-bottom to .story-content.

Good work though, it is indeed fast.

------
alex_doom
Christ on a cracker. This is exactly what I've been hoping someone would make.
Loaded all my feeds, without crashing. Keyboard shortcuts and working folders.
And everything happens when you click on it, not 3 seconds later.

Where do I give you all my money?

------
mseepgood
Wow, this is great! Login with Google account, import data from Google Reader
worked seamlessly, clean UI. Thanks!

~~~
tracker1
Agreed, my only nit to pick is I can't seem to re-view old/read articles.

~~~
mjibson
Click on a feed, then change the "X unread items" dropdown at the top to "all
items".

~~~
bitsoda
This works, but it resets back to the 'no unread items' view if I reload the
page.

~~~
mjibson
Yeah, that's not saved as a preference yet. Some day...

------
xbpx
Good work on the UI - it has that clean responsive feel I really enjoyed about
Google Reader. It took about 90 seconds for my 200 or so feeds to transfer
from Goog. This is during the period this article is trending on HN - that is
about an hour faster than Feedly managed when it was under load.

I just logged in with my two year old Samsung Nexus and it ground to a halt
and took down Chrome. Hopefully the mobile experience is given a high priority
- more and more of my feed consumption is doen mobile.

Overall Great work!

~~~
mjibson
Thanks. I'm glad it's scaling - that was one of the goals. Yes, a high number
of stories will perform poorly. Doing infinite scroll like google reader does
is on my todo list, which will fix those problems.

------
ElGalambo
First impression: Excellent. I see you got some inspiration from a certain
Userscript and I applaud you for your choice. You managed to save lots of
whitespace.

I have only one criticism with the UI. You removed the toolbar normally found
in the end of each feed item. This is aesthetically pleasing but it introduces
a couple of usability issues: 1) I sometimes accidentally click on an item and
want to keep it unread, but the 'Mark as Unread' button is unavailable. 2) In
full view, the only separator between feed items is a thin grey line. Please
make it thicker and darker.

I have noticed a problem with all Readers that I hope you could address. Some
XML's only show the latest item. If the reader doesn't update quickly enough,
it would miss some items. I would appreciate it if you somehow made your
reader update these problematic feeds more often.

Edit: When I click on an item to mark it as read, the only indication my click
registered is that the unread count becomes reduced by one. Please show more
prominent indication that the item is read.

------
xbpx
If Reader parity is the goal then there seems to be a feature bug. Audio feeds
like the New Yorker Comment Podcast and Science Magazine Podcast do not have
audio elements in the feed. Though clicking on the title does bring up a
download dialogue for the media file. Is this a feature that will be
implemented in the near future? G-Reader was a great podcast tool.

~~~
mjibson
Yes, I'd like to add that. I've added an issue for this:

[https://github.com/mjibson/goread/issues/11](https://github.com/mjibson/goread/issues/11)

------
swdunlop
Very nice.. I've wanted a nontrivial example of a Go webapp -- the ones I
write are strictly JSON API's that lurk behind our presentation UI.

You can see the actual dependencies used by hitting godoc.org:

[http://godoc.org/github.com/mjibson/goread/goapp?imports](http://godoc.org/github.com/mjibson/goread/goapp?imports)

~~~
bdonlan
Actually building and running the thing, though, seems to be quite tricky -
app engine forbids importing syscall, but half of the dependencies of this app
seem to import syscall.

~~~
bradfitz
I think you're doing something wrong.

Are you using the Go App Engine SDK? Then you just run dev_appserver.py $DIR
and it runs.

------
6cxs2hd6
Clean, fast. I like.

But I miss some shortcut keys. Especially SHIFT+J and +K to navigate among
feeds. (Bonus points if you do it the GReader way -- keep moving the highlight
among feeds until J or K is released, and only then refresh to show that last
one as the new feed).

------
stevewilber
This is perfect. It was incredibly easy to move over from Google Reader.

If you decide to run some simple ads, I would be happy to pay for an ad-free
version. I'd say this has potential to do a lot more than just pay for itself.

------
mseepgood
I would like to have a marker on the left pane to see which feed is selected.

------
human_error
Is this how Google Reader used to look like? I'm asking this because UI is
very similar to CommaFeed
([https://www.commafeed.com/](https://www.commafeed.com/))

~~~
mjibson
Wow. I've never seen that. It does look uncannily familiar. Bootstrap will do
that.

------
xweb
Love it! Signed up immediately, imported my Google Reader no problem, and so
far very quick in use. I've tried a couple others over the last few months but
this is, by far, my favorite. Bookmarked.

------
errnoh
Thank you.

I've tried bunch of possible candidates to replace Reader and this is the
first one that actually feels responsive enough for daily use.

And the fact that it's open source and written in Go is just icing on the
cake.

------
mrcasual
Fast and simple, which is good, but it took me less than a minute to look at
it and move on. Some of the key reasons:

* Impossible to reorder feeds on the fly just like in Google Reader. That's the #1 must-have feature for me.

* Inability to tag/label stories AND feeds.

* No integration with Instapaper and sharing services.

* Inability to star/like stories.

I couldn't care less about shortcuts and responsive design. Google made it
easy to organize feeds and share/archive/search stories. That's what others
have been unable to replicate so far.

Good luck! :)

------
codereflection
Looking through your commit history, looks like you've been working very hard
on Go read since you started in March. It looks and feels fantastic.

Thank you for going OSS with this project!

------
lnmx
I checked out the source; what's the trick for getting the dependencies into
the tree?

"go get" is not supported in appengine [1], and if I manually check out
individual projects, I end up with extra code that will not compile in the
appengine environment.

[1] [https://groups.google.com/d/msg/golang-
nuts/8BJ0fxrWZd4/PXyI...](https://groups.google.com/d/msg/golang-
nuts/8BJ0fxrWZd4/PXyIppvzXJ8J)

~~~
dsymonds
"go get" _is_ supported in App Engine, but only the download part, since the
package will be built with your app (either through the dev_appserver, or when
you upload it).

go get -d <package>

~~~
lnmx
Thank you for the clarification, David.

I also had a problem getting the app to upload: "appcfg.py update" would hang
during the build phase (may be Windows-specific):
[https://code.google.com/p/googleappengine/issues/detail?id=9...](https://code.google.com/p/googleappengine/issues/detail?id=9554)

------
rebelnz
Very nice. You mentioned the 'moving to folders' is coming soon? That would be
great! Also - would it be better to make the add subscription easier to find?
I was unsure where it was at first. Although I would use the keyboard shortcut
'a' now the average user may be inclined to look for an icon or something more
obvious? Anyway - great job - just my 2 cents.

~~~
mjibson
Yeah. I added a big button.

------
Glench
You know, of all the google reader clones, I haven't seen any implement my
favorite feature, which is the 'gu' keyboard shortcut to start typing a feed
name and 'enter' to go to that feed. Anyone know of readers that have this
feature? Bonus points if it's a fuzzy search instead of an exact search.

~~~
conesus
NewsBlur has this, just hit 'g'. It's also fuzzy and allows you to make typos,
yet it keeps the last useful results on screen.

------
heywire
Am I really the only one who still goes to all of my favorite sites to read
articles rather than having them in a feed? I had tried several times to see
what all the fuss was about with Google Reader, and I just didn't get the
enjoyment out of the presentation there compared to the sites themselves.

~~~
arafalov
Depends on use case. If you have websites that only publish every couple of
weeks (or several times a year), you want some sort of subscription to not
miss what they have to say (e.g. new version announcement).

~~~
heywire
Good point, that is not something I had thought about.

------
joeblau
Would there be a problem if Google just open sourced it's previous platform?
They could open source the project and lots of developers could put their own
take on the platform. It's probably simple to install on GAE (which they can
monetize) and they wouldn't have to maintain the reader.

~~~
icebraining
They said in the past that Reader used a lot of internal APIs to fetch,
process and store the feeds, some of which may not be exposed even on GAE.

------
7hny
Please consider changing trigger point value for collapsing sidebar. Default
Bootstrap trigger point value is an obstacle while browsing in full HD split-
screen mode (two windows at once).

Feedly is also collapsing sidebar to early, this is a small thing, but causes
a great deal of disruption in such use case.

------
nsns
Isn't the main lesson from Google Reader's being discontinued, not about UI or
features, but rather about continued support? The main "feature" for a new
reader should be it's ability and commitment to remain viable for the long
term.

~~~
mythz
It's Open Source, so you're free to host it locally or go with anyone else who
wants to pick it up and host it themselves.

------
_nb
Great work! I think this might finally be the one (and not a moment too soon!)

Is there any way to categorize new feeds? It imported my previous
categories/folders just fine, but I don't see any to move new feeds to
categories/folders.

~~~
mjibson
Not yet, but soon.

------
rkrzr
I'm having trouble with the import from Google Reader. It was taking very long
(>1h) and now it seems to have only imported a few feeds. I did have a lot of
feeds, so maybe that is a problem?

~~~
mjibson
Yes, having a lot or large feeds was buggy. That's been fixed just now.

------
1st1
Awesome, finally I'm not so scared of July 1st. Would be great if posts were
marked as read on scrolling, without the need to click on them or hit the
"space".

------
percentcer
Am I the only person who uses g-u? It's been notoriously absent from almost
all these "revival readers" I've tried, save feedly.

------
fudged71
Hell yes! This is just what I needed. The other ones had strange UIs or were
too slow for me. This was basically just as good as Google Reader!

------
smmnyc
I'm wondering what your experience was like with AngularJS? Pleasant or
difficult? This seems like a perfect app for it. Congrats!

~~~
mjibson
Pleasant. This is the 4th or so angular app I've written, so I've become
somewhat familiar with it.

------
ffog
Love the simplicity, this is the perfect reader for me.

However, it seems down right now. Just getting "loading..." Anyone else?

~~~
mjibson
Just reload. The datastore sometimes errors out and there's no retry logic on
the client. It's also possible your feeds have triggered some new bug.

~~~
games
I'm having the same problem as ffog and have tried several different browsers,
reloading, clearing cache, etc.

------
mythz
Fast, clean and Simple with keyboard shortcuts - nice alternative to the other
Reader replacements out there.

------
roldie
Nice. Very fast. Can't wait for sort by oldest first and ability to
star/import starred items.

------
ekanna
Excellent! In just few clicks i was able to migrate from google reader to
this. Nice integration.

------
mjibson
I'm seeing errors from users with giant feeds. I'll try to fix those within a
day.

------
fdb
Looks great. One remark: links should really open in a new window
(target=_blank).

------
o0-0o
Maybe I humbly suggest, or be pointed to the search function in this product?

~~~
mjibson
No such feature yet. Sounds useful.

~~~
srik1234
Search is the main feature that I find it extremely helpful with GReader. I
usually search for something in my GReader, before going to the web search.
This should be at the top of priority list.

------
abdullahkhalids
Will test it when I can signup without depending on google.

------
foodbaby
Real nice. BTW, any idea how the HN RSS feed is ordered?

------
drivebyacct2
> The number of date formats we encountered is comical (so much so that I
> started a blog to document them). At current count, there are around 60 date
> formats I’ve seen.

Ha! Is that really not in the spec for Atom/RSS/RDF?

~~~
mjibson
I believe it is in the spec, but just not followed.

~~~
drivebyacct2
But of course. Who approaches a problem like "I need to serialize dates into
this standardized document format" and decides to go with "Let's concat
strings in our own format" instead of going with "Let's check the spec and use
someDateLibrary.dateToString('that fmt string')? (Or better, something like
Go's time API, but that's beside the point)

------
trebor
I would use it if I didn't have to use a Google Account to sign in. Why not
Mozilla Persona?

~~~
mjibson
I have the opposite reaction: I only want to use my Google account to sign in.
If a site doesn't have that, I'm less likely to use it.

~~~
callahad
Would you be more likely to use Persona if it acted as proxy to your gmail
account, as it does on [http://beta.123done.org](http://beta.123done.org)?

~~~
mjibson
I have no idea what persona is or what it's for. I'll look into it at some
point and see if it can be integrated.

