Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Polar – an offline web browser with annotations and tagging (getpolarized.io)
382 points by burtonator 8 months ago | hide | past | web | favorite | 69 comments



I thought you guys would like this.

I've mentioned it a few times in comments on HN and you've provided a ton of valuable feedback.

Basically, it's like an IDE but for books and web content.

You put all your documents in it, tag them, and you can read them directly within Polar. You can annotate them, add comments, create flashcards, create highlights, etc.

The flashcards sync to Anki and you can view all your annotations on the sidebar.

I wanted it to be similar to an IDE where you have everything at your fingertips.

I also wanted to add support for "incremental reading" where you can suspend and resume a document where you left off before:

https://en.wikipedia.org/wiki/Incremental_reading

... and it keeps track of your reading progress.

It's also very hackable. It's based on Electron and React so if you're a developer you can extend it easily enough.

I'm also going to write a plugin API so that people can add 3rd party extensions.

Anyway. Hope you guys like it.


I _cannot_ wait to start playing around with this. It really does seem to tick off all the boxes I've been looking for. Do you have any plans / strategy / tips for handling video? I tend to watch a lot of conference talks and lectures on YouTube, and find that I'd like to take some notes while watching. Having a single personal knowledge base handle that (or at least the notes aspect) instead of a one-off solution for other types of media would be useful to avoid fragmentation.


> I _cannot_ wait to start playing around with this.

Love the enthusiasm!

> Do you have any plans / strategy / tips for handling video?

Yes. Working on it now.

The next revision will actually handle capturing video and we just show an embedded player now. So at LEAST you can use Polar to keep track of the videos, tag them, etc.

I'd like to make a full player including annotations too so you can keep track of the pointer in the video so you can jump to the time where the annotation/comment was kept.

Flashcards too.

With videos becoming 1 hour for a lecture having them in something like Polar is critical.


Something like this would be great for video, and I've been watching a lot of talks in video form lately.

Generally, though, I'm watching talks on iOS devices... bouncing between devices makes everything more complicated.


I've already got dedicated eBook readers and article stashers (Pocket, usually), but I can see an excellent use for this: Gamefaqs walkthrough companion.

Right now if I want to play through a game with a walkthrough (usually HTML or text only), I load it into my tablet and leave it in a browser window. This is an inefficient way of doing things, because the page generally sits there for about a month as I'm slowly progressing through the game, and very often the browser will forget its place if the tab loses focus or if the tablet goes to sleep.

This will also probably be much better for going through Programming Books than my normal e-reader app.


Exactly ... this is what incremental reading is designed to solve.


"Send tab/link to Polar" Firefox and chrome extensions could be useful.


Very nice.

I feel like there was a missed UI opportunity with the pagemarks and annotations: When you add your page marks, the percentage bar at the top increments, but there is no easy way to jump to where you were last reading. Likewise, if you add multiple pagemarks, if e.g. you've read multiple sections, there is no way to know this from the progress bar.

Instead of highlighting the progress bar at the top, I would just highlight the scrollbar on the right. This would (1) free up space at the top, (2) make it trivial to jump to where I was reading last, because I would just click on that spot on the scroll bar, (3) make it obvious if I have marked multiple page marks, and (4) open that pattern up for other uses, such as marking on the scroll bar where I have made annotations (which are presumably very important passages).

I am thinking of the scrollbar in an IDE. For instance, in VS Code, I can see all the lines with errors etc directly on the scroll bar, and I can click on those marks to instantly jump to that section.


> Likewise, if you add multiple pagemarks, if e.g. you've read multiple sections, there is no way to know this from the progress bar.

oh.. I think we could solve that by migrating to another visualization. I'll have to mock it up.

I had thought of this but of course Polar is very MVPish right now.. but getting to the point of being a real product her soon.

The problem with highlighting on the right is that it's hard to do multiple columns but maybe I could have a full page visualization on the right.


Very nice. What happens when two clients open the same document library at the same time? Do they only overwrite content if you add comments etc?

We created a web based multiuser web annotation and discussion system, for inline annotations around ‘99-00, then the dotcom bubble burst and we had to shut down. I have been looking for something along those lines since then. There have been several attempts by others but it seems that the alternatives (PDF, screenshots, email etc) are hard to displace.


> Very nice. What happens when two clients open the same document library at the same time? Do they only overwrite content if you add comments etc?

Polar is a single instance app. You can't open multiple docs at once. It will just re-focus the window you already have open.


I think they are asking what happens if another program edits the index while the Polar program is running, which still might happen if the library is stored in something like Dropbox and multiple computers have it open at the same time.


Oh.. Right. Right now it wouldn't be good. My plan was to implement filesystem events but unfortunately the Electron scalable version of filesystem watching isn't actually supported anymore.

I think I might be able to re-implement by checking only the files that are open.

That said. I'm planing on implementing cloud support natively in Polar. Offline-first and always but we will have cloud sync.


Thanks for the answer. Yes I have something like Evernote open on multiple devices at once. Not a killer feature, mostly used on a phone or table and a computer. Not applicable yet for this, but was curious.

Keep up the good work!


Very interested to start messing around with this.

Regardless of whether it ends up being something I start using, I'm very glad to see people working in this direction. Thanks a ton for sharing.

One question - since this is Electron, it's based off Chrome's engine, right? Does that mean when you talk about adding 3rd party extensions you're including Chrome extensions in that? Having a hackable browser that I can directly mess with gets rid of the need for me to have a lot of extensions -- but there are a still a few (uMatrix, uBlock Origin) that would be a huge benefit. Even when I'm just downloading a document for offline access, I still don't really want to have a bunch of trackers in the stuff I access.


> Does that mean when you talk about adding 3rd party extensions you're including Chrome extensions in that?

Alas, no.

Electron doesn't support Chrome extensions.

I agree that extensions would be awesome but Electron won't ever get extensions while it's based on libchromiumcontent.

NW.js (an Electron alternative) does support extensions though and if Electron drags its feet I might consider porting Polar to NW.js. That might be a herculean effort though.


Since no one mentioned it: is it possible to create mind maps. I'm not sure whether it's a good feature but I'm curious.


Very nice. I'm filled with enthusiasm about this. I've been meaning to "scratch" the "itch" re: offline browsing so that I can clear out the hundreds of suspended browser tabs that I'm toting around. This looks great. (Now, I just need to do, like, 10 tabs / day.)


I never considered how useful such a system would be for me; couldn't care less about anki and incremental reading: just having a binary done/not in a compressed view alone is huge. Combined with the webpage capturing aspect for archival and view.. And a usecase for electron that isn't purely developer-convenience? This is the kind of thing that makes days

I've got some ~150 tabs I'm holding onto just because I'm not ready to archive them into bookmarks yet, just waiting to be moved into this thing. If this thing works as well as I'm imagining, this is really going to fix my shit (and probably save me tons of ram)

Edit: A couple issues I've noticed from a few minutes of usage however, at least from my interest of using it as an archive: there doesn't appear to be a proper way to retrieve the original url of a document (important for sharing, especially to non-polar users), and its not clear if the polar archival format stores it in the first place. And ofc, a convenient way to find the polar archive, for sharing to polar users

There also doesn't appear to be a proper way to edit & save a polar archive (important for archiving; cleaning up). You can't edit properly pre-fetching the webpage, because of the webview/js trick you're doing, and you can edit but not save when viewing the archive post-fetch. It would also be a useful bandaid until readability or something is attached properly.

Since these all appear to be absent from roadmap, I'm wondering if the project has (my) interest as a shareable archival system, or is it intended primarily for personal usage? In terms of my own interest, I'd be looking to replace bookmarks & tabs with this: the offline archive has both value for speed/convenience, but also the defense against site-decay; but bookmarks/tabs also operate as my reference for sharing, and the site would be preferred, with polar second, and webarchive last.

With the current (roadmap-planned) setup, I'd need to maintain both systems: bookmarks for (slowly) decaying, shareable archiving, and polar for personal, long-term archiving/reading. And probably a third system for proper, shareable, long-term archiving (ie webarchive).


Wow, looks great, polished. Annotations are a wonderful extension. I've been using iBooks for pdf collecting, but this should blow that out of the water, with the cross-platform part being especially useful.

Thanks for the hard work!


I was soo looking for a tool like this. I considered Zotero which is much too research-focussed for me. I really appreciate the annotation support and Anki integration! Thank you :)


Speaking as a technical writer, this look like it could have a lot of applications for me. The annotations are wonderful. An IDE for books and web content is a great concept that I never knew I needed.


I love you. This is the direction I always wanted the internet to go - tools that provide a more interactive/active experience with what I'm reading.


This looks really amazing! I'll have to try this out with documentation, and it would likely be great for html books as well. Thank you!


Does support RSS feeds as data sources?


No.. not yet but it's an interesting idea to support updated sources.

We're going to ship a chrome extension to sync with Polar and I'd like to have 1 click button for Feedly, Hacker News, Reddit, etc.


Are there plans for OCR?


Yes but not in the short term. I'm considering some cloud integration so we could farm off some tasks to OCR and NLP libraries in the cloud. Some of this is too complex for the desktop though.

Some of these features I would have to charge for though as there aren't OSS alternatives.


I'd ask that you consider local alternatives, or at the very least make it opt-in. It would be very annoying to have some awesome open-source document manager send off my documents to the cloud.

There are some surprisingly powerful OCR libraries you could run locally, Tesseract.js not the least of them.

Cheers, and awesome project!

http://tesseract.projectnaptha.com/


Very interesting direction. I like people taking their web browsing and realizing its a serious thing for a more serious set of tools then we currently have.

I'd like annotations to be the way forward for the web.

Just as a thought experiment, this comment I'm making.

Where does it go to live permanently? Why is all this typing I'm doing ephemeral on my computer?

Why can I make comments all over the web and it's very likely I have no way of tracking the fact that I made the comment longer term (even if I spend 30 minutes typing some instructions or help for example).

No regular way of hearing responses to it?

No solid way of copying it or morphing it for another output other then laboriously (possibly having to google my own comment) finding it on the original site (which was it?) etc.

Why can't I search my own comments over time?

==Hint: This is not a plea for yet another flakey web service==

It's a weird world when you consider how much you type into these boxes all to evaporate for your own use.

As part of building a different world what relationship does Polar have with https://web.hypothes.is/blog/annotation-is-now-a-web-standar...

Why or why not?


> As part of building a different world what relationship does Polar have with https://web.hypothes.is/blog/annotation-is-now-a-web-standar....

We will double down and investigate web standards once Polar is a bit more solid. Right now it's just extra work when I haven't validated that Polar is going to be useful to 100s of thousands of people.


I was hoping and looking for ePub support, and it's listed in the long term section of the roadmap. That's the main feature I'd want. This looks really nice (though I'm not a fan of Electron apps). I can't wait for the ePub support.

Another thing I'd like to see is a mobile app, on iOS and Android.


I'm also really interested in a mobile app. Currently I am using Pocket, but something like this (especially synced) would be golden. Pocket is just too unpredictable in it's web rendering. I now prefer to see web view to not lose any additional content (images, videos, unique content designs).


You can print the ePub to PDF and load it into Polar. Not perfect for now but it works. It's what I do.

We're going to re-write the viewer backend at some point and once that's done I think ePub will be straight forward.


I really like where this is going, it seems a bit rough around the edges right now (which is totally fine!) but functional enough to be useful. Really happy it's free, open source and offline first (I can just sync through Drive or Dropbox).

Couple thoughts on some exploring: - Webpage capture: seems that only links with `http` in front work, would be great to get resolving working nicer (say news.ycominator.com should work). Can't see this being a large issue as mostly I would copy paste from my browser, which keeps the http

- The "inspect element" menu in the editor seems a bit unnecessary, as I don't really think of PDFs as HTML. I guess this may be nice for some folks who are familiar with HTML but for me its a bit confusing.

- I managed to crash a page by highlighting a section and using the page down buttons(unsure exactly what caused this)

- I had some performance issues when resizing the page on a longer pdf (140ish pages), crashed and then wouldn't reopen until I restarted the application

As I said a bit rough around the edges still, but as you hopefully get more use these bugs/inconveniences will get ironed out. Excited to see this develop and great work!


> Can't see this being a large issue as mostly I would copy paste from my browser, which keeps the http

We will ship a chrome extension so you can do one page sync between Polar and your browser.

We're going to use Firebase for the auth and sending of this data between instances.

> - The "inspect element" menu in the editor seems a bit unnecessary, as I don't really think of PDFs as HTML. I guess this may be nice for some folks who are familiar with HTML but for me its a bit confusing.

I will probably remove it by default and add a 'developer' mode which turns these on.

> - I had some performance issues when resizing the page on a longer pdf (140ish pages), crashed and then wouldn't reopen until I restarted the application

That's weird. It's usually pretty solid for me.


Waiting forward for the Chrome extension. I'd rather not have another app open all the time / or open it each time I want to add a bookmark.

Meanwhile, I'd find helpful a shortcut for the 'Capture' command – ideally it would also be pre-populated with the clipboard, like Pocket does on mobile – when detecting a url in the clipboard.


It looks like the PHZ files are just ZIP files (which I really like), however they're not compressed. As a test I created a new PHZ with compression (using Info-ZIP) with the contents of an existing PHZ file. I found that the compressed PHZ seems to open fine in Polar Bookshelf.

What's the rationale for not using compression in PHZ files? Is there any reason for me not to update all my PHZ files to using compression?


Oh.. That might be a bug. They are supposed to be compressed! I'll check into that!


Looks promising, but I feel like I've already run into a major bug - I downloaded, opened a .pdf. Made a text highlight > all good. Delete the highlight. Try to make another highlight > Now I can no longer make a highlight at all.

Replicated twice running 64-bit on a windows machine.

Once the bugs are ironed out this could be really useful, good luck!


Yes. Still a few small bugs like this here and there. working on ironing those out.


This looks great, I'd love to have a local backup of saved documents. Currently I'm using pocket which I love because it's so easy to share a document from any of my different devices (personal laptop, mobile phone, work laptop). However I'm worried about the service going away or any of the "desktop version" of the links going stale (pocket's rendering isn't always perfect). Any thoughts about support for pulling from Pocket as a data source? Pocket has an API you could hit, find any new links, and download them.


I'm building Polar as a 30 year document repository. Meaning I want you guys to have confidence that Polar will exist in 30 years. That's why it's OSS.


https://wallabag.org covers your complaints with pocket and also has an API.


That's a very interesting concept that I'll have to check out. I guess initially I was thinking of a single location to backup pocket in case it goes down, however hosting my own server (like the wallabag solution) is an interesting idea that I hadn't considered yet.


Thanks for all the great feedback guys!

Feel free to jump on our Discord if you have any comments:

https://discord.gg/GT8MhA6


Haven't had a chance to watch the videos, so maybe this has been answered...

I see 'added' and 'last updated' columns on content - do you keep historical versions or update in place? If you only update in place, then the content can break or change for any number of reasons. Even if it's only configurable, I'd rather use the extra space (text is tiny anyways) to store the history (with an infrequent update schedule).


Loving the idea, will definitely battle test it.

Idea: consider linking it up with fx. Zotero, so you don't need to maintain two document repositories.


On macOS it doesn't let you open all PDFs and I wouldn't know why, expect maybe file size.


Any specific error? Let me know what's happening and we'll fix it.


What's the .phz archive format that's used to store each document in .polar/stash?


It's our own file format for storing compressed HTML and associated resources in one bundle.


Any chance of supporting HAR or WARC?


Would be very helpful if it does also save links x levels down, e.g. for reading documentation.


This is neat, but to me it looks more conceptually like "personal ebook reader you can sync across machines (and can also import web pages)" than "offline browser".

Edit: Similarly, I find lack of any epub support kind of weird.


Unfortunately, the snap dies with this error for me on Ubuntu: https://www.pastery.net/vfyfyj/


Yes. The debs work on Ubuntu. REALLY sorry about the snap packages.

They are apparently broken due to a bug in Ubuntu's snap but I don't have a work around.

Could you try to install canberra-gtk-module and let me know if the problem goes away?


Unfortunately it seems to already be installed, so I don't know what else I could install :/ Almost all the entries in apt that have canberra in the name are already installed.

EDIT: The deb works perfectly indeed, thanks!


Segfaults on my 18.04 ubuntu.


This looks great! Been hoping for something like this for a while that is especially: free, open source, office first!


Is the Windows version portable?

Website doesn't say.


What does that mean? I don't follow. It supports 64 and 32 bit windows.


https://portableapps.com/about

Basically, they're apps that don't need to be installed to a single system. They can execute from a dropbox directory, usb drive, etc.


Oh. Thanks. I think the current app is an installer BUT there is a Windows version that doesn't require a setup and is just a flat .exe.

I think we could publish one if this is frequently requested.


Would use if it supported Android.


It's on the roadmap for the flashcards and annotations. Probably not reading for the short term as reading PDFs on Android doesn't sound super fun.


I read PDFs on an Android 8" tablet without problems with the EasyPDF app. It mostly works on the phone's 5" screen too if the text is not too small (reflow mode doesn't work well for many files).


This looks very attractive. Is the project open to setting up bounties to express certain features?


yes.. I think we're open to bounties. Wondering what apps people are using for that part.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: