HACKER Q&A
📣 ers35

What features for an offline Hacker News reader?


HN is a treasure trove of information. The primary way I read HN is to bookmark stories and read the comments when the discussion is complete. Others actively participate. Some get jobs via "Who is hiring?". We all use HN differently.

There are ways to read HN other than this website. However, I have not found one that meets all of these requirements:

  - Self-hosted.
  - Offline access of data.
  - Query data via SQL.
  - Full text search of stories and comments.
  - Notification of replies to comments.
Some ideas:

1) A tool that maintains a copy of the HN API[1] in an SQLite database, with indexing and full-text search[2]. This supports the development of the other tools.

2) A command line tool that demonstrates how to use the database and supports the development of scripts. For example:

  #!/bin/sh
  # Run notify whenever someone replies to a comment.
  $ hn replies username | notify
3) A web UI for browsing and searching. This can be hosted locally or on a remote server.

What features interest you?

  [1]: https://github.com/HackerNews/API
  [2]: https://www.sqlite.org/fts5.html

  👤 markus_zhang Accepted Answer ✓
I'm thinking about the same thing i.e. building a personal HN reader with Python and QT.

I have never explored the API, and I'm not a professional programmer, but allow me to describe the ideas from top of head when I read your posts:

Requirements:

1 - Self-hosted: So essentially, it's too heavy to keep the whole sites offline. However, because I always read by topic (e.g. I'd type "SICP ycombinator" in Google and read the top pages), I think one approach is to let the user enter a topic, say "SICP", as well as number of top-level stories to return, say 25, and invoke the API to return stories. The app will then dump the stories and their comments into a database with data modelling that suits for a forum.

2 - Offline access of data: Essentially what I mentioned above. The app should also allow user to remove a story (and its children), modify topics, favorite a story and create new topics. I think those are the barebone requirements. The backend would interact with the database and do updates.

3 - Query data via SQL: I think it might be too much work to parse queries, and the easiest thing to do is to just access a string as query and pass that to the database engine. Or maybe only allow user certain actions and let the backend assemble the queries.

4 - Full text search of stories and comments: Not sure what to do as my programming knowledge is very limited. I saw the second link you provided and it is very interesting.

5 - Notification of replies to comments: Maybe give user a button to update all his favorite stories.

TBH I really want to see what other people's implementations will be.

👤 geoah
I'd be very interested in something that can cache the actual article/page from the main HN link as well as a reading list.

There are times when I add HN articles to mobile chrome's "read later" list, and when I have time go through them. Being able to read them later when not having internet access would be amazing (London's tube is annoying when it comes to internet, internet only exists in the stations).

👤 Tomte
A kill file. De-emphasizing comments I have already seen. notifying me of replies to my comments. Notifying me of my submissions that got re-upped while I was sleeping, so I can see the discussion.
👤 tylerjwilk00
I recently built an HN client [1] to scratch an itch and try out the HN API. It caches data to SQL and redis.

It has never been shared and I only use it personally so expect lots of bugs.

I may open source the code after I clean it up.

[1] http://app.hackerdelivery.com/

👤 mlang23
If up/downvoting would work in Lynx, I wouldn't actually need a CLI tool to read HN. Well, I guess you can't have anerything.

I wouldn't need offline mode, but a CLI tool to read HN would be very much welcome.

👤 stevenicr
I'd like to be able to move a discussion thread to the bottom, or collapse a discussion thread. Perhaps sort by timestamp the comments option as well.

Sometimes the scroll gets real and I want to get past a discussion to see other angles of whatever subject - scrolling trying to find where thread 2 ends and a different subject comes up is challenging at times - so I often go to a new tab on a different story and don't come back.

Even click/tap to draw a line from comment 2 to all sub comments so I can find where comment 3 starts would be helpful.

👤 saxelsen
I use the Android app Materialistic, which I really enjoy.

In particular I like the fact that I can bookmark posts and comments with it.

I very often jump to the discussion to read opinions - there's often a treasure trove of stuff there of someone who knows something more than what the article states.

A really annoying, missing feature for the discussion/comments parts, is that the threads are extremely difficult to tell apart and traverse. On a small screen, the threads are very close together and only distinguishable through color differences. Scrolling through them is a nightmare, whereas I've really come to love the Reddit app's feature with a button that allows you to scroll to the next sibling comment, e.g. if you want TO quickly see the next comment to the OP, instead of scrolling endlessly and not knowing how far along you are.

👤 rcarmo
Definitely sane, responsive thread layouts with more readable text. HN has, let us say, a very “traditional” approach to HTML, and it is _very_ hard to read on some devices (I wish that someone revised the markup accordingly).

Which is why I usually read it on https://hackerweb.app/ instead.

Forking https://hackerweb.app and adding offline support would be a good starting point, IMHO.

👤 gitgud
The query api at: https://hn.algolia.com is pretty amazing, searches all stories.

The biggest qualms for me would be; notifications of comments and searching my previous comments.

Other than that Hacker News is actually highly reliable and available and I've never personally needed an off-line version (except maybe on a plane)

👤 folmar
I doubt we need another reader program. What I would really see is a two-way translation to NNTP, and the user could use any client and addons they want - it's old and popular, you get pretty much everything possible off the shelf, and it quite matches the semantics of HN (minus the voting).
👤 rchaud
One thing I'd find useful:

- save individual posts as "favorites", and allow users to tag them somehow (e.g. 'finance', or 'front-end') for easy searchability. The saved quote should be available for offline reading, and should link back to the original topic. The original topic doesn't need to be available offline.

👤 SkyMarshal
Don’t forget to include the basics - upvote, downvote, unvote, comment, submit, save submission, save comment - and have all that sync when reconnecting to internet.
👤 anotheryou
- rip links (at least article's text)

- mark links and comments as seen

- hide seen

- fold comments to only show top level

- favorite links (preferably to a list that is easily dumped in to e.g. pocket with a few buttons for that)

👤 caseyf7
Show only top-level comments so you can skip long off-topic discussions and drill-down on the discussions you are interested in.
👤 contingencies
Suggest considering implementing a bridge to NNTP, or SMTP mail spools. Then use offline NTTP or mail readers.
👤 kwhitefoot
Sounds interesting. Here are a couple of extra ideas:

- an Android app embodying the same features,

- The ability to comment while offline.

👤 andrei_says_
All posts and comments I’ve upvoted and bookmarked are searchable.
👤 floatingatoll
To clarify, why is “self-hosted” an important criteria to you?