What is the best approach to solving this problem?
I can make a table that records all the posts that each user has ever seen, then filter out seen posts after a list of recommendations are created. But this seems like a solution that will become more and more costly over time as more people see more posts, and we will need to check it every single time we recommend posts.
This feels like a standard problem that must have been answered by others over the past decades. Perhaps there is a framework that I am not aware of that takes an opinionated approach for this problem as well as other problems in the content recommendation space. But then again, I see seen contents all the time on YouTube, so maybe this is not a solvable problem with current computing capacity.
Please share your experiences and insights!
OK, so like email with a "show only new messages" filter.
> more and more costly over time
If it's tolerable to have false positives (potentially showing a user a post that they have seen before), a Bloom filter [0] might be a good match.
If you use this structure as an index to record which posts the user has seen, then it might return a post the user has already seen before (it may answer the question "is this post ID in the set?" incorrectly), but it can for sure return a post the user has not seen before (in that it will answer "is this post ID definitely not in the set?" correctly).
[0] https://en.wikipedia.org/wiki/Bloom_filter
(Edited for clarity.)