Design a News Feed — Fan-Out, Ranking, and Real-Time Updates

The feed is the product#

For social platforms, the feed IS the product. Instagram, Twitter, LinkedIn — users open the app to scroll the feed. If it's slow, stale, or irrelevant, they leave.

Designing a news feed combines data fanout, ranking algorithms, caching, and real-time updates into one system.

Two approaches: push vs pull#

Fan-out on write (push)#

When User A posts, immediately push the post to every follower's feed.

How: User A has 500 followers. On post creation, write 500 entries to 500 individual feed caches.

Pros: Reads are instant — the feed is pre-computed. Cons: Writes are expensive. Celebrities with 50M followers = 50M writes per post.

Fan-out on read (pull)#

When User B opens their feed, fetch posts from everyone they follow and merge them.

How: User B follows 200 people. Query each person's recent posts, merge, rank, return.

Pros: Writes are cheap — one write per post. Cons: Reads are slow — merging 200 timelines in real-time.

The hybrid approach (what everyone uses)#

• Regular users (< 10K followers): fan-out on write. Pre-compute feeds.
• Celebrities (> 10K followers): fan-out on read. Fetch their posts at read time and merge with the pre-computed feed.

This gives fast reads for 99% of users while avoiding the celebrity problem.

Ranking the feed#

Chronological feeds are simple but not engaging. Modern feeds use ranking:

Signals:

• Engagement prediction: ML model predicts probability of like, comment, share
• Relationship closeness: How often do you interact with this person?
• Content type: Do you prefer videos, images, or text?
• Recency: Newer posts get a boost, but great old posts can surface
• Diversity: Don't show 10 posts from the same person

Simple ranking formula:

score = engagement_probability × relationship_score × recency_decay × diversity_bonus

Feed storage#

Per-user feed cache (Redis):

feed:user123 → [post_id_1, post_id_2, ..., post_id_500]

Sorted set in Redis. Each entry is a post ID with the ranking score. Trim to last 500 entries.

Post storage (database):

posts:
  id, author_id, content, media_url, created_at, like_count, comment_count

When rendering the feed: fetch post IDs from Redis, batch-load full post data from database/cache.

Real-time updates#

User is scrolling the feed. Someone they follow posts something new.

Options:

1. WebSocket push: Server pushes "1 new post" notification. User clicks to load.
2. Polling: Client checks every 30 seconds. Simple but wasteful.
3. Long polling: Client holds connection open until new content exists.

Most apps use WebSocket for active users and push notifications for inactive users.

Scaling considerations#

See feed architecture#

On Codelit, search "Twitter" or "social network" in ⌘K to see complete feed architectures — fan-out service, timeline cache, ranking engine, and real-time sync.

---

Design your feed system: search "Twitter" on Codelit.io and explore the feed architecture.