@tanstack/react-query Common Issues and Solutions Series

Handling Duplicate Data When Manually Updating the Cache in Infinite Queries

When working with paginated data using TanStack/React Query’s useInfiniteQuery, you might encounter a situation where you need to manually update the cache to include new data without re-fetching all previous pages. A common example is adding a new comment to a comments list without re-fetching the entire comments thread.

This manual cache update can lead to duplicates when the same data is eventually fetched again during pagination, resulting in duplicate items in your UI and React warnings about duplicate keys.

Scenario

You have a comments section that:

• Fetches comments using useInfiniteQuery with pagination.
• Allows users to add new comments.
• Manually updates the cache to include new comments without refetching all pages.
• Displays comments in a scrollable area that loads more comments as the user scrolls.

// Fetch comments with pagination
const { data, fetchNextPage, hasNextPage } = useInfiniteQuery({
  queryKey: ['comments', postId],
  queryFn: fetchComments,
  getNextPageParam: (lastPage) => lastPage.nextPage,
});

// Manually update cache when a new comment is added
const addCommentToCache = (newComment) => {
  queryClient.setQueryData(['comments', postId], (oldData) => {
    if (!oldData) return;
    const lastPage = oldData.pages[oldData.pages.length - 1];
    lastPage.comments.push(newComment);
    return { ...oldData };
  });
};

// Add a new comment
const handleAddComment = async (commentText) => {
  const newComment = await addCommentToServer(commentText);
  addCommentToCache(newComment);
};

Problem Occurs When:

1. A user adds a new comment, which is manually added to the last page in the cache.
2. The user scrolls to load more comments, triggering fetchNextPage().
3. The newly added comment is fetched again from the server and added to a new page.
4. The same comment now exists in multiple pages, leading to duplicates in the UI and React warnings about duplicate keys.

Why Does This Issue Happen?

When manually updating the cache to include new data:

• Manual Addition: The new comment is added to the cache manually to avoid refetching all paginated data.
• Server Fetching: When fetching more data during pagination, the server returns the same new comment because it now exists in the database.
• Duplication: This results in the same comment appearing in both the manually updated cache and the newly fetched data.
• Duplicate Components and Keys: You don’t want your user to see their comment twice when they are scrolling through the comments section. And you don’t want duplicate keys for your react components. You may not have the comment ids as keys, but its likely that as you write your comments, you may realize that using comment ids as keys is probably the best way to assign keys. And since React requires that each item in a list rendered with .map() has a unique key prop. Duplicate comments with the same id lead to duplicate key values, causing React to warn about duplicates and potentially leading to rendering issues.

How to Solve It?

Solution: Deduplicate Data Before Rendering

To prevent duplicates, you need to ensure that each comment appears only once in your UI, even if it exists in multiple pages of your cached data. You can achieve this by deduplicating the merged comments before rendering them.

Step-by-Step Guide:

1. Merge All Pages into a Single Array:

Combine all pages of comments into one array using flatMap().

const mergedComments = data?.pages.flatMap((page) => page.comments) || [];

2. Deduplicate Comments Based on a Unique Identifier:

Use a Map or an object to remove duplicates based on the commentId.

const uniqueComments = Array.from(
  mergedComments.reduce((map, comment) => {
    map.set(comment.commentId, comment);
    return map;
  }, new Map()).values()
);

3. Render the Unique Comments:

Use the uniqueComments array to render your comments list.

return (
  <div>
    {uniqueComments.map((comment) => (
      <Comment key={comment.commentId} data={comment} />
    ))}
  </div>
);