RAG Engineering-based Query Specific Summarization

Here, I present the idea behind Query-Specific Summarization and RAG, and how it can be implemented. Use case. And how to test it.

Retrieval Augmented Generation is very well used in organizations with their own data. Many organizations have their own data that they do not want to upload to generic servers where a generic LLM can be used.

Sometimes adding extra data can make retrieval less precise.

This aligns well with data privacy, another reason why everyone wants their own LLM model. This ensures that prompt engineering is done in the best way, without confusion for the LLM, and is crisp and precise with RAG engineering.

These days, LLM can produce generic summarization to reasonable levels.

But what if we want to summarize user-specific information?

What does this mean?

This aims to make the Context Window for LLM Query useful!

This means we are given a document and a query: for example, the document describes the day in a region, and the query asks, “Was it humid that day?” So the summarizer will focus only on this question; it will forget any other details such as the number of people at the beach or the number of cars in traffic jams, but only the query.

How do we summarize user-specific details without exposing the entire knowledge base to other servers, or making it cumbersome?

Well, here comes the use of RAG (Retrieval Augmented Generation).

When the data in an organization is enormous, we do not want it exposed raw to other servers, and we want accurate results, we use RAG for retrieval. Here, we propose RAG for summarization.

The difference is that we have a text document we want to guide in summarizing.

Similarly, RAG can be used for query-focused summarization.

What is “Query Focused Summarization”?

We have a text document, which we don’t have time ot analyze. We want to summarize it, but we have internal constraints that prevent us from doing so. We have a query and would like to summarize it now.

That is, summarization is relative to the organization; for example, the organization’s help documents explain the buttons in the app it made.

We receive a document from the user. It’s a lengthy email they send to the organization’s customer care. We need to reply to them, but first, we must understand their problem. The subject of the email was “New Year Sale Fraud”, but they wrote a big email.

We need to summarize it, given the query for only buttons that fall under the “New Year Sale Fraud” theme. We can append the query with, “what is the problem and how to resolve it.”

So how do we do it?

We have a huge company database.

Let’s chuck the data.

Then convert the data chunks into vectors.

These vectors are stored as key-value pairs.

Once the user query comes.

The similarity is computed between the query vector and the vectors in our company database.

Select the top k vectors, and hence the top k chunks, and these are selected.

These are appended in the document and sent to the summarizer.

This is how to perform query-focused summarization with RAG.

This will produce a document that explains why the user wrote such an email, what the problem is, and how to resolve it.

Since it works well for Information Retrieval.

It can do well for Query-focused Summarization as well.

These need to be tested on standardized query-focused summarization datasets or on a company’s internal servers.

Thank you for reading.

Subscribe for updates.