I have been integrating LLMs into my projects for some time now and recently had the opportunity to present AI search to a customer. The result is the beginning of a project to perform search results grounded in an organizations data.
While there are many examples of implementing RAG I wanted to build a project that could be the base of an actual solution. I will provide a short example using aiasearch and then touch on the implementation. I will also include a few areas for improvement.
Use Case – Building Permit Search:
Users need to be able to search a repository of building permit approval documents. They would like to find documents pertaining to a specific address, or multiple addresses such as those on the same street.
The city of Boston provides this data and a subset of it has been used in this example. https://data.boston.gov/dataset/approved-building-permits
Example:
Question: List all building permits on Tremont Street including the permit number, work performed and address.
Response:
Based on the provided documents, here are the building permits on Tremont Street:
1. Permit ALT1008773
– Address: 789-793 Tremont ST, Roxbury MA 2118
– Work Performed: Change Occupancy from multifamily 177 units to 190 Residential Apartments
…
Getting Started:
The file main.py provides an example of using aiasearch.
Loading Data
There is a CsvLoader() class that allows specifying which columns should be used in the text of each document and if they should be prepended with a label or concatenated to the end of the text.
For example the row:
“A893260”, “2018-11-16 16:02:45+00”, “Amendment to a Long Form”, “230-232 Newbury Street”, “Boston”, “MA”
Can be sent to the LLM as:
“Permit Number: A893260
Issued Date: 2018-11-16 16:02:45+00
Amendment to a Long Form Other 230-232 Newbury Street amended drawings to reflect new rear openings as requested by Building Inspector. 230-232 Newbury ST Boston MA”
Document Search
Documents are searched using both and semantic and keyword searches. The results of both are combined and passed to the LLM with the user’s input to obtain the final result.
A local Chroma store is used here by instantiating the class ChromaVectorStore. The documents are passed to the store which will call Ollama to retrieve vector representations of each sentence and add them to the store.
When a search is performed a vector similarity comparison will return matched documents.
Another store may be used by creating a class that extends VectorStore. An option would be Elasticsearch which can maintain both a vector and keyword store.
The RankBm25KeywordStore extends KeywordStore which may also be implemented in a custom class to support other keyword stores. Again the documents are passed to the store for searching.
AI Result
Documents obtained from the combined semantic and keyword searches are passed along with the user’s original query to an LLM. Here OllamaProvider is used and can be created with the name of the model to use. OllamaProvider implements Provider allowing you to create your own LLM wrapper. AnthropicProvider is also included.
The Provider class allows specification of the directory to find prompt text for each model. This allows you to save your custom prompts outside of the project. The included prompts are in the prompts folder such as /aiasearch/prompts/llama3.1.yaml.
The result is obtained by calling OllamaProvider. query_grounded().
Next Steps
Only the output of the LLM is returned which may or may not include the source documents or links to them. It would be advantageous to also return links to the source documents as well.
Add reranking to the combined semantic and keyword search to remove duplicates.
Experiment with Multi-Query Translation to rewrite the input query multiple times
Tavily could be used to retrieve up to date web search results.
Decomposition – Add additional LLM calls to identify data that can be used for search filtering. An example in this use case would be to use an LLM query to find a permit number and filter the documents in the search context based on the number provided it is in the document metadata. A separate LLM query that is not based on the source documents could also be combined with the result to provide external information.