src/readme.md

OpenAI Embedding and Query Processing

This Python script is designed to interact with OpenAI's GPT-3.5 Turbo model and the OpenAI Embedding API. It provides functionality for creating and loading embeddings, calculating vector similarity, ordering document sections by query similarity, and constructing prompts for the GPT-3.5 Turbo model.

Key Features

1. Embedding Creation and Loading: The script includes functions for creating embeddings for each row in a dataframe using the OpenAI Embedding API (compute_doc_embeddings) and for loading embeddings from a CSV file (load_embeddings).

2. Vector Similarity: The vector_similarity function calculates the similarity between two vectors. This is used to compare the embedding of a user's query with the embeddings of document sections.

3. Document Section Ordering: The order_document_sections_by_query_similarity function compares the embedding of a user's query with the embeddings of document sections and returns a list of document sections sorted by relevance in descending order.

4. Prompt Construction: The construct_prompt function constructs a prompt for the GPT-3.5 Turbo model based on a user's query and the most relevant document sections.

5. Query Answering: The answer_query_with_context function uses the GPT-3.5 Turbo model to generate a response to a user's query. It constructs a prompt based on the user's query and the most relevant document sections, sends this prompt to the GPT-3.5 Turbo model, and returns the model's response.

Usage

To use this script, you will need to import it into your Python project and call the functions as needed. For example, you might use the compute_doc_embeddings function to create embeddings for your document sections, the order_document_sections_by_query_similarity function to order the sections by relevance to a user's query, and the answer_query_with_context function to generate a response to the query.

Dependencies

This script requires the following Python packages:

• OpenAI
• Tiktoken
• Numpy
• Pandas
• Configparser
• Warnings

These can be installed using pip:

pip install openai tiktoken numpy pandas configparser warnings