A Python-based application that leverages Small Language Models (SLMs) for domain-related Text-to-SQL tasks. This tool converts natural language queries into SQL statements and provides natural language interpretations of the results.

• Features
• Prerequisites
• Installation
• Configuration
• Usage
• File Structure
• Contributing
• License

• Natural Language to SQL: Convert plain English queries into SQL statements
• SQL Execution: Execute generated SQL queries against your database
• Natural Language Response: Get human-readable interpretations of query results
• Domain-Specific: Optimized for specific domain tasks (e.g., trading, finance)
• Local LLM Integration: Uses Ollama for local language model processing

Before setting up this repository, ensure you have the following installed:

• Python 3.8 or higher
• pip (Python package installer)
• Ollama (for local LLM processing)
• PostgreSQL database server (Download)

1. Clone the repository:

   git clone https://github.com/TheQuantScientist/TextToSQL.git
   cd TextToSQL

2. Create a virtual environment (recommended):

   python -m venv texttosql_env
   source texttosql_env/bin/activate  # On Windows: texttosql_env\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Set up PostgreSQL database:

   • Install PostgreSQL on your system
   • Create a database for the application
   • Note down the database credentials (host, port, database name, username, password)

5. Set up Ollama:

   • Install Ollama on your system
   • Start the Ollama server locally
   • Pull the required language model (refer to config.py for model specifications)

Before running the application, you need to configure the database connection and LLM settings:

1. PostgreSQL Database Configuration: Update the database connection settings in config.py:

   DATABASE_CONFIG = {
       'host': 'localhost',
       'port': 5432,
       'database': 'your_database_name',
       'username': 'your_username',
       'password': 'your_password'
   }

2. LLM Configuration: Ensure the correct model is specified in config.py

3. Prompt Templates: Customize prompts in prompts.py if needed for your specific domain

1. Edit your query: Open main.py and update the user_query variable with your question:

   user_query = "Calculate a trading strategy for gold and stock"

2. Run the application:

   python main.py

3. View the output: The application will display:

   • The generated SQL query
   • Raw query results from the database
   • Natural language interpretation of the response

The main entry point of the application. This file orchestrates the entire Text-to-SQL pipeline:

• Accepts user queries
• Coordinates between different modules
• Displays the final results
• Contains the primary execution logic

Configuration management for the entire application:

• Database Configuration: Credentials, and database-specific settings
• LLM Configuration: Model specifications, API endpoints, and language model parameters
• Data Field Configuration: Defines the structure and mapping of data fields
• Environment Variables: Manages environment-specific settings

Template management for prompt engineering:

• SQL Generation Prompts: Prompt remplate for converting natural language to SQL
• Imports FIELDS_JSON from config: This contains the names and details of the columns in the finance_economics_dataset for SQL generation agent to understand.
• Natural Language Prompts: Prompt template for converting SQL results back to human-readable text

Database utility functions and connection management:

• Database Connection: Establishes and manages database connections
• Schema Introspection: Retrieves database schema information for context
• Query Execution: Executes SQL queries with error handling
• Result Processing: Formats and processes query results

Language Model utility functions and initialization:

• LLM Initialization: Sets up and configures the language model (Ollama)
• Error Handling: Manages LLM-related errors and fallbacks

The core intelligence of the application:

• Query Processing: Analyzes and processes natural language queries
• SQL Generation: Coordinates the conversion from natural language to SQL
• Query Execution: Runs the SQL against the database
• Result Interpretation: Converts SQL results into natural language responses

Lists all Python dependencies required for the project. Install these using:

pip install -r requirements.txt

Here's a typical workflow:

1. Input Query: "Calculate and come up with a trading strategy for gold and stock, provide clear analysis"

2. Generated SQL:

   SELECT date, gold_price
   FROM 
    finance_economics_dataset
   ORDER BY date
   LIMIT 50;

3. Result:

1. Update prompts.py with domain-specific examples
2. Modify config.py to include relevant data fields
3. Adjust agent.py logic if needed for specialized processing

1. Update connection parameters in config.py
2. Modify db_utils.py for database-specific optimizations
3. Test with your specific database schema

1. Ollama Connection Error: Ensure Ollama server is running locally
2. PostgreSQL Connection Error:
   • Verify PostgreSQL service is running
   • Check database credentials in config.py
   • Ensure the database exists and user has proper permissions
   • Test connection using: psql -h localhost -U your_username -d your_database
   • Show SELECT table_name FROM information_schema.tables WHERE table_schema = 'public';
3. Model Loading Error: Check if the specified model is downloaded in Ollama
4. Dependency Issues: Ensure all requirements are installed correctly

• Built with Ollama for local LLM processing
• Utilizes prompt engineering techniques
• Designed for domain-specific Text-to-SQL applications

For issues and questions:

1. Check the troubleshooting section
2. Open an issue on GitHub
3. Review the documentation in each Python file for detailed implementation notes

Note: This application is designed for domain-specific use cases. Customize the prompts and configuration files according to your specific requirements and database schema.