DeepEval POC – Prompt Evaluation A comprehensive proof-of-concept demonstrating how to evaluate LLM prompts using deepeval framework with custom business metrics for real estate market analysis. Overview This POC evaluates a sophisticated real estate analysis prompt that deduces buyer profiles and generates investment theses based on property data. It includes custom metrics for: – Buyer profile identification accuracy – Format compliance (3 bullets, 80 char limit) – Theme structure validation (Unit-Project-Location) Project Structure deepeval/ ├── prompts/ │ └── real_estate_analysis_prompt.txt # Main analysis prompt template ├── data/ │ └── test_data.py # Test scenarios (Waterfront, Towers, etc.) ├── metrics/ │ └── custom_metrics.py # Custom evaluation metrics ├── models/ │ └── llm_integration.py # Gemini model integration ├── evaluate.py # DeepEval framework integration ├── simple_evaluate.py # Standalone evaluation script ├── requirements.txt # Dependencies ├── .env # Configuration (API keys) └── README.md # This file Setup Instructions 1. Prerequisites Python 3.9+ Google Gemini API key 2. Environment Setup “`bash Clone the repository git clone cd deepeval Create virtual environment python -m venv .venv Activate virtual environment On macOS/Linux: source…Read More
deepeval-poc
