.. _performance-optimization:

==========================
Performance Optimization
==========================

The tiger Rating Engine includes comprehensive performance optimizations that enable efficient processing of high-volume bulk rating operations. This guide covers the caching system, performance monitoring, and optimization strategies.

.. contents:: Table of Contents
   :local:
   :depth: 2

Overview
========

Performance Improvements
------------------------

.. figure:: _static/screenshots/performance_gains.png
   :alt: Performance Gains
   :align: center
   :width: 100%

   *Screenshot Placeholder: Bar chart comparing processing time before optimization (80-120s for 1000 ratings) vs after (10-20s), showing 4-12x speedup*

Optimization results:

.. table:: Performance Improvements
   :widths: 40 25 35

   +--------------------------------+------------------+--------------------+
   | Metric                         | Before           | After              |
   +================================+==================+====================+
   | Database queries (1000 ratings)| ~15,000          | ~300 (98% reduction)|
   +--------------------------------+------------------+--------------------+
   | Execution time (1000 ratings)  | 80-120 seconds   | 10-20 seconds      |
   +--------------------------------+------------------+--------------------+
   | Cache hit rate                 | 0%               | 85-99%             |
   +--------------------------------+------------------+--------------------+
   | Memory usage                   | High (unbounded) | Low (chunked)      |
   +--------------------------------+------------------+--------------------+

Key optimization features:

* **Redis Caching**: 70-90% faster through cached product configurations
* **Query Optimization**: 98% reduction in database queries
* **Chunked Processing**: Reduced memory footprint for large batches
* **Connection Pooling**: Better resource utilization
* **Performance Monitoring**: Real-time metrics and tracking

Caching System
==============

Redis Configuration
-------------------

The system uses Redis for high-performance caching:

.. code-block:: python

   # config/settings/base.py
   CACHES = {
       "default": {
           "BACKEND": "django_redis.cache.RedisCache",
           "LOCATION": REDIS_URL,
           "OPTIONS": {
               "CLIENT_CLASS": "django_redis.client.DefaultClient",
               "IGNORE_EXCEPTIONS": True,  # Don't crash if Redis is down
               "CONNECTION_POOL_KWARGS": {
                   "max_connections": 50,
                   "retry_on_timeout": True,
               },
           },
           "KEY_PREFIX": "rating",
           "TIMEOUT": 1800,  # 30 minutes default
       }
   }

Configuration settings:

.. table:: Cache Settings
   :widths: 40 60

   +--------------------------------+------------------------------------------------+
   | Setting                        | Description                                    |
   +================================+================================================+
   | ``BULK_RATING_CACHE_TTL``      | Cache timeout (default: 1800 seconds)          |
   +--------------------------------+------------------------------------------------+
   | ``BULK_RATING_CHUNK_SIZE``     | Batch processing size (default: 1000)          |
   +--------------------------------+------------------------------------------------+
   | ``max_connections``            | Redis connection pool size (default: 50)       |
   +--------------------------------+------------------------------------------------+

What Gets Cached
----------------

.. figure:: _static/screenshots/cache_layers.png
   :alt: Cache Layers
   :align: center
   :width: 100%

   *Screenshot Placeholder: Diagram showing cache layers - Product Config, Rating Factors, Rating Tables, Table Values, Formulas, Coverage Definitions*

Cached data includes:

1. **Product Configurations**:
   * Product metadata and settings
   * Version information
   * Key: ``rating:product:{code}:latest``

2. **Rating Factors**:
   * All factors for a product
   * Validation rules
   * Key: ``rating:factors:{product_id}``

3. **Rating Tables**:
   * Table definitions
   * Dimension configurations
   * Key: ``rating:table:{product_id}:{table_code}``

4. **Table Values**:
   * Individual lookup results
   * Key: ``rating:table_val:{table_id}:{dimension_hash}``

5. **Formulas**:
   * Formula definitions
   * Execution order
   * Key: ``rating:formulas:{product_id}:{coverage_code}``

6. **Coverage Definitions**:
   * Coverage metadata
   * Validation rules
   * Key: ``rating:coverage:{coverage_code}``

Cache Performance
-----------------

Performance impact of caching:

.. table:: Cache Impact by Operation
   :widths: 30 25 25 20

   +--------------------+---------------+---------------+------------+
   | Operation          | Without Cache | With Cache    | Speedup    |
   +====================+===============+===============+============+
   | Product load       | 15-25ms       | 0.2-1ms       | 15-125x    |
   +--------------------+---------------+---------------+------------+
   | Factors load       | 10-20ms       | 0.2-1ms       | 10-100x    |
   +--------------------+---------------+---------------+------------+
   | Table lookup       | 10-20ms       | 0.1-0.5ms     | 20-200x    |
   +--------------------+---------------+---------------+------------+
   | Coverage load      | 5-10ms        | 0.1-0.5ms     | 10-100x    |
   +--------------------+---------------+---------------+------------+

Using the Cache
===============

Cache Warming
-------------

Pre-load product data before bulk processing:

.. code-block:: python

   from tiger_rating.rating.utils.cache import warm_cache_for_product

   # Warm cache for specific product
   warm_cache_for_product(
       product_id="123e4567-...",
       product_code="BOP"
   )

   # This caches:
   # - Product configuration
   # - All rating factors
   # - All formulas
   # - All rating tables

When to warm cache:

* Before large bulk rating jobs
* During deployment
* After product configuration changes
* When adding new products

Cache Invalidation
------------------

Clear cache after configuration changes:

.. code-block:: python

   from tiger_rating.rating.utils.cache import invalidate_product_cache

   # After updating product configuration
   invalidate_product_cache("BOP", product_id="123e4567-...")

   # This clears:
   # - Product configuration
   # - Product factors
   # - Formulas
   # - Rating tables

The system automatically invalidates caches when:

* Product configurations are updated
* Rating factors are modified
* Formulas are changed
* Rating tables are updated

Manual cache clearing:

.. code-block:: python

   from django.core.cache import cache

   # Clear all rating caches
   cache.delete_pattern("rating:*")

   # Clear specific patterns
   cache.delete_pattern("rating:product:*")  # All products
   cache.delete_pattern("rating:table:*")    # All tables
   cache.delete_pattern("rating:factors:*")  # All factors

Cache Statistics
----------------

Track cache effectiveness:

.. code-block:: python

   from tiger_rating.rating.utils.cache import cache_stats

   # Get statistics
   stats = cache_stats.get_stats()
   print(stats)

   # Output:
   # {
   #     "hits": 850,
   #     "misses": 150,
   #     "total": 1000,
   #     "hit_rate": 85.0
   # }

   # Reset statistics
   cache_stats.reset()

Good cache performance:

* **Hit rate >80%**: Excellent (cache is effective)
* **Hit rate 60-80%**: Good (some optimization possible)
* **Hit rate <60%**: Poor (check cache warming and TTL)

Performance Monitoring
======================

Monitoring Tools
----------------

The system includes comprehensive performance monitoring:

.. code-block:: python

   from tiger_rating.rating.utils.performance import (
       PerformanceMonitor,
       track_execution_time,
       log_query_count,
   )

   # Context manager usage
   with PerformanceMonitor("bulk rating") as monitor:
       # ... perform operations ...
       pass

   # Returns metrics:
   # {
   #     "operation": "bulk rating",
   #     "execution_time_ms": 5240,
   #     "execution_time_s": 5.24,
   #     "query_count": 127
   # }

Function decorators:

.. code-block:: python

   @track_execution_time
   @log_query_count
   def process_ratings(requests):
       # ... implementation ...
       pass

   # Logs:
   # Function process_ratings executed in 1250ms
   # Function process_ratings executed 45 queries

Query Analysis
--------------

Analyze database queries:

.. code-block:: python

   from tiger_rating.rating.utils.performance import QueryOptimizer
   from django.db import connection

   # Analyze queries
   analysis = QueryOptimizer.analyze_queries(connection.queries)
   print(analysis)

   # Output:
   # {
   #     "total_queries": 100,
   #     "total_time_ms": 523.45,
   #     "average_time_ms": 5.23,
   #     "slowest_query_ms": 45.67,
   #     "fastest_query_ms": 0.12
   # }

   # Find N+1 query problems
   n_plus_one = QueryOptimizer.find_n_plus_one_queries(
       connection.queries,
       threshold=5
   )

Performance logging:

.. code-block:: python

   # Logs include performance metrics
   logger.info("Bulk rating job completed", extra={
       "job_id": job.id,
       "total_requests": 1000,
       "execution_time_ms": 15240,
       "query_count": 342,
       "cache_hit_rate": 94.5,
       "success_rate": 98.7
   })

Optimization Strategies
=======================

Bulk Rating Optimization
------------------------

The bulk rating service is optimized for high-volume processing:

**Features:**

1. **Cache Warming**:
   * Identifies unique products in batch
   * Pre-loads all product data
   * Reduces database queries by 98%

2. **Chunked Processing**:
   * Processes in chunks (default: 1000)
   * Reduces memory footprint
   * Better progress tracking

3. **Query Optimization**:
   * Uses ``select_related()`` for foreign keys
   * Batch operations where possible
   * Reuses cached data across requests

Performance expectations:

.. table:: Bulk Rating Performance
   :widths: 25 35 40

   +----------------+------------------------+---------------------------+
   | Batch Size     | Processing Time        | Throughput                |
   +================+========================+===========================+
   | 100 requests   | 1-2 seconds            | 50-100 ratings/sec        |
   +----------------+------------------------+---------------------------+
   | 500 requests   | 5-10 seconds           | 50-100 ratings/sec        |
   +----------------+------------------------+---------------------------+
   | 1,000 requests | 10-20 seconds          | 50-100 ratings/sec        |
   +----------------+------------------------+---------------------------+
   | 5,000 requests | 50-100 seconds         | 50-100 ratings/sec        |
   +----------------+------------------------+---------------------------+
   | 10,000 requests| 100-200 seconds        | 50-100 ratings/sec        |
   +----------------+------------------------+---------------------------+

Database Optimization
---------------------

Optimize database performance:

**Indexes:**

Ensure proper indexes exist:

.. code-block:: sql

   -- Product lookups
   CREATE INDEX idx_product_code ON rating_products(code, is_active);

   -- Factor lookups
   CREATE INDEX idx_factor_product ON rating_factors(product_id);

   -- Table lookups
   CREATE INDEX idx_table_product ON rating_tables(product_id, table_code);

**Query Optimization:**

.. code-block:: python

   # Use select_related for foreign keys
   products = RatingProduct.objects.select_related(
       'created_by'
   ).filter(is_active=True)

   # Use prefetch_related for many-to-many
   products = RatingProduct.objects.prefetch_related(
       'factors',
       'coverages',
       'formulas'
   ).filter(is_active=True)

   # Avoid N+1 queries
   # BAD:
   for product in products:
       factors = product.factors.all()  # N+1 queries

   # GOOD:
   products = products.prefetch_related('factors')
   for product in products:
       factors = product.factors.all()  # Single query

Memory Optimization
-------------------

Manage memory for large batches:

**Chunked Processing:**

.. code-block:: python

   # Process in chunks to avoid loading everything
   chunk_size = 1000
   for chunk in chunks(rating_requests, chunk_size):
       process_chunk(chunk)

**Iterator Usage:**

.. code-block:: python

   # Use iterator() for large querysets
   for run in RatingRun.objects.filter(...).iterator(chunk_size=100):
       process_run(run)

**Clear References:**

.. code-block:: python

   # Clear large objects when done
   results = process_large_batch()
   # ... use results ...
   del results  # Free memory

Best Practices
==============

Development Best Practices
---------------------------

1. **Enable Caching**: Always test with caching enabled

2. **Monitor Metrics**: Check cache hit rates and execution times

3. **Warm Cache**: Pre-warm before large batches

4. **Profile Code**: Use performance monitoring tools

5. **Test at Scale**: Test with production-sized batches

Production Best Practices
--------------------------

1. **Redis Sizing**: Allocate sufficient memory for Redis
   * Estimate: ~1-5MB per product configuration
   * Example: 100 products = ~100-500MB

2. **Connection Pooling**: Use configured pool (max 50)

3. **Monitor Redis**: Set up monitoring for:
   * CPU usage
   * Memory usage
   * Connection count
   * Cache hit rate

4. **Cache Warming**: Warm cache during deployment:

.. code-block:: python

   # deployment_script.py
   from tiger_rating.rating.models import RatingProduct
   from tiger_rating.rating.utils.cache import warm_cache_for_product

   for product in RatingProduct.objects.filter(is_active=True):
       warm_cache_for_product(str(product.id), product.code)

5. **Health Checks**: Monitor system health:

.. code-block:: python

   from django.core.cache import cache

   def health_check():
       # Check cache connectivity
       try:
           cache.set('health_check', 'ok', 10)
           value = cache.get('health_check')
           cache_healthy = (value == 'ok')
       except:
           cache_healthy = False

       return {
           "cache": "healthy" if cache_healthy else "unhealthy",
           "database": check_database(),
           "redis": check_redis_stats()
       }

Troubleshooting
===============

Cache Issues
------------

**Issue: Cache not working**

Symptoms:
* No performance improvement
* Cache hit rate is 0%

Solutions:

1. Check Redis is running:

.. code-block:: bash

   docker ps | grep redis

2. Check Redis connectivity:

.. code-block:: python

   from django.core.cache import cache
   cache.set('test', 'value', 10)
   print(cache.get('test'))  # Should print 'value'

3. Check settings: Verify ``CACHES`` configuration in ``config/settings/base.py``

4. Check logs for cache errors:

.. code-block:: bash

   just logs django | grep -i cache

**Issue: High memory usage**

Symptoms:
* Redis using too much memory

Solutions:

1. Reduce cache TTL:

.. code-block:: python

   BULK_RATING_CACHE_TTL = 900  # 15 minutes

2. Reduce chunk size:

.. code-block:: python

   BULK_RATING_CHUNK_SIZE = 500

3. Clear old caches:

.. code-block:: python

   cache.delete_pattern("rating:*")

4. Monitor Redis memory:

.. code-block:: bash

   docker stats redis

Performance Issues
------------------

**Issue: Slow performance despite caching**

Symptoms:
* Still slow even with caching enabled

Solutions:

1. Check cache hit rate (should be >80% for warm cache):

.. code-block:: python

   from tiger_rating.rating.utils.cache import cache_stats
   print(cache_stats.get_stats())

2. Warm cache before processing:

.. code-block:: python

   from tiger_rating.rating.utils.cache import warm_cache_for_product
   warm_cache_for_product(product_id, product_code)

3. Check for N+1 queries:

.. code-block:: python

   from django.db import connection
   print(f"Query count: {len(connection.queries)}")

4. Enable DEBUG temporarily to see queries:

.. code-block:: python

   # In Django shell
   from django.conf import settings
   settings.DEBUG = True
   # ... run operation ...
   from django.db import connection
   print(len(connection.queries))

**Issue: Memory errors**

Symptoms:
* Out of memory errors for large batches

Solutions:

1. Reduce chunk size:

.. code-block:: python

   BULK_RATING_CHUNK_SIZE = 500  # Default is 1000

2. Process in smaller batches

3. Increase server memory

4. Use iterator for large querysets

Monitoring and Metrics
======================

Key Metrics to Track
--------------------

Monitor these performance indicators:

.. table:: Performance Metrics
   :widths: 30 40 30

   +---------------------------+----------------------------------+----------------------+
   | Metric                    | Description                      | Target               |
   +===========================+==================================+======================+
   | Cache Hit Rate            | % of cache hits vs misses        | >80%                 |
   +---------------------------+----------------------------------+----------------------+
   | Avg Execution Time        | Time per rating (ms)             | <20ms                |
   +---------------------------+----------------------------------+----------------------+
   | Query Count               | DB queries per rating            | <5                   |
   +---------------------------+----------------------------------+----------------------+
   | Success Rate              | % successful ratings             | >95%                 |
   +---------------------------+----------------------------------+----------------------+
   | Throughput                | Ratings per second               | 50-100               |
   +---------------------------+----------------------------------+----------------------+
   | Redis Memory              | Memory used by Redis             | <1GB typical         |
   +---------------------------+----------------------------------+----------------------+

Logging Best Practices
----------------------

Log performance data:

.. code-block:: python

   import logging

   logger = logging.getLogger('performance')

   # Log job completion with metrics
   logger.info("Bulk job completed", extra={
       "job_id": job.id,
       "total": job.total_requests,
       "success": job.success_count,
       "errors": job.error_count,
       "execution_time_ms": job.execution_time_ms,
       "cache_hit_rate": cache_stats.get_stats()['hit_rate'],
       "queries_per_rating": query_count / job.total_requests
   })

Example log output:

.. code-block:: text

   [INFO] Bulk job completed
     job_id: JOB-2025-01-17-001
     total: 1000
     success: 987
     errors: 13
     execution_time_ms: 15240
     cache_hit_rate: 94.5
     queries_per_rating: 3.4

Next Steps
==========

* **Implement**: Apply optimizations to your deployment
* **Monitor**: Set up performance monitoring
* **Test**: Validate with production-sized batches
* **Learn More**: See :doc:`bulk_rating` for usage details

.. tip::
   Start by enabling caching and warming the cache for your most-used products. This alone can provide 4-12x speedup for bulk operations.

.. note::
   Performance improvements are most dramatic for bulk operations. Individual ratings may see smaller gains but will benefit from reduced database load.