# Rollover Loan Exclusion Fix

## Problem Description

Rolled-over loans were appearing in various reports (loans due, delinquent loans, arrears, etc.) even though they should be excluded. When a loan is rolled over:
- The original loan should have `status='rolled_over'` AND `is_rolled_over=True`
- A new loan is created with the rolled-over amount
- The original loan should NOT appear in any active loan reports

### Example Issue
**Loan LOAN-000128** was appearing in the "Loans Due Enhanced" report even though it had been rolled over. The loan details showed:
- Status: Active (should be 'rolled_over')
- Outstanding Balance: KES 20,599.60
- This was confusing because the loan had already been rolled over into a new loan

## Root Cause

The report services and views were only filtering by:
```python
Loan.objects.filter(status='active', is_deleted=False)
```

This did NOT exclude loans where:
- `is_rolled_over=True` (the flag indicating a loan has been rolled over)
- `status='rolled_over'` (the status indicating a loan has been rolled over)

## Solution

Updated all report services and views to properly exclude rolled-over loans:

```python
Loan.objects.filter(
    status='active',
    is_deleted=False,
    is_rolled_over=False
).exclude(status='rolled_over')
```

## Files Modified

### 1. `reports/simple_reports_service.py`
Updated the following methods to exclude rolled-over loans:
- `get_summary_metrics()` - Dashboard summary metrics
- `get_loans_due_today()` - Loans due today
- `get_delinquent_loans()` - Delinquent loans categorization
- `get_overdue_loans_analytics()` - Overdue loans analytics
- `get_missed_payments_summary()` - Missed payments summary

**Changes:**
```python
# Before
loans_qs = Loan.objects.filter(status='active', is_deleted=False)

# After
loans_qs = Loan.objects.filter(
    status='active', 
    is_deleted=False,
    is_rolled_over=False
).exclude(status='rolled_over')
```

### 2. `reports/comprehensive_reports.py`
Updated the following methods to exclude rolled-over loans:
- `get_loans_due_report()` - Comprehensive loans due report
- `get_delinquent_loans_report()` - Delinquent loans with severity categorization
- `get_loans_in_arrears_report()` - Loans in arrears with risk assessment

**Changes:**
```python
# Before
loans_due = Loan.objects.filter(
    status='active',
    due_date__isnull=False,
    due_date__date__gte=start_date,
    due_date__date__lte=end_date
)

# After
loans_due = Loan.objects.filter(
    status='active',
    is_deleted=False,
    is_rolled_over=False,
    due_date__isnull=False,
    due_date__date__gte=start_date,
    due_date__date__lte=end_date
).exclude(status='rolled_over')
```

### 3. `reports/views.py`
Updated the following:
- `get_filtered_loans_for_user()` - Base filtering function used by all report views
- `portfolio_details()` - Portfolio metrics view

**Changes:**
```python
# Before
if base_queryset is None:
    base_queryset = Loan.objects.filter(is_deleted=False)

# After
if base_queryset is None:
    base_queryset = Loan.objects.filter(
        is_deleted=False,
        is_rolled_over=False
    ).exclude(status='rolled_over')
```

## Impact

This fix affects ALL report pages and dashboard metrics:

### Dashboard Reports
- ✅ Summary metrics (total active loans, portfolio value, outstanding balance)
- ✅ Loans due today widget
- ✅ Delinquent loans widget
- ✅ Overdue loans analytics
- ✅ Missed payments summary

### Report Pages
- ✅ `/reports/loans-due/` - Loans due today report
- ✅ `/reports/loans-due/enhanced/` - Enhanced loans due report
- ✅ `/reports/delinquent-loans/` - Delinquent loans report
- ✅ `/reports/loans-in-arrears/` - Loans in arrears report
- ✅ `/reports/portfolio-details/` - Portfolio details
- ✅ `/reports/outstanding-loans/` - Outstanding loans

### Comprehensive Reports Service
- ✅ All date-range filtered reports
- ✅ All severity-categorized reports
- ✅ All risk assessment reports

## Testing

Run the test script to verify the fix:

```bash
python test_rollover_exclusion_fix.py
```

The test script will:
1. Count total active loans (including rolled-over)
2. Count rolled-over loans
3. Count active loans (excluding rolled-over)
4. Test SimpleReportsService methods
5. Test ComprehensiveReportsService methods
6. Verify no rolled-over loans appear in any report results

Expected output:
```
Tests passed: 6/6
✅ ALL TESTS PASSED! Rolled-over loans are properly excluded from all reports.
```

## Deployment Steps

### For Development/Testing
```bash
# No database changes needed - this is a code-only fix
# Just restart the Django server
python manage.py runserver
```

### For Production (cPanel)
```bash
# 1. Upload the modified files via FTP/File Manager:
#    - reports/simple_reports_service.py
#    - reports/comprehensive_reports.py
#    - reports/views.py

# 2. Restart the application
touch tmp/restart.txt

# 3. Clear any cached data (if using cache)
# In Django shell or via management command:
python manage.py shell
>>> from django.core.cache import cache
>>> cache.clear()
>>> exit()
```

## Verification

After deployment, verify the fix by:

1. **Check a rolled-over loan:**
   - Find a loan with `is_rolled_over=True` or `status='rolled_over'`
   - Note its loan number (e.g., LOAN-000128)

2. **Check reports:**
   - Go to `/reports/loans-due/enhanced/`
   - Search for the rolled-over loan number
   - It should NOT appear in the results

3. **Check dashboard:**
   - Go to the main dashboard
   - The "Loans Due Today" widget should not show rolled-over loans
   - The "Total Active Loans" count should exclude rolled-over loans

4. **Run the test script:**
   ```bash
   python test_rollover_exclusion_fix.py
   ```

## Rollback Plan

If issues arise, revert the changes:

```bash
# Revert to previous version
git checkout HEAD~1 reports/simple_reports_service.py
git checkout HEAD~1 reports/comprehensive_reports.py
git checkout HEAD~1 reports/views.py

# Restart application
touch tmp/restart.txt
```

## Notes

- This fix is **backward compatible** - no database schema changes required
- The fix uses both `is_rolled_over=False` filter AND `.exclude(status='rolled_over')` for maximum safety
- All existing reports will automatically benefit from this fix
- No changes needed to templates or frontend code

## Related Requirements

This fix addresses:
- **Requirement 4.2:** Rolled-over loans should not appear in active loan reports
- **Requirement 4.3:** Rolled-over loans should have both status='rolled_over' AND is_rolled_over=True
- **Requirement 5.1:** Soft-deleted loans should not appear in reports
- **Requirement 5.2:** Reports should only show truly active loans

## Future Improvements

Consider adding:
1. A database constraint to ensure `is_rolled_over=True` implies `status='rolled_over'`
2. A management command to audit and fix any inconsistent rollover states
3. Unit tests for all report methods to ensure rolled-over loans are excluded
4. A "Rolled-over Loans" report page to view historical rollover data