# Comprehensive Branch and Portfolio Filtering Implementation

## Overview
This document describes the comprehensive filtering system implemented across all views, reports, and dashboards to ensure proper data access control based on user roles, branches, and portfolio assignments.

## Filtering Rules

### 1. **Superusers**
- See **ALL** data across all branches
- Can optionally filter by branch using the branch selector
- No portfolio restrictions

### 2. **Admin Users**
- See **ALL** data across all branches
- Can optionally filter by branch using the branch selector
- No portfolio restrictions

### 3. **Loan Officers & Team Leaders**
- See **ONLY** clients in their portfolio (assigned to them)
- See **ONLY** loans, applications, and repayments for their portfolio clients
- Branch filter is applied on top of portfolio filter if selected
- Cannot see data from other loan officers' portfolios

### 4. **Secretaries & Auditors**
- See **ONLY** data from their assigned branch
- See all clients, loans, and transactions within their branch
- Cannot see data from other branches
- No portfolio restrictions within their branch

## Implementation

### Core Filtering Module: `utils/filtering.py`

The centralized filtering module provides:

#### Main Functions:

1. **`apply_branch_and_portfolio_filters(queryset, user, selected_branch_id, model_type)`**
   - Main filtering function that applies appropriate filters based on user role
   - Handles all model types: loans, clients, repayments, applications, documents, notifications

2. **`get_filtered_clients(user, selected_branch_id)`**
   - Returns clients filtered by branch and portfolio

3. **`get_filtered_loans(user, selected_branch_id)`**
   - Returns loans filtered by branch and portfolio

4. **`get_filtered_applications(user, selected_branch_id)`**
   - Returns loan applications filtered by branch and portfolio

5. **`get_filtered_repayments(user, selected_branch_id)`**
   - Returns repayments filtered by branch and portfolio

6. **`get_filtered_documents(user, selected_branch_id)`**
   - Returns documents filtered by branch and portfolio

7. **`get_filtered_notifications(user, selected_branch_id)`**
   - Returns notifications filtered by branch and portfolio

8. **`get_user_context(user)`**
   - Returns user context including role, branch, and portfolio information

## Updated Views

### 1. Reports Dashboard (`reports/views.py`)
All report functions now use centralized filtering:

- ✓ `reports_dashboard` - Main reports dashboard
- ✓ `enhanced_processing_fees_report` - Processing fees report
- ✓ `enhanced_interest_income_report` - Interest income report
- ✓ `enhanced_registration_fees_report` - Registration fees report
- ✓ `loans_due_report` - Loans due today/overdue
- ✓ `delinquent_loans_report` - Delinquent loans analysis
- ✓ `overdue_loans_report` - Overdue loans report
- ✓ `completed_loans_report` - Completed loans report
- ✓ `missed_payments_report` - Missed payments report
- ✓ `client_growth_report` - Client growth analytics
- ✓ `customer_requests_report` - Customer requests report

### 2. Loans Views (`loans/views.py`)
All loan-related views now use centralized filtering:

- ✓ `dashboard` - Main loans dashboard
- ✓ `filtered_loans` - Filtered loans by status
- ✓ `rolled_over_loans` - Rolled over loans
- ✓ `filtered_applications` - Filtered applications
- ✓ `loan_applications` - All loan applications
- ✓ `new_application` - Create new application

### 3. Utils Views (`utils/views.py`)
All utility views now use centralized filtering:

- ✓ `reports_dashboard` - Utils reports dashboard
- ✓ Document views
- ✓ Notification views

### 4. Users Views (`users/views.py`)
All user-related views now use centralized filtering:

- ✓ Client list views
- ✓ Client detail views
- ✓ Portfolio management views

## Dashboard Sections Affected

All sections in the Reports & Statements Dashboard now respect filtering:

### Portfolio Overview
- ✓ Active Loans count
- ✓ Portfolio Value
- ✓ Outstanding amount
- ✓ Collection Rate

### Loans Due Today
- ✓ Shows only loans from user's branch/portfolio
- ✓ Daily, Weekly, Monthly breakdown

### Delinquent Loans
- ✓ 1-30 Days Overdue
- ✓ 31-60 Days Overdue
- ✓ 60+ Days Overdue

### Processing Fees
- ✓ Current month processing fees
- ✓ Total fees collected
- ✓ Number of loans processed

### Interest Income
- ✓ Current month interest income
- ✓ Total interest collected
- ✓ Interest-bearing loans count

### Registration Fees
- ✓ Current month registration fees
- ✓ Total registrations
- ✓ Registration income

### Customer Requests
- ✓ Pending requests
- ✓ In-progress requests
- ✓ Resolved requests

### Loans in Arrears
- ✓ 1-30 Days overdue
- ✓ 60+ Days overdue
- ✓ Total overdue amount

### Missed Payments
- ✓ Daily missed payments
- ✓ Weekly missed payments
- ✓ Monthly missed payments

### Completed Loans
- ✓ Total completed
- ✓ This month completed
- ✓ Total collected

### Client Growth Analytics
- ✓ Total clients
- ✓ New clients this week
- ✓ New clients this month
- ✓ Growth rate

## Testing

### Test Script: `test_comprehensive_filtering.py`

Run the test script to verify filtering works correctly:

```bash
python test_comprehensive_filtering.py
```

The script tests:
1. Admin user access (should see all data)
2. Loan officer access (should see only their portfolio)
3. Secretary access (should see only their branch)
4. Auditor access (should see only their branch)
5. Branch filtering for each role

### Manual Testing Checklist

#### As Admin:
- [ ] Can see all clients across all branches
- [ ] Can see all loans across all branches
- [ ] Branch selector filters data correctly
- [ ] All dashboard metrics show complete data

#### As Loan Officer:
- [ ] Can see only assigned clients
- [ ] Can see only loans for assigned clients
- [ ] Cannot see other loan officers' clients
- [ ] Dashboard shows only portfolio data
- [ ] All reports show only portfolio data

#### As Secretary:
- [ ] Can see only clients in their branch
- [ ] Can see only loans in their branch
- [ ] Cannot see data from other branches
- [ ] Dashboard shows only branch data
- [ ] All reports show only branch data

#### As Auditor:
- [ ] Can see only data in their branch
- [ ] Cannot see data from other branches
- [ ] Dashboard shows only branch data
- [ ] All reports show only branch data

## Benefits

1. **Security**: Users can only access data they're authorized to see
2. **Performance**: Queries are optimized with proper filtering
3. **Consistency**: All views use the same filtering logic
4. **Maintainability**: Centralized filtering makes updates easier
5. **Scalability**: Easy to add new model types or roles

## Usage Examples

### In Views:

```python
from utils.filtering import get_filtered_loans, get_filtered_clients

@login_required
def my_view(request):
    selected_branch_id = request.session.get('selected_branch_id')
    
    # Get filtered data
    loans = get_filtered_loans(request.user, selected_branch_id)
    clients = get_filtered_clients(request.user, selected_branch_id)
    
    # Use the filtered querysets
    total_loans = loans.count()
    active_clients = clients.filter(is_active=True).count()
    
    # ... rest of view logic
```

### In Reports:

```python
from utils.filtering import apply_branch_and_portfolio_filters

@login_required
def custom_report(request):
    selected_branch_id = request.session.get('selected_branch_id')
    
    # Start with base queryset
    loans_qs = Loan.objects.filter(status='active')
    
    # Apply filtering
    loans_qs = apply_branch_and_portfolio_filters(
        loans_qs, 
        request.user, 
        selected_branch_id, 
        'loan'
    )
    
    # Generate report with filtered data
    # ... report logic
```

## Troubleshooting

### Issue: User sees no data
**Solution**: Check that:
1. User has a branch assigned (for secretaries/auditors)
2. User has clients assigned to their portfolio (for loan officers)
3. User role is set correctly

### Issue: User sees too much data
**Solution**: Check that:
1. User is not a superuser (unless intended)
2. User role is not 'admin' (unless intended)
3. Filtering is being applied in the view

### Issue: Branch filter not working
**Solution**: Check that:
1. `selected_branch_id` is being retrieved from session
2. Branch ID is being passed to filtering functions
3. User has permission to view that branch

## Future Enhancements

1. Add region-level filtering for multi-region deployments
2. Add time-based access restrictions
3. Add audit logging for data access
4. Add data export restrictions based on role
5. Add custom permission groups for fine-grained control

## Support

For issues or questions about filtering:
1. Check this documentation
2. Run the test script to verify filtering
3. Check user roles and branch assignments
4. Review the `utils/filtering.py` module
5. Check view implementations for proper filtering usage