# Guide: Fix Manual Payments Issue

## Problem
Payments are showing as "Manual" instead of "⚡ Auto (SasaPay)" because SasaPay IPNs (payment notifications) are not being received or are failing to process.

## Quick Diagnosis

### Step 1: Upload diagnostic script to production server

```bash
# On your local machine
scp diagnose_production_ipn.py user@grazuri.uzuriapps.xyz:/home/grazuri/branch_system/

# SSH into production
ssh user@grazuri.uzuriapps.xyz

# Navigate to project directory
cd /home/grazuri/branch_system/

# Run diagnostic
python diagnose_production_ipn.py
```

### Step 2: Review the diagnostic output

The script will tell you:
- ✅ Whether IPNs are arriving from SasaPay
- ⚠️ Why IPNs are failing to create automatic payments
- 📊 Breakdown of manual vs automatic payments
- 🔍 Specific reasons for each failed IPN

---

## Common Issues & Solutions

### Issue 1: NO IPNs ARRIVING (Most Common)

**Symptoms:**
- Script shows "Total IPNs received: 0"
- All payments are manual
- SMS messages work but payments must be entered manually

**Root Cause:** 
SasaPay is not sending payment notifications to your webhook.

**Solution:**

1. **Login to SasaPay Merchant Portal**
   - URL: https://merchant.sasapay.co.ke/ (or your SasaPay portal URL)
   - Use your merchant credentials

2. **Configure IPN URL**
   - Navigate to: **Settings** → **IPN/Callback Configuration**
   - Set **IPN URL** to: `https://grazuri.uzuriapps.xyz/payments/sasapay/ipn/`
   - Click **Verify URL** (if available)
   - Click **Save**

3. **Test Configuration**
   - Make a small test payment (KES 10)
   - Wait 1-2 minutes
   - Run diagnostic script again
   - Check if IPN was received

4. **If still not working:**
   ```bash
   # Check if webhook is accessible from internet
   curl https://grazuri.uzuriapps.xyz/payments/sasapay/ipn/
   
   # Should return 200 OK with message about SasaPay service version
   ```
   
   - Check firewall settings (port 443 must be open)
   - Check SSL certificate is valid
   - Contact SasaPay support to verify configuration

---

### Issue 2: IPNs ARRIVING BUT FAILING TO MATCH BORROWERS

**Symptoms:**
- Script shows IPNs with status: `no_match`, `no_loan`, `no_balance`
- Some payments automatic, some manual
- Failed IPNs listed in diagnostic output

**Root Cause:** 
IPN arrives but cannot match payment to a borrower due to:
- Phone number format mismatch
- BillRef doesn't match any borrower
- No active loan found

**Solution:**

1. **Check Phone Number Formats**
   
   The system tries to match using:
   - BillRef as phone number (primary method)
   - BillRef as ID number
   - BillRef as loan number
   - MSISDN (usually hashed by SasaPay, rarely works)

   **Phone format variants tried:**
   - `+254XXXXXXXXX`
   - `07XXXXXXXX`
   - `7XXXXXXXX`
   - `254XXXXXXXXX`

2. **Fix Borrower Records**
   
   ```python
   # Run in Django shell on production
   python manage.py shell
   
   # Check borrower phone formats
   from users.models import CustomUser
   
   # Find borrowers with non-standard phone formats
   borrowers = CustomUser.objects.filter(role='borrower', is_active=True)
   for b in borrowers:
       phone = b.phone_number
       if not phone.startswith('+254'):
           print(f"{b.get_full_name()}: {phone} (should be +254XXXXXXXXX)")
   ```

3. **Standardize Phone Numbers**
   
   All borrower phone numbers should be in format: `+254XXXXXXXXX`
   
   Example:
   - ❌ `0722909678`
   - ❌ `722909678`
   - ✅ `+254722909678`

4. **Configure BillRef in SasaPay**
   
   When customers make payments, BillRef should contain:
   - **Option 1:** Borrower phone number (e.g., `+254722909678`)
   - **Option 2:** Borrower ID number (if stored in system)
   - **Option 3:** Loan number (e.g., `BSH/202603/00178`)

5. **View Failed IPNs in Admin**
   
   - URL: `https://grazuri.uzuriapps.xyz/payments/sasapay/ipn-gaps/`
   - Shows all IPNs that didn't create payments
   - Can manually resolve each one

6. **View Unknown Payments**
   
   - URL: `https://grazuri.uzuriapps.xyz/payments/sasapay/unknown-payments/`
   - Shows payments that couldn't be matched
   - Manually assign to correct borrower/loan

---

### Issue 3: PROCESSING FEE PAYMENTS

**Symptoms:**
- IPN status: `no_loan`
- Payment made before loan approval

**This is EXPECTED behavior:**
- Processing fee payments come BEFORE loan is approved
- System logs these as `no_loan` status
- Sends processing fee confirmation SMS
- Saved in `unknown_payments` for admin review

**No action needed** - this is working correctly.

---

## Manual Recovery of Payments

If IPNs were not configured and you have backlog of manual payments:

### Option 1: Check for Logged IPNs

```bash
# SSH into production
ssh user@grazuri.uzuriapps.xyz
cd /home/grazuri/branch_system/

# Run recovery script
python manage.py shell
```

```python
from payments.sasapay_models import SasaPayIPNLog
from datetime import datetime, timedelta

# Check if any IPNs were received but not processed
seven_days_ago = datetime.now() - timedelta(days=7)
failed = SasaPayIPNLog.objects.filter(
    created_at__gte=seven_days_ago,
    processing_status='failed'
)

print(f"Failed IPNs: {failed.count()}")

# These can be manually recovered via admin panel:
# /payments/admin/recover-ipn-payments/
```

### Option 2: Nothing to Recover

If NO IPNs were ever received (IPN URL was never configured):
- ❌ **No automatic recovery possible**
- ✅ **Manual payments entered by staff are correct**
- ✅ **Once IPN is configured, future payments will be automatic**

---

## Testing After Configuration

### 1. Make Test Payment

```bash
# Small test amount
Amount: KES 10
BillRef: [borrower phone number, e.g., +254722909678]
```

### 2. Check IPN Arrived

```bash
# Run diagnostic again
python diagnose_production_ipn.py
```

Should show:
```
Total IPNs received: 1
IPN Processing Status:
   PROCESSED: 1 (100%)
```

### 3. Verify Payment Created

Check admin panel:
- URL: `https://grazuri.uzuriapps.xyz/loans/repayments/`
- Latest payment should show: **⚡ Auto (SasaPay)**

### 4. Check SMS Sent

- URL: `https://grazuri.uzuriapps.xyz/payments/sms/`
- Should show confirmation SMS with payment details

---

## Monitoring Going Forward

### Daily Check

Visit IPN Gaps dashboard:
- URL: `https://grazuri.uzuriapps.xyz/payments/sasapay/ipn-gaps/`
- Look for any failed IPNs
- Resolve issues as they appear

### Weekly Review

Run diagnostic script weekly:
```bash
python diagnose_production_ipn.py
```

Expected healthy output:
```
✅ SYSTEM IS WORKING CORRECTLY!
  - IPNs are being received from SasaPay
  - Payments are being processed automatically
  - No issues detected
```

---

## Quick Reference

| Issue | Location | Action |
|-------|----------|--------|
| No IPNs arriving | SasaPay Merchant Portal | Configure IPN URL |
| Failed IPNs | `/payments/sasapay/ipn-gaps/` | Review and fix matching |
| Unknown payments | `/payments/sasapay/unknown-payments/` | Manually assign to loans |
| SMS logs | `/payments/sms/` | View all SMS sent |
| Phone format issues | Admin → Users → Borrowers | Standardize to +254X format |

---

## Need Help?

If diagnostic script shows issues you can't resolve:

1. **Check the detailed output** - it shows exactly why each IPN failed
2. **Review borrower phone numbers** - most common issue
3. **Verify IPN URL in SasaPay portal** - must match exactly
4. **Test webhook accessibility** - firewall/SSL issues
5. **Contact SasaPay support** - for portal configuration help

---

## Summary

The system IS designed to automatically process payments from SasaPay, but:

1. **IPN URL must be configured in SasaPay portal** ← Most common issue
2. **Borrower phone numbers must match BillRef format**
3. **All configuration is in place in your Django app** ✅
4. **Webhook endpoint is working and accessible** ✅

Once IPN is configured, payments will automatically:
- ✅ Create repayment records
- ✅ Update loan balances
- ✅ Send SMS confirmations
- ✅ Generate receipts
- ✅ Show as "⚡ Auto (SasaPay)" in admin

**Run the diagnostic script first to see exactly what the issue is!**