gpt_chrome_autodrafter_v3/integration_plan.md

# Integration Plan for Enhanced Thread Extractor

## Overview
This document outlines how to integrate the enhanced thread extraction system with the existing Gmail Auto-Draft extension.

## Key Improvements in Enhanced System

### 1. **Batch Extraction Enhancements**
- Dedicated `extractBatchThreadIds()` method with multiple fallback strategies
- Detailed logging of extraction methods and success rates
- Option to return full thread info (subject, sender) for better debugging
- Click-based extraction as last resort (optional)

### 2. **One-off Draft Improvements**
- More reliable thread ID extraction with 7+ fallback methods
- Better handling of different Gmail views (conversation, search, label)
- Automatic token refresh and retry logic
- Enhanced message ID extraction when thread ID fails

### 3. **Performance & Reliability**
- Intelligent caching with automatic invalidation
- Session storage for thread history tracking
- Continuous monitoring with event dispatching
- Graceful degradation when methods fail

## Integration Steps

### Step 1: Add Enhanced Extractor to Extension

1. **Add to manifest.json** (web_accessible_resources):
```json
{
  "resources": ["email_parser.js", "enhanced_thread_extractor.js"],
  "matches": ["https://mail.google.com/*"]
}
```

2. **Load in content.js** (at the top):
```javascript
// Load enhanced thread extractor
const script = document.createElement('script');
script.src = chrome.runtime.getURL('enhanced_thread_extractor.js');
script.onload = function() {
  console.log('Enhanced thread extractor loaded');
  // The extractor auto-initializes and starts monitoring
};
(document.head || document.documentElement).appendChild(script);
```

### Step 2: Update Existing Functions

Replace current `extractThreadId` and `getSelectedThreadIds` functions with the enhanced versions:

```javascript
// In content.js - Remove old implementations and use enhanced ones
// The enhanced extractor provides backward-compatible functions:
// - extractThreadId(context)
// - getSelectedThreadIds()
```

### Step 3: Enhance Batch Draft Function

Update `generateDraftsForThreadsHybrid` to use enhanced extraction:

```javascript
async function generateDraftsForThreadsHybrid() {
  try {
    updateProgressUI('Starting', 'Preparing to generate drafts...');
    
    const token = await getOAuthTokenFromBackground();
    if (!token) throw new Error('Failed to get OAuth token');

    // Use enhanced extraction with full info
    const threadInfo = await threadExtractor.extractThreadId('selected', {
      returnFullInfo: true,
      allowClickExtraction: false // Disable for batch to avoid UI disruption
    });
    
    if (!threadInfo || !threadInfo.length) {
      updateProgressUI('Error', 'Please select at least one email thread.', true);
      return;
    }
    
    updateProgressUI('Processing', `Found ${threadInfo.length} selected threads`);
    
    // Process each thread with better error handling
    let successCount = 0;
    let errorCount = 0;
    
    for (let i = 0; i < threadInfo.length; i++) {
      const info = threadInfo[i];
      
      try {
        updateProgressUI('Processing', 
          `Generating draft ${i + 1}/${threadInfo.length}: "${info.subject}" from ${info.sender}...`);
        
        // Continue with existing logic using info.threadId
        // ...existing code...
      } catch (error) {
        console.error(`Failed to process thread ${info.threadId}:`, error);
        errorCount++;
      }
    }
    
    // ... rest of function
  } catch (error) {
    console.error('Batch draft generation failed:', error);
    updateProgressUI('Error', error.message, true);
  }
}
```

### Step 4: Enhance One-off Draft Function

Update `oneOffAutoDraftForCurrentThread` with retry logic:

```javascript
async function oneOffAutoDraftForCurrentThread(contextPrompt = null) {
  try {
    updateProgressUI('Initializing', 'Getting authentication token...');
    const token = await getOAuthTokenFromBackground();
    if (!token) throw new Error('Failed to get OAuth token');

    updateProgressUI('Analyzing', 'Extracting thread ID...');
    
    // Use enhanced extraction with retries
    let threadId = null;
    let attempts = 0;
    const maxAttempts = 3;
    
    while (!threadId && attempts < maxAttempts) {
      attempts++;
      try {
        threadId = await threadExtractor.extractThreadId('current', {
          forceRefresh: attempts > 1, // Force refresh on retry
          token: token // Provide token for API fallback
        });
      } catch (error) {
        console.warn(`Attempt ${attempts} failed:`, error);
        if (attempts < maxAttempts) {
          updateProgressUI('Analyzing', `Retrying thread extraction (attempt ${attempts + 1})...`);
          await new Promise(r => setTimeout(r, 1000));
        }
      }
    }
    
    if (!threadId) {
      updateProgressUI('Error', 'Could not determine current thread ID', true);
      throw new Error('Could not determine current thread ID. Make sure you are viewing an email thread.');
    }
    
    console.log('Extracted thread ID:', threadId);
    
    // ... rest of existing function
  } catch (error) {
    console.error('Failed to create one-off autodraft:', error);
    updateProgressUI('Error', error.message, true);
    throw error;
  }
}
```

### Step 5: Add Event Listeners

Listen for thread ID detection events:

```javascript
// In content.js main()
document.addEventListener('threadIdDetected', (event) => {
  console.log('Thread ID detected:', event.detail.threadId);
  
  // Update any UI elements that depend on thread ID
  const threadIdDisplay = document.querySelector('.thread-id-display');
  if (threadIdDisplay) {
    threadIdDisplay.textContent = event.detail.threadId;
  }
  
  // Enable/disable buttons based on thread availability
  const draftButton = document.querySelector('.gpt-autodraft-button');
  if (draftButton) {
    draftButton.disabled = false;
  }
});
```

### Step 6: Add Fallback UI

When thread extraction fails completely:

```javascript
function showThreadSelectionFallback() {
  const fallbackUI = document.createElement('div');
  fallbackUI.innerHTML = `
    <div style="padding: 16px; background: #fff3cd; border: 1px solid #ffeeba; border-radius: 4px;">
      <h4>Unable to detect thread automatically</h4>
      <p>Please try one of the following:</p>
      <ul>
        <li>Refresh the page and try again</li>
        <li>Make sure you're viewing a single email thread</li>
        <li>Enter thread ID manually: <input type="text" id="manual-thread-id" placeholder="16-character ID"></li>
      </ul>
      <button id="retry-extraction">Retry</button>
      <button id="use-manual-id">Use Manual ID</button>
    </div>
  `;
  
  document.body.appendChild(fallbackUI);
  
  // Add handlers
  document.getElementById('retry-extraction').onclick = async () => {
    const threadId = await threadExtractor.extractThreadId('current', { forceRefresh: true });
    if (threadId) {
      fallbackUI.remove();
      // Continue with draft generation
    }
  };
}
```

## Testing Checklist

### Basic Functionality
- [ ] Single thread extraction works in conversation view
- [ ] Batch extraction works with multiple selected threads
- [ ] Extraction works after page navigation
- [ ] Cache properly invalidates on navigation

### Edge Cases
- [ ] Works in search results view
- [ ] Works in label/category views
- [ ] Handles threads with no visible ID in DOM
- [ ] Gracefully fails with helpful error messages

### Performance
- [ ] No noticeable lag during extraction
- [ ] Batch operations complete within reasonable time
- [ ] Memory usage remains stable with monitoring enabled

### Integration
- [ ] Backward compatibility maintained
- [ ] No conflicts with existing extension code
- [ ] All existing features continue to work

## Monitoring & Maintenance

### Success Rate Tracking
```javascript
// Add to background.js
const extractionStats = {
  attempts: 0,
  successes: 0,
  failures: {},
  
  logAttempt(method, success, error) {
    this.attempts++;
    if (success) {
      this.successes++;
    } else {
      this.failures[method] = (this.failures[method] || 0) + 1;
    }
    
    // Log stats every 100 attempts
    if (this.attempts % 100 === 0) {
      console.log('Extraction stats:', {
        successRate: (this.successes / this.attempts * 100).toFixed(2) + '%',
        failures: this.failures
      });
    }
  }
};
```

### Gmail Update Detection
Monitor for Gmail structure changes:

```javascript
// Check periodically for Gmail updates
setInterval(() => {
  const knownSelectors = [
    '[data-legacy-thread-id]',
    '[data-thread-id]',
    'div[role="checkbox"]',
    'span.bog'
  ];
  
  const missing = knownSelectors.filter(sel => 
    document.querySelectorAll(sel).length === 0
  );
  
  if (missing.length > 0) {
    console.warn('Gmail structure may have changed. Missing selectors:', missing);
    // Send telemetry or notification
  }
}, 3600000); // Check every hour
```

## Rollback Plan

If the enhanced extractor causes issues:

1. Remove script injection in content.js
2. Restore original extractThreadId and getSelectedThreadIds functions
3. Clear sessionStorage thread history: `sessionStorage.removeItem('gmail_thread_history')`
4. Remove event listeners for threadIdDetected

## Benefits Summary

1. **Reliability**: 7+ extraction methods vs current 4
2. **Performance**: Intelligent caching reduces repeated work
3. **Debugging**: Detailed logging of extraction methods
4. **Flexibility**: Options for different use cases
5. **Future-proof**: Multiple fallbacks handle Gmail updates better
6. **User Experience**: Graceful degradation with helpful errors