Contact Index Management SOP

graph LR
  A[LACRM Contact Download] --> D[Email Extraction]
  B[Google Contacts] --> D
  C[Gmail Participants] --> D
  D --> E[Email Normalization]
  E --> F[Duplicate Resolution]
  F --> G[Contact Index Build]
  G --> H[Validation & Storage]

1 Purpose

Maintain a unified contact resolution system that maps email addresses to canonical contact identifiers across all integrated platforms, enabling accurate contact attribution for meetings, emails, and client interactions.

2 Triggers

Scheduled: Daily at 6 AM UTC via automated pipeline
Manual: Operator execution when contact resolution issues detected
Event-driven: New client onboarding, contact database changes, integration issues

3 Inputs

LACRM contact database via API (contacts, emails, names)
Google Contacts exports with email addresses and contact details
Gmail message participants (sender/recipient email extraction)
Existing contact index for incremental updates and change detection

4 Steps

4.1 1. LACRM Contact Extraction

API Call: GetContacts with pagination to retrieve all active contacts
Data Fields: Contact ID, first name, last name, primary email, all email addresses
Normalization: Clean email formats, remove duplicates, standardize names
Validation: Verify email format compliance and contact completeness

4.2 2. Google Contacts Integration

OAuth Access: Use Google Workspace token for contacts API access
Data Retrieval: Download all workspace contacts with email addresses
Deduplication: Identify contacts already present in LACRM system
Email Extraction: Extract all email addresses per contact for comprehensive mapping

4.3 3. Gmail Participant Analysis

Message Metadata: Process Gmail API message headers (no content)
Email Discovery: Extract sender/recipient addresses from recent communications
Pattern Recognition: Identify frequently contacted individuals not in formal contact systems
Validation: Verify email format and filter internal/system addresses

4.4 4. Email Normalization

Format Standardization: Convert all emails to lowercase, trim whitespace
Domain Validation: Verify email domain format and basic syntax compliance
Duplicate Detection: Identify multiple email addresses belonging to same individual
Canonical Assignment: Assign primary email address for each unique contact

4.5 5. Contact Index Construction

Primary Mapping: Email address → LACRM Contact ID (when available)
Name Resolution: Associate names with email addresses for fuzzy matching
Confidence Scoring: Assign confidence levels for contact attribution accuracy
Metadata Enrichment: Include last seen date, communication frequency, source systems

4.6 6. Validation and Storage

Quality Checks: Verify index completeness and identify missing critical contacts
Format Validation: Ensure JSON structure compliance for downstream consumers
Change Detection: Compare with previous index to identify additions/updates
Atomic Update: Replace index file atomically to prevent partial reads

5 Exceptions

5.1 Missing LACRM Contacts

Detection: Email addresses with high interaction frequency but no LACRM mapping
Response: Generate report for manual contact creation review
Escalation: Alert client experience team for contact database maintenance

5.2 Duplicate Email Detection

Scenario: Same email address associated with multiple LACRM contacts
Resolution: Flag for manual review, temporarily assign to most recent contact
Prevention: Implement LACRM deduplication workflow and data quality checks

5.3 API Authentication Failures

LACRM Issues: API key expiration or rate limiting
Google Issues: OAuth token refresh failures or scope limitations
Recovery: Automatic retry with exponential backoff, manual escalation after 3 failures

5.4 Data Quality Issues

Invalid Emails: Malformed email addresses or system-generated addresses
Missing Names: Email addresses without associated contact names
Orphaned Records: Contacts in index but deleted from source systems

6 Owner Handoffs

Data Engineering → Client Experience for contact creation and assignment decisions
Data Engineering → Compliance for PII handling and data retention issues
Client Experience → Data Engineering for contact merge and deduplication requests

7 SLAs

Daily Index Build: Complete within 30 minutes of 6 AM UTC start
Manual Rebuild: Complete within 15 minutes for immediate resolution needs
Error Resolution: Automatic retry within 10 minutes, manual escalation at 1 hour
Data Freshness: Index reflects latest contact changes within 24 hours

8 Controls

Email Validation: RFC-compliant format checking and domain verification
PII Protection: Contact data encrypted in transit and at rest
Access Control: Index file permissions restricted to authorized automation accounts
Audit Logging: Complete change tracking with timestamp and source attribution

9 Audit Artifacts

Index History: Timestamped contact index versions for change analysis
Quality Reports: Contact coverage analysis and missing contact identification
Error Logs: API failures, validation issues, and resolution outcomes
Performance Metrics: Index build time, contact count changes, coverage statistics

10 Operational Dependencies

LACRM API: Primary source of truth for contact assignments and relationships
Google Workspace: OAuth access for contacts and Gmail participant extraction
Email Systems: Message metadata for comprehensive email address discovery
Downstream Consumers: Meeting aggregation, touch analysis, contact attribution

11 Technical Architecture

Storage Format: JSON file with email-to-contact-id mappings and metadata
Update Strategy: Atomic replacement to prevent partial reads during updates
Caching: In-memory caching during processing to optimize API calls
Validation: Schema validation and data consistency checks before publication

12 Monitoring & Alerts

Coverage Metrics: Percentage of emails with successful contact attribution
Quality Indicators: Duplicate detection rate, validation failure count
Performance Tracking: Index build duration, API response times, error rates
Business Impact: Meeting attribution accuracy, email touch analysis completeness

13 FAQs

How are personal emails handled vs. business emails? All email addresses are indexed regardless of domain. Personal emails are often used for client communications and require accurate attribution.

What happens when a contact has multiple email addresses? All emails are mapped to the same LACRM contact ID. The system handles one-to-many email-to-contact relationships.

How are deleted contacts handled? Contacts deleted from LACRM are removed from the index during the next rebuild. Historical data remains in archived index versions for audit purposes.

Can the index be used for marketing segmentation? The index is designed for operational contact resolution only. Marketing segmentation should use LACRM tags and custom fields directly.