Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
311 lines (242 loc) · 12.7 KB

ARCHITECTURE.md

File metadata and controls

311 lines (242 loc) · 12.7 KB

Foundation Security Services Setup - Architecture

This document describes the architectural design, implementation patterns, and current status of the OpenSecOps Foundation security services setup component.

🏗️ System Architecture Overview

Purpose & Scope

The Foundation Security Services Setup is a Python-based validation and configuration tool for AWS security services across organizational accounts. It provides:

  • Comprehensive Discovery: Automated detection of current security service configurations
  • Standards Validation: Verification against OpenSecOps security standards
  • Gap Analysis: Identification of missing configurations and security weaknesses
  • Cost Optimization: Detection of spurious activations in unexpected regions
  • Dry-Run Safety: Preview mode for safe validation without making changes

Supported Security Services

The system manages six core AWS security services:

  1. Amazon GuardDuty - Threat detection service
  2. AWS Security Hub - Centralized security findings management
  3. Amazon Detective - Security investigation service
  4. Amazon Inspector - Vulnerability assessment service
  5. AWS Config - Resource configuration compliance
  6. IAM Access Analyzer - Access analysis and unused access detection

🎯 Core Architectural Principles

1. Functional Composition Over Inheritance

  • Factory Functions: create_service_status() for type-safe object creation
  • Pure Functions: Stateless functions with predictable inputs/outputs
  • Composable Operations: Building complex workflows from simple, reusable components

2. Type Safety & Data Integrity

  • Dataclass-Based Status Objects: Type-safe, validated data structures
  • Standardized Field Names: Consistent naming across all services
  • Backward Compatibility: Dictionary conversion for legacy integration

3. Cross-Account AWS Operations

  • Role Assumption: Automated cross-account access using Control Tower execution roles
  • Multi-Account Discovery: Organization-wide security service visibility
  • Permission Isolation: Minimal required permissions with fail-safe error handling

4. Zero-Cost Testing Architecture

  • Global Mocking Strategy: Comprehensive AWS API call prevention
  • Performance Optimization: <3 second test execution for 221 tests
  • Security Isolation: Zero risk of real AWS charges during development

📊 Data Architecture

Service Status Hierarchy

@dataclass
class ServiceRegionStatus:
    """Base class for all service region status objects"""
    region: str
    service_enabled: bool = False
    needs_changes: bool = False
    delegation_status: str = 'unknown'
    member_count: int = 0
    issues: List[str] = field(default_factory=list)
    actions: List[str] = field(default_factory=list)
    service_details: List[str] = field(default_factory=list)
    errors: List[str] = field(default_factory=list)

Service-Specific Extensions:

  • GuardDutyRegionStatus: Adds organization_auto_enable
  • SecurityHubRegionStatus: Adds hub ARN, control policies, finding aggregation
  • DetectiveRegionStatus: Adds graph_arn for investigation graphs
  • InspectorRegionStatus: Adds scan_types_enabled count
  • AccessAnalyzerRegionStatus: Adds analyzer type counts
  • ConfigRegionStatus: Adds records_global_iam flag

Anomalous Region Detection

@dataclass
class AnomalousRegionStatus:
    """Standardized structure for unexpected service activations"""
    region: str
    resource_count: int
    resource_details: List[Dict[str, Any]] = field(default_factory=list)
    account_details: List[Dict[str, Any]] = field(default_factory=list)

🔧 Component Architecture

1. Service Modules Pattern

Each security service follows a consistent modular pattern:

# modules/[service_name].py
def setup_[service](enabled, params, dry_run, verbose):
    """Main entry point for service configuration"""
    # 1. Parameter validation and logging
    # 2. Delegation status checking
    # 3. Regional configuration analysis  
    # 4. Anomalous region detection
    # 5. Gap analysis and recommendations
    # 6. Dry-run preview or actual changes

def check_[service]_in_region(region, admin_account, security_account, 
                             cross_account_role, verbose=False):
    """Region-specific configuration analysis"""
    # Returns standardized ServiceRegionStatus object

2. Shared Utilities (modules/utils.py)

Core Infrastructure:

  • get_client(): Cross-account AWS client creation with role assumption
  • printc(): Colored console output for user feedback
  • Color constants: RED, GREEN, YELLOW, LIGHT_BLUE, GRAY, END, BOLD

Architectural Components:

  • DelegationChecker: Organization-wide service delegation verification
  • AnomalousRegionChecker: Standardized unexpected resource detection
  • Factory functions: create_service_status(), create_anomalous_status()

3. DelegationChecker Pattern

Centralized Delegation Logic:

class DelegationChecker:
    @staticmethod
    def check_service_delegation(service_principal, admin_account, 
                               security_account, cross_account_role, verbose=False):
        """Uniform delegation checking across all services"""
        # Returns standardized delegation status with error handling

Usage Pattern:

delegation_result = DelegationChecker.check_service_delegation(
    service_principal='guardduty.amazonaws.com',
    admin_account=admin_account,
    security_account=security_account,
    cross_account_role=cross_account_role,
    verbose=verbose
)

4. AnomalousRegionChecker Pattern

Centralized Anomaly Detection:

class AnomalousRegionChecker:
    @staticmethod
    def check_service_anomalous_regions(service_name, expected_regions,
                                      admin_account, security_account, 
                                      cross_account_role, verbose=False):
        """Parameterized anomaly detection for all services"""
        # Service-specific configuration via internal mapping
        # Standardized resource detection and account detail collection

Service Configuration Mapping:

  • Service-specific API patterns (list methods, pagination, exceptions)
  • Cross-account support flags and member detection
  • Resource detail extraction and account visibility

🧪 Testing Architecture

📚 For comprehensive testing architecture, standards, and implementation guidance, see TESTING.md

Foundation Testing Summary

Achievement Metrics:

  • ✅ 185/185 tests passing (100% success rate)
  • ✅ 98% performance improvement (77+ seconds → 2.81 seconds)
  • ✅ 99.7% warning reduction (4661+ warnings → 13 warnings)
  • ✅ Zero AWS costs during testing (comprehensive mocking)

Key Innovation: Data-Driven Mock Configuration Foundation pioneered the SERVICE_MOCK_CONFIGS pattern that eliminates complex case structures while providing comprehensive AWS service simulation.

Testing Implementation Details: See TESTING.md for complete patterns, TDD methodology, and performance optimization techniques.

📈 Current Status & Metrics

Implementation Completeness

✅ Fully Implemented (100%):

  • ✅ All 6 security services with standardized status structures
  • ✅ Cross-account delegation checking via DelegationChecker
  • ✅ Anomalous region detection via AnomalousRegionChecker
  • ✅ Type-safe dataclass architecture with factory functions
  • ✅ Dataclass Direct Usage: Complete elimination of dictionary conversion layer
  • ✅ Comprehensive testing with zero AWS cost guarantee
  • ✅ Professional output formatting for enterprise deployment

Architecture Quality

Design Consistency:

  • Pattern Uniformity: DelegationChecker and AnomalousRegionChecker follow identical class-based patterns
  • Import Consistency: All modules use same import pattern from utils
  • Type Safety: Full dataclass usage throughout call stack
  • Clean Architecture: No legacy scaffolding or obsolete code

Implementation Quality:

  • Consolidated Logic: Anomalous region detection unified through AnomalousRegionChecker
  • Standardized Field Names: Consistent naming across all services (resource_count, account_details)
  • Clean Data Flow: Direct dataclass usage throughout call stack

Enhanced Security Features

Account-Level Visibility:

  • ✅ GuardDuty: Admin account + member account details with detector status
  • ✅ Security Hub: Admin account + member account details with hub status
  • ✅ Detective: Graph-based member account detection
  • ✅ Inspector: Embedded account scanning status details
  • ✅ Config: Configuration recorder status per region
  • ✅ Access Analyzer: Analyzer status with organization-wide scope

🔮 Future Enhancement Opportunities

Architectural Evolution

Enhanced Error Recovery:

  • Current: Basic error handling with error arrays
  • Future: Structured error types with recovery suggestions
  • Benefits: Better user guidance for permission and configuration issues

Functional Enhancements

Advanced Anomaly Detection:

  • Current: Resource presence detection
  • Future: Configuration drift analysis (unexpected settings, policy changes)
  • Benefits: Deeper security posture monitoring

Cost Optimization Intelligence:

  • Current: Basic unexpected region detection
  • Future: Cost impact analysis with specific charge estimates
  • Benefits: Quantified financial impact of configuration drift

Multi-Organization Support:

  • Current: Single organization analysis
  • Future: Cross-organization security service comparison
  • Benefits: Multi-tenant management capabilities

Integration Opportunities

Infrastructure as Code Integration:

  • Current: Discovery and validation tool
  • Future: Generate Terraform/CloudFormation from current state
  • Benefits: Configuration as code workflows

CI/CD Pipeline Integration:

  • Current: Manual execution
  • Future: Automated security configuration validation in pipelines
  • Benefits: Continuous security compliance verification

Infrastructure Mutation Pattern Evolution

Strategic Pattern Foundation: The DelegationChecker and AnomalousRegionChecker patterns establish a proven architectural foundation that will be invaluable for future infrastructure mutation capabilities:

Current Read-Only Pattern:

class DelegationChecker:
    @staticmethod
    def check_service_delegation(service_principal, admin_account, ...)
    
class AnomalousRegionChecker:
    @staticmethod 
    def check_service_anomalous_regions(service_name, expected_regions, ...)

Future Mutation Pattern Extensions:

class ServiceDeployer:
    @staticmethod
    def deploy_service_configuration(service_name, target_regions, ...)
    
class ServiceUpdater:
    @staticmethod
    def update_service_settings(service_name, configuration_changes, ...)
    
class ServiceTeardown:
    @staticmethod
    def teardown_service_resources(service_name, target_regions, ...)

Pattern Benefits for Infrastructure Mutation:

  • Consistent API: Same parameterized approach for all CRUD operations
  • Cross-Account Support: Role assumption patterns already proven
  • Service Abstraction: Generic service handling with specific configurations
  • Error Handling: Established patterns for AWS API failures and permissions
  • Proven Patterns: Architectural patterns validated in production

Strategic Value: This pattern foundation means adding infrastructure deployment, updates, and teardown capabilities will follow the same proven architectural principles, ensuring consistency and maintainability as the system evolves from read-only validation to full infrastructure lifecycle management.

🏆 Architecture Strengths

Design Excellence

  1. Consistency: Uniform patterns across all components
  2. Maintainability: Clear separation of concerns and functional composition
  3. Scalability: Easily extensible to additional AWS security services
  4. Type Safety: Dataclass-based architecture with validation

Operational Excellence

  1. Reliability: Comprehensive error handling and graceful degradation
  2. Security: Safe cross-account operations with proper role isolation
  3. Usability: Professional output formatting with clear recommendations
  4. Documentation: Comprehensive inline documentation and architectural clarity

The Foundation Security Services Setup represents a mature, production-ready implementation with consistent architectural patterns and proven scalability. The codebase demonstrates professional software engineering practices suitable for enterprise security operations.