跳转至

第四天:企业级系统架构设计与实现

学习目标

  • 掌握企业级多智能体系统架构设计原则
  • 学会设计和实现核心系统组件
  • 掌握安全、监控和运维最佳实践
  • 学会系统性能优化和故障处理
  • 了解实际项目中的架构实现和最佳实践

参考项目

本课程深入讲解企业级多智能体系统的架构设计原则、核心组件实现和最佳实践,帮助学员构建可扩展、高可用的生产级系统。

💡 实际代码参考:完整的企业级架构实现可参考项目中的以下文件:

  • multi_agent_system/main.py - 多智能体系统主入口和整体架构
  • multi_agent_system/src/agents/base_agent.py - BDI智能体基础架构
  • multi_agent_system/src/communication/message_bus.py - 企业级消息总线
  • multi_agent_system/src/workflows/langgraph_workflow.py - 工作流引擎架构
  • multi_agent_system/src/monitoring/langsmith_integration.py - 监控系统集成
  • src/examples/customer_service_system.py - 企业级客服系统实现
  • multi_agent_system/src/agents/ - 安全组件实现
  • multi_agent_system/src/communication/ - 基础设施组件

代码引用: 完整的企业级架构实现请参考 multi_agent_system/main.py

企业级架构的核心特性:

核心组件:

  • ArchitectureLayer: 架构层次管理(表示层、业务层、服务层、数据层、基础设施层)
  • EnterpriseComponent: 企业级组件定义(依赖关系、可扩展性、可用性、安全级别)
  • EnterpriseArchitectureManager: 架构管理器,支持组件注册和系统启动
  • MultiAgentSystem: 多智能体系统主类,实现企业级架构

企业级特性:

  • 分层架构设计,清晰的职责分离
  • 组件化管理,支持动态注册和监控
  • 异步启动机制,按依赖顺序启动组件
  • 集成监控和安全策略

💡 实际代码参考:完整的企业级架构实现可参考项目中的以下文件:

  • main.py - 多智能体系统主入口和整体架构
  • src/examples/customer_service_system.py - 企业级客服系统实现
  • message_bus.py - 消息总线和通信中间件
  • langgraph_workflow.py - 工作流引擎实现
  • langsmith_integration.py - 监控系统集成

架构特色:本项目采用现代微服务架构,集成了BDI智能体模型、LangGraph工作流引擎、LangSmith监控平台,提供了完整的企业级多智能体解决方案。


1. 企业级架构设计原则

1.1 设计原则概述

1.1.1 核心设计原则

**代码引用**: 完整的企业级架构设计原则实现请参考 `multi_agent_system/main.py` 中的 `EnterpriseArchitecturalPrinciples` 和 `MultiAgentSystem` 类

企业级架构设计原则的核心特性:

**设计原则**:
- `EnterpriseArchitecturalPrinciples`: 定义了8大核心原则(可扩展性、可靠性、安全性、可维护性、性能、可观测性、灵活性、成本效率)
- `MultiAgentSystem`: 主系统类,实现企业级架构的完整生命周期管理

**系统初始化流程**:
1. 基础智能体架构初始化(BDI模型)
2. 专业化智能体初始化(业务特定智能体)
3. 通信中间件初始化(消息总线)
4. 工作流引擎初始化(LangGraph)
5. 监控系统初始化(LangSmith)
6. 示例应用初始化(客服系统)

**企业级特性**:
- 模块化设计,每个组件独立可替换
- 异步架构,支持高并发处理
- 全链路监控,集成LangSmith
- 容错设计,自动故障恢复

1.2 实际项目架构分析

1.2.1 项目架构层次

# 基于实际项目的架构层次分析
class ProjectArchitectureLayers:
    """项目架构层次"""

    def __init__(self):
        self.presentation_layer = {
            "description": "表示层 - 用户接口和API",
            "components": [
                "main.py - 系统主入口",
                "API接口 - RESTful API设计",
                "管理界面 - 系统监控面板"
            ],
            "technologies": ["FastAPI", "WebSocket", "HTTP/HTTPS"]
        }

        self.business_layer = {
            "description": "业务层 - 智能体编排和业务逻辑",
            "components": [
                "customer_service_system.py - 客服业务逻辑",
                "langgraph_workflow.py - 工作流编排",
                "业务规则引擎 - 动态规则配置"
            ],
            "technologies": ["LangGraph", "业务规则引擎", "工作流引擎"]
        }

        self.service_layer = {
            "description": "服务层 - 微服务和中间件",
            "components": [
                "base_agent.py - BDI智能体服务",
                "message_bus.py - 消息通信服务",
                "langsmith_integration.py - 监控服务"
            ],
            "technologies": ["异步消息队列", "服务发现", "负载均衡"]
        }

        self.data_layer = {
            "description": "数据层 - 数据存储和管理",
            "components": [
                "智能体状态存储",
                "消息历史存储",
                "监控数据存储",
                "配置数据存储"
            ],
            "technologies": ["Redis", "PostgreSQL", "文件存储", "时序数据库"]
        }

# 架构特性对比
class ArchitectureFeatures:
    """架构特性对比"""

    @staticmethod
    def get_feature_comparison():
        return {
            "传统单体架构": {
                "优点": ["简单部署", "开发快速", "调试容易"],
                "缺点": ["扩展困难", "技术栈固定", "故障影响全局"],
                "适用场景": "小型项目、原型开发"
            },
            "微服务架构": {
                "优点": ["独立扩展", "技术多样性", "故障隔离"],
                "缺点": ["复杂度高", "网络开销", "数据一致性"],
                "适用场景": "大型企业应用、高并发系统"
            },
            "智能体架构": {
                "优点": ["自主决策", "动态协作", "智能适应"],
                "缺点": ["行为不可预测", "调试困难", "性能开销"],
                "适用场景": "复杂业务场景、智能化应用"
            },
            "本项目架构": {
                "优点": ["结合三者优势", "企业级特性", "完整生态"],
                "缺点": ["学习成本高", "初期复杂"],
                "适用场景": "企业级AI应用、智能化业务系统"
            }
        }

1.2 分层架构模式

1.2.1 四层架构设计

class EnterpriseArchitecture:
    """企业级四层架构"""

    def __init__(self):
        self.presentation_layer = PresentationLayer()
        self.business_layer = BusinessLayer()
        self.service_layer = ServiceLayer()
        self.data_layer = DataLayer()

    class PresentationLayer:
        """表示层 - API网关、Web界面"""
        def __init__(self):
            self.api_gateway = "API网关"
            self.web_interface = "Web管理界面"
            self.mobile_app = "移动端应用"

    class BusinessLayer:
        """业务层 - 智能体编排、业务逻辑"""
        def __init__(self):
            self.agent_orchestrator = "智能体编排器"
            self.workflow_engine = "工作流引擎"
            self.business_rules = "业务规则引擎"

    class ServiceLayer:
        """服务层 - 微服务、中间件"""
        def __init__(self):
            self.agent_services = "智能体服务集群"
            self.message_queue = "消息队列"
            self.cache_service = "缓存服务"
            self.auth_service = "认证授权服务"

    class DataLayer:
        """数据层 - 数据库、存储"""
        def __init__(self):
            self.primary_database = "主数据库"
            self.cache_database = "缓存数据库"
            self.file_storage = "文件存储"
            self.log_storage = "日志存储"

1.3 微服务架构模式

1.3.1 智能体微服务设计

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import asyncio
from typing import Dict, Any

class AgentMicroservice:
    """智能体微服务基类"""

    def __init__(self, service_name: str, port: int):
        self.service_name = service_name
        self.port = port
        self.app = FastAPI(title=f"{service_name} Service")
        self.health_status = "healthy"
        self._setup_routes()

    def _setup_routes(self):
        """设置路由"""
        @self.app.get("/health")
        async def health_check():
            return {"status": self.health_status, "service": self.service_name}

        @self.app.post("/process")
        async def process_request(request: ProcessRequest):
            try:
                result = await self.process_task(request.task, request.context)
                return {"success": True, "result": result}
            except Exception as e:
                raise HTTPException(status_code=500, detail=str(e))

    @abstractmethod
    async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
        """处理任务的抽象方法"""
        pass

class ProcessRequest(BaseModel):
    task: Dict[str, Any]
    context: Dict[str, Any] = {}

# 具体智能体服务实现
class AnalyzerService(AgentMicroservice):
    """分析器服务"""

    def __init__(self):
        super().__init__("analyzer", 8001)

    async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
        """执行数据分析任务"""
        # 模拟分析逻辑
        await asyncio.sleep(0.1)  # 模拟处理时间

        return {
            "analysis_type": task.get("type", "general"),
            "insights": ["insight1", "insight2", "insight3"],
            "confidence": 0.95,
            "processing_time": 0.1
        }

class ExecutorService(AgentMicroservice):
    """执行器服务"""

    def __init__(self):
        super().__init__("executor", 8002)

    async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
        """执行具体任务"""
        # 模拟执行逻辑
        await asyncio.sleep(0.2)

        return {
            "execution_status": "completed",
            "actions_taken": task.get("actions", []),
            "results": {"status": "success", "output": "Task completed successfully"}
        }

2. 核心组件架构

2.1 智能体管理器

2.1.1 智能体生命周期管理

import asyncio
import uuid
from enum import Enum
from typing import Dict, List, Optional, Callable
from datetime import datetime

class AgentStatus(Enum):
    INITIALIZING = "initializing"
    READY = "ready"
    BUSY = "busy"
    ERROR = "error"
    SHUTDOWN = "shutdown"

class AgentInfo:
    def __init__(self, agent_id: str, agent_type: str, capabilities: List[str]):
        self.agent_id = agent_id
        self.agent_type = agent_type
        self.capabilities = capabilities
        self.status = AgentStatus.INITIALIZING
        self.created_at = datetime.utcnow()
        self.last_activity = datetime.utcnow()
        self.task_count = 0
        self.error_count = 0

class AgentManager:
    """智能体管理器"""

    def __init__(self):
        self.agents: Dict[str, AgentInfo] = {}
        self.agent_instances: Dict[str, Any] = {}
        self.load_balancer = LoadBalancer()
        self.health_monitor = HealthMonitor()

    async def register_agent(self, agent_type: str, capabilities: List[str], 
                           agent_factory: Callable) -> str:
        """注册智能体"""
        agent_id = str(uuid.uuid4())

        # 创建智能体信息
        agent_info = AgentInfo(agent_id, agent_type, capabilities)
        self.agents[agent_id] = agent_info

        try:
            # 创建智能体实例
            agent_instance = await agent_factory()
            self.agent_instances[agent_id] = agent_instance

            # 更新状态
            agent_info.status = AgentStatus.READY

            # 启动健康监控
            asyncio.create_task(self.health_monitor.monitor_agent(agent_id))

            return agent_id

        except Exception as e:
            agent_info.status = AgentStatus.ERROR
            raise Exception(f"智能体注册失败: {e}")

    async def assign_task(self, task: Dict[str, Any], 
                         required_capabilities: List[str]) -> Optional[str]:
        """分配任务给合适的智能体"""
        # 查找具备所需能力的智能体
        suitable_agents = []
        for agent_id, agent_info in self.agents.items():
            if (agent_info.status == AgentStatus.READY and 
                all(cap in agent_info.capabilities for cap in required_capabilities)):
                suitable_agents.append(agent_id)

        if not suitable_agents:
            return None

        # 使用负载均衡器选择最优智能体
        selected_agent_id = self.load_balancer.select_agent(suitable_agents)

        # 更新智能体状态
        agent_info = self.agents[selected_agent_id]
        agent_info.status = AgentStatus.BUSY
        agent_info.task_count += 1
        agent_info.last_activity = datetime.utcnow()

        return selected_agent_id

    async def complete_task(self, agent_id: str, success: bool = True):
        """任务完成回调"""
        if agent_id in self.agents:
            agent_info = self.agents[agent_id]
            agent_info.status = AgentStatus.READY
            agent_info.last_activity = datetime.utcnow()

            if not success:
                agent_info.error_count += 1

    def get_agent_statistics(self) -> Dict[str, Any]:
        """获取智能体统计信息"""
        total_agents = len(self.agents)
        status_counts = {}

        for agent_info in self.agents.values():
            status = agent_info.status.value
            status_counts[status] = status_counts.get(status, 0) + 1

        return {
            "total_agents": total_agents,
            "status_distribution": status_counts,
            "total_tasks_processed": sum(agent.task_count for agent in self.agents.values()),
            "total_errors": sum(agent.error_count for agent in self.agents.values())
        }

class LoadBalancer:
    """负载均衡器"""

    def __init__(self):
        self.agent_loads: Dict[str, float] = {}

    def select_agent(self, agent_ids: List[str]) -> str:
        """选择负载最低的智能体"""
        if not agent_ids:
            raise ValueError("没有可用的智能体")

        # 简单的轮询策略
        min_load = float('inf')
        selected_agent = agent_ids[0]

        for agent_id in agent_ids:
            load = self.agent_loads.get(agent_id, 0)
            if load < min_load:
                min_load = load
                selected_agent = agent_id

        # 增加负载
        self.agent_loads[selected_agent] = self.agent_loads.get(selected_agent, 0) + 1

        return selected_agent

    def update_agent_load(self, agent_id: str, load_delta: float):
        """更新智能体负载"""
        current_load = self.agent_loads.get(agent_id, 0)
        self.agent_loads[agent_id] = max(0, current_load + load_delta)

class HealthMonitor:
    """健康监控器"""

    async def monitor_agent(self, agent_id: str):
        """监控智能体健康状态"""
        while True:
            try:
                # 执行健康检查
                is_healthy = await self._check_agent_health(agent_id)

                if not is_healthy:
                    await self._handle_unhealthy_agent(agent_id)

                await asyncio.sleep(30)  # 30秒检查一次

            except Exception as e:
                print(f"健康监控错误 {agent_id}: {e}")
                await asyncio.sleep(60)  # 错误时延长检查间隔

    async def _check_agent_health(self, agent_id: str) -> bool:
        """检查智能体健康状态"""
        # 实现具体的健康检查逻辑
        # 例如:检查响应时间、内存使用、错误率等
        return True  # 简化实现

    async def _handle_unhealthy_agent(self, agent_id: str):
        """处理不健康的智能体"""
        print(f"检测到不健康的智能体: {agent_id}")
        # 实现恢复策略:重启、替换等

2.2 通信总线

2.2.1 企业级消息总线

import asyncio
import json
from typing import Dict, List, Callable, Any
from dataclasses import dataclass, asdict
from datetime import datetime
import aioredis

@dataclass
class Message:
    id: str
    sender: str
    receiver: str
    message_type: str
    payload: Dict[str, Any]
    timestamp: datetime
    priority: int = 1
    ttl: int = 3600  # 消息存活时间(秒)

class MessageBus:
    """企业级消息总线"""

    def __init__(self, redis_url: str = "redis://localhost:6379"):
        self.redis_url = redis_url
        self.redis = None
        self.subscribers: Dict[str, List[Callable]] = {}
        self.message_handlers: Dict[str, Callable] = {}
        self.is_running = False

    async def initialize(self):
        """初始化消息总线"""
        self.redis = await aioredis.from_url(self.redis_url)
        self.is_running = True

        # 启动消息处理循环
        asyncio.create_task(self._message_processing_loop())

    async def publish(self, message: Message) -> bool:
        """发布消息"""
        try:
            # 序列化消息
            message_data = {
                **asdict(message),
                'timestamp': message.timestamp.isoformat()
            }

            # 发布到Redis
            await self.redis.lpush(
                f"queue:{message.receiver}",
                json.dumps(message_data)
            )

            # 设置TTL
            await self.redis.expire(f"queue:{message.receiver}", message.ttl)

            # 发布通知
            await self.redis.publish(
                f"notify:{message.receiver}",
                message.id
            )

            return True

        except Exception as e:
            print(f"消息发布失败: {e}")
            return False

    async def subscribe(self, agent_id: str, handler: Callable[[Message], None]):
        """订阅消息"""
        if agent_id not in self.subscribers:
            self.subscribers[agent_id] = []

        self.subscribers[agent_id].append(handler)

        # 启动订阅监听
        asyncio.create_task(self._subscribe_to_notifications(agent_id))

    async def _subscribe_to_notifications(self, agent_id: str):
        """监听消息通知"""
        pubsub = self.redis.pubsub()
        await pubsub.subscribe(f"notify:{agent_id}")

        async for message in pubsub.listen():
            if message['type'] == 'message':
                # 处理新消息通知
                await self._process_agent_messages(agent_id)

    async def _process_agent_messages(self, agent_id: str):
        """处理智能体消息"""
        queue_key = f"queue:{agent_id}"

        while True:
            # 从队列中获取消息
            message_data = await self.redis.rpop(queue_key)

            if not message_data:
                break

            try:
                # 反序列化消息
                message_dict = json.loads(message_data)
                message_dict['timestamp'] = datetime.fromisoformat(message_dict['timestamp'])
                message = Message(**message_dict)

                # 调用处理器
                if agent_id in self.subscribers:
                    for handler in self.subscribers[agent_id]:
                        try:
                            await handler(message)
                        except Exception as e:
                            print(f"消息处理错误: {e}")

            except Exception as e:
                print(f"消息反序列化错误: {e}")

    async def _message_processing_loop(self):
        """消息处理主循环"""
        while self.is_running:
            try:
                # 清理过期消息
                await self._cleanup_expired_messages()

                # 处理优先级消息
                await self._process_priority_messages()

                await asyncio.sleep(1)

            except Exception as e:
                print(f"消息处理循环错误: {e}")
                await asyncio.sleep(5)

    async def _cleanup_expired_messages(self):
        """清理过期消息"""
        # 获取所有队列
        queue_keys = await self.redis.keys("queue:*")

        for queue_key in queue_keys:
            # 检查队列TTL
            ttl = await self.redis.ttl(queue_key)
            if ttl == -1:  # 没有设置TTL
                await self.redis.expire(queue_key, 3600)  # 设置默认TTL

    async def _process_priority_messages(self):
        """处理优先级消息"""
        # 实现优先级消息处理逻辑
        pass

    async def get_queue_stats(self, agent_id: str) -> Dict[str, Any]:
        """获取队列统计信息"""
        queue_key = f"queue:{agent_id}"

        queue_length = await self.redis.llen(queue_key)
        ttl = await self.redis.ttl(queue_key)

        return {
            "agent_id": agent_id,
            "queue_length": queue_length,
            "ttl": ttl
        }

2.3 工作流引擎

2.3.1 企业级工作流引擎

from enum import Enum
from typing import Dict, List, Any, Optional, Callable
import asyncio
import uuid
from datetime import datetime

class WorkflowStatus(Enum):
    PENDING = "pending"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"
    CANCELLED = "cancelled"

class TaskStatus(Enum):
    WAITING = "waiting"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"
    SKIPPED = "skipped"

@dataclass
class WorkflowTask:
    task_id: str
    name: str
    agent_type: str
    required_capabilities: List[str]
    inputs: Dict[str, Any]
    outputs: Dict[str, Any] = None
    status: TaskStatus = TaskStatus.WAITING
    dependencies: List[str] = None
    retry_count: int = 0
    max_retries: int = 3
    timeout: int = 300  # 5分钟超时
    created_at: datetime = None
    started_at: datetime = None
    completed_at: datetime = None
    error_message: str = None

@dataclass
class Workflow:
    workflow_id: str
    name: str
    description: str
    tasks: List[WorkflowTask]
    status: WorkflowStatus = WorkflowStatus.PENDING
    created_at: datetime = None
    started_at: datetime = None
    completed_at: datetime = None
    context: Dict[str, Any] = None

class WorkflowEngine:
    """企业级工作流引擎"""

    def __init__(self, agent_manager: AgentManager, message_bus: MessageBus):
        self.agent_manager = agent_manager
        self.message_bus = message_bus
        self.workflows: Dict[str, Workflow] = {}
        self.running_tasks: Dict[str, asyncio.Task] = {}
        self.task_callbacks: Dict[str, Callable] = {}

    async def create_workflow(self, name: str, description: str, 
                            tasks: List[Dict[str, Any]]) -> str:
        """创建工作流"""
        workflow_id = str(uuid.uuid4())

        # 创建任务对象
        workflow_tasks = []
        for task_data in tasks:
            task = WorkflowTask(
                task_id=str(uuid.uuid4()),
                name=task_data["name"],
                agent_type=task_data["agent_type"],
                required_capabilities=task_data.get("required_capabilities", []),
                inputs=task_data.get("inputs", {}),
                dependencies=task_data.get("dependencies", []),
                max_retries=task_data.get("max_retries", 3),
                timeout=task_data.get("timeout", 300),
                created_at=datetime.utcnow()
            )
            workflow_tasks.append(task)

        # 创建工作流
        workflow = Workflow(
            workflow_id=workflow_id,
            name=name,
            description=description,
            tasks=workflow_tasks,
            created_at=datetime.utcnow(),
            context={}
        )

        self.workflows[workflow_id] = workflow
        return workflow_id

    async def start_workflow(self, workflow_id: str) -> bool:
        """启动工作流"""
        if workflow_id not in self.workflows:
            return False

        workflow = self.workflows[workflow_id]
        workflow.status = WorkflowStatus.RUNNING
        workflow.started_at = datetime.utcnow()

        # 启动工作流执行
        asyncio.create_task(self._execute_workflow(workflow))

        return True

    async def _execute_workflow(self, workflow: Workflow):
        """执行工作流"""
        try:
            while True:
                # 查找可执行的任务
                ready_tasks = self._get_ready_tasks(workflow)

                if not ready_tasks:
                    # 检查是否所有任务都完成
                    if self._is_workflow_completed(workflow):
                        workflow.status = WorkflowStatus.COMPLETED
                        workflow.completed_at = datetime.utcnow()
                        break
                    else:
                        # 等待正在执行的任务
                        await asyncio.sleep(1)
                        continue

                # 并行执行准备好的任务
                for task in ready_tasks:
                    asyncio.create_task(self._execute_task(workflow, task))

                await asyncio.sleep(0.1)  # 短暂等待

        except Exception as e:
            workflow.status = WorkflowStatus.FAILED
            print(f"工作流执行失败 {workflow.workflow_id}: {e}")

    def _get_ready_tasks(self, workflow: Workflow) -> List[WorkflowTask]:
        """获取准备执行的任务"""
        ready_tasks = []

        for task in workflow.tasks:
            if task.status != TaskStatus.WAITING:
                continue

            # 检查依赖是否满足
            if self._are_dependencies_satisfied(workflow, task):
                ready_tasks.append(task)

        return ready_tasks

    def _are_dependencies_satisfied(self, workflow: Workflow, task: WorkflowTask) -> bool:
        """检查任务依赖是否满足"""
        if not task.dependencies:
            return True

        task_dict = {t.task_id: t for t in workflow.tasks}

        for dep_id in task.dependencies:
            if dep_id not in task_dict:
                return False

            dep_task = task_dict[dep_id]
            if dep_task.status != TaskStatus.COMPLETED:
                return False

        return True

    async def _execute_task(self, workflow: Workflow, task: WorkflowTask):
        """执行单个任务"""
        task.status = TaskStatus.RUNNING
        task.started_at = datetime.utcnow()

        try:
            # 分配智能体
            agent_id = await self.agent_manager.assign_task(
                {"task": task.inputs, "task_id": task.task_id},
                task.required_capabilities
            )

            if not agent_id:
                raise Exception("没有可用的智能体")

            # 执行任务(带超时)
            result = await asyncio.wait_for(
                self._run_task_on_agent(agent_id, task),
                timeout=task.timeout
            )

            # 任务成功完成
            task.outputs = result
            task.status = TaskStatus.COMPLETED
            task.completed_at = datetime.utcnow()

            # 更新工作流上下文
            workflow.context[task.task_id] = result

            # 通知智能体管理器任务完成
            await self.agent_manager.complete_task(agent_id, success=True)

        except asyncio.TimeoutError:
            task.error_message = "任务执行超时"
            await self._handle_task_failure(workflow, task)

        except Exception as e:
            task.error_message = str(e)
            await self._handle_task_failure(workflow, task)

    async def _run_task_on_agent(self, agent_id: str, task: WorkflowTask) -> Dict[str, Any]:
        """在智能体上运行任务"""
        # 获取智能体实例
        agent_instance = self.agent_manager.agent_instances.get(agent_id)

        if not agent_instance:
            raise Exception(f"智能体实例不存在: {agent_id}")

        # 执行任务
        if hasattr(agent_instance, 'process_task'):
            return await agent_instance.process_task(task.inputs)
        else:
            raise Exception("智能体不支持任务处理")

    async def _handle_task_failure(self, workflow: Workflow, task: WorkflowTask):
        """处理任务失败"""
        task.retry_count += 1

        if task.retry_count <= task.max_retries:
            # 重试任务
            task.status = TaskStatus.WAITING
            print(f"任务重试 {task.task_id} (第{task.retry_count}次)")
        else:
            # 任务最终失败
            task.status = TaskStatus.FAILED
            workflow.status = WorkflowStatus.FAILED
            print(f"任务最终失败 {task.task_id}: {task.error_message}")

    def _is_workflow_completed(self, workflow: Workflow) -> bool:
        """检查工作流是否完成"""
        for task in workflow.tasks:
            if task.status in [TaskStatus.WAITING, TaskStatus.RUNNING]:
                return False
        return True

    def get_workflow_status(self, workflow_id: str) -> Optional[Dict[str, Any]]:
        """获取工作流状态"""
        if workflow_id not in self.workflows:
            return None

        workflow = self.workflows[workflow_id]

        task_stats = {}
        for status in TaskStatus:
            task_stats[status.value] = sum(
                1 for task in workflow.tasks if task.status == status
            )

        return {
            "workflow_id": workflow_id,
            "name": workflow.name,
            "status": workflow.status.value,
            "created_at": workflow.created_at.isoformat() if workflow.created_at else None,
            "started_at": workflow.started_at.isoformat() if workflow.started_at else None,
            "completed_at": workflow.completed_at.isoformat() if workflow.completed_at else None,
            "total_tasks": len(workflow.tasks),
            "task_statistics": task_stats
        }

3. 安全架构设计

3.1 认证与授权

3.1.1 JWT认证系统

import jwt
import bcrypt
from datetime import datetime, timedelta
from typing import Dict, Optional, List
from fastapi import HTTPException, Depends
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials

class AuthenticationService:
    """认证服务"""

    def __init__(self, secret_key: str, algorithm: str = "HS256"):
        self.secret_key = secret_key
        self.algorithm = algorithm
        self.token_expiry = timedelta(hours=24)
        self.refresh_token_expiry = timedelta(days=7)

    def hash_password(self, password: str) -> str:
        """密码哈希"""
        salt = bcrypt.gensalt()
        return bcrypt.hashpw(password.encode('utf-8'), salt).decode('utf-8')

    def verify_password(self, password: str, hashed_password: str) -> bool:
        """验证密码"""
        return bcrypt.checkpw(password.encode('utf-8'), hashed_password.encode('utf-8'))

    def create_access_token(self, user_id: str, roles: List[str]) -> str:
        """创建访问令牌"""
        payload = {
            "user_id": user_id,
            "roles": roles,
            "exp": datetime.utcnow() + self.token_expiry,
            "iat": datetime.utcnow(),
            "type": "access"
        }
        return jwt.encode(payload, self.secret_key, algorithm=self.algorithm)

    def create_refresh_token(self, user_id: str) -> str:
        """创建刷新令牌"""
        payload = {
            "user_id": user_id,
            "exp": datetime.utcnow() + self.refresh_token_expiry,
            "iat": datetime.utcnow(),
            "type": "refresh"
        }
        return jwt.encode(payload, self.secret_key, algorithm=self.algorithm)

    def verify_token(self, token: str) -> Dict[str, any]:
        """验证令牌"""
        try:
            payload = jwt.decode(token, self.secret_key, algorithms=[self.algorithm])
            return payload
        except jwt.ExpiredSignatureError:
            raise HTTPException(status_code=401, detail="令牌已过期")
        except jwt.InvalidTokenError:
            raise HTTPException(status_code=401, detail="无效令牌")

class AuthorizationService:
    """授权服务"""

    def __init__(self):
        self.permissions = {
            "admin": ["*"],  # 管理员拥有所有权限
            "agent_manager": ["agent:create", "agent:read", "agent:update", "agent:delete"],
            "workflow_manager": ["workflow:create", "workflow:read", "workflow:update", "workflow:delete"],
            "monitor": ["system:read", "metrics:read"],
            "user": ["agent:read", "workflow:read"]
        }

    def check_permission(self, user_roles: List[str], required_permission: str) -> bool:
        """检查权限"""
        for role in user_roles:
            role_permissions = self.permissions.get(role, [])

            # 检查是否有通配符权限
            if "*" in role_permissions:
                return True

            # 检查具体权限
            if required_permission in role_permissions:
                return True

            # 检查模式匹配权限
            for permission in role_permissions:
                if self._match_permission(permission, required_permission):
                    return True

        return False

    def _match_permission(self, pattern: str, permission: str) -> bool:
        """权限模式匹配"""
        if pattern.endswith("*"):
            prefix = pattern[:-1]
            return permission.startswith(prefix)
        return pattern == permission

# FastAPI依赖注入
security = HTTPBearer()
auth_service = AuthenticationService("your-secret-key")
authz_service = AuthorizationService()

async def get_current_user(credentials: HTTPAuthorizationCredentials = Depends(security)):
    """获取当前用户"""
    token = credentials.credentials
    payload = auth_service.verify_token(token)
    return payload

def require_permission(permission: str):
    """权限装饰器"""
    def decorator(func):
        async def wrapper(*args, **kwargs):
            # 从依赖注入中获取用户信息
            user = kwargs.get('current_user')
            if not user:
                raise HTTPException(status_code=401, detail="未认证")

            user_roles = user.get('roles', [])
            if not authz_service.check_permission(user_roles, permission):
                raise HTTPException(status_code=403, detail="权限不足")

            return await func(*args, **kwargs)
        return wrapper
    return decorator

3.2 数据加密与安全

3.2.1 数据加密服务

from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
import base64
import os
from typing import Union

class EncryptionService:
    """数据加密服务"""

    def __init__(self, password: str = None):
        if password:
            self.key = self._derive_key(password)
        else:
            self.key = Fernet.generate_key()

        self.cipher = Fernet(self.key)

    def _derive_key(self, password: str) -> bytes:
        """从密码派生密钥"""
        password_bytes = password.encode()
        salt = b'salt_'  # 在生产环境中应使用随机盐
        kdf = PBKDF2HMAC(
            algorithm=hashes.SHA256(),
            length=32,
            salt=salt,
            iterations=100000,
        )
        key = base64.urlsafe_b64encode(kdf.derive(password_bytes))
        return key

    def encrypt(self, data: Union[str, bytes]) -> str:
        """加密数据"""
        if isinstance(data, str):
            data = data.encode()

        encrypted_data = self.cipher.encrypt(data)
        return base64.urlsafe_b64encode(encrypted_data).decode()

    def decrypt(self, encrypted_data: str) -> str:
        """解密数据"""
        encrypted_bytes = base64.urlsafe_b64decode(encrypted_data.encode())
        decrypted_data = self.cipher.decrypt(encrypted_bytes)
        return decrypted_data.decode()

    def encrypt_dict(self, data: Dict[str, Any]) -> Dict[str, str]:
        """加密字典数据"""
        encrypted_dict = {}
        for key, value in data.items():
            encrypted_dict[key] = self.encrypt(str(value))
        return encrypted_dict

    def decrypt_dict(self, encrypted_dict: Dict[str, str]) -> Dict[str, str]:
        """解密字典数据"""
        decrypted_dict = {}
        for key, encrypted_value in encrypted_dict.items():
            decrypted_dict[key] = self.decrypt(encrypted_value)
        return decrypted_dict

class SecureDataStore:
    """安全数据存储"""

    def __init__(self, encryption_service: EncryptionService):
        self.encryption = encryption_service
        self.data_store = {}  # 在实际应用中应使用数据库

    async def store_sensitive_data(self, key: str, data: Dict[str, Any]) -> bool:
        """存储敏感数据"""
        try:
            # 加密数据
            encrypted_data = self.encryption.encrypt_dict(data)

            # 存储加密后的数据
            self.data_store[key] = {
                "data": encrypted_data,
                "created_at": datetime.utcnow().isoformat(),
                "encrypted": True
            }

            return True
        except Exception as e:
            print(f"数据存储失败: {e}")
            return False

    async def retrieve_sensitive_data(self, key: str) -> Optional[Dict[str, Any]]:
        """检索敏感数据"""
        try:
            if key not in self.data_store:
                return None

            stored_data = self.data_store[key]

            if stored_data.get("encrypted", False):
                # 解密数据
                decrypted_data = self.encryption.decrypt_dict(stored_data["data"])
                return decrypted_data
            else:
                return stored_data["data"]

        except Exception as e:
            print(f"数据检索失败: {e}")
            return None

4. 系统实现

4.1 完整系统集成

4.1.1 企业级多智能体系统

from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel
import asyncio
from typing import Dict, List, Any, Optional

class EnterpriseMultiAgentSystem:
    """企业级多智能体系统"""

    def __init__(self):
        self.app = FastAPI(title="Enterprise Multi-Agent System")
        self.agent_manager = AgentManager()
        self.message_bus = MessageBus()
        self.workflow_engine = WorkflowEngine(self.agent_manager, self.message_bus)
        self.auth_service = AuthenticationService("your-secret-key")
        self.authz_service = AuthorizationService()
        self.encryption_service = EncryptionService()
        self.secure_store = SecureDataStore(self.encryption_service)

        self._setup_routes()

    def _setup_routes(self):
        """设置API路由"""

        @self.app.post("/auth/login")
        async def login(credentials: LoginRequest):
            # 验证用户凭据(简化实现)
            if credentials.username == "admin" and credentials.password == "password":
                access_token = self.auth_service.create_access_token(
                    user_id="admin",
                    roles=["admin"]
                )
                refresh_token = self.auth_service.create_refresh_token("admin")

                return {
                    "access_token": access_token,
                    "refresh_token": refresh_token,
                    "token_type": "bearer"
                }
            else:
                raise HTTPException(status_code=401, detail="无效凭据")

        @self.app.post("/agents/register")
        async def register_agent(
            request: AgentRegistrationRequest,
            current_user: Dict = Depends(get_current_user)
        ):
            # 检查权限
            if not self.authz_service.check_permission(
                current_user.get("roles", []), 
                "agent:create"
            ):
                raise HTTPException(status_code=403, detail="权限不足")

            # 注册智能体
            agent_id = await self.agent_manager.register_agent(
                request.agent_type,
                request.capabilities,
                request.agent_factory
            )

            return {"agent_id": agent_id, "status": "registered"}

        @self.app.post("/workflows/create")
        async def create_workflow(
            request: WorkflowCreateRequest,
            current_user: Dict = Depends(get_current_user)
        ):
            # 检查权限
            if not self.authz_service.check_permission(
                current_user.get("roles", []), 
                "workflow:create"
            ):
                raise HTTPException(status_code=403, detail="权限不足")

            # 创建工作流
            workflow_id = await self.workflow_engine.create_workflow(
                request.name,
                request.description,
                request.tasks
            )

            return {"workflow_id": workflow_id, "status": "created"}

        @self.app.post("/workflows/{workflow_id}/start")
        async def start_workflow(
            workflow_id: str,
            current_user: Dict = Depends(get_current_user)
        ):
            # 检查权限
            if not self.authz_service.check_permission(
                current_user.get("roles", []), 
                "workflow:update"
            ):
                raise HTTPException(status_code=403, detail="权限不足")

            # 启动工作流
            success = await self.workflow_engine.start_workflow(workflow_id)

            if success:
                return {"status": "started"}
            else:
                raise HTTPException(status_code=404, detail="工作流不存在")

        @self.app.get("/system/status")
        async def get_system_status(
            current_user: Dict = Depends(get_current_user)
        ):
            # 检查权限
            if not self.authz_service.check_permission(
                current_user.get("roles", []), 
                "system:read"
            ):
                raise HTTPException(status_code=403, detail="权限不足")

            # 获取系统状态
            agent_stats = self.agent_manager.get_agent_statistics()

            return {
                "system_status": "healthy",
                "agent_statistics": agent_stats,
                "timestamp": datetime.utcnow().isoformat()
            }

    async def initialize(self):
        """初始化系统"""
        await self.message_bus.initialize()
        print("企业级多智能体系统初始化完成")

    async def shutdown(self):
        """关闭系统"""
        self.message_bus.is_running = False
        print("企业级多智能体系统已关闭")

# 请求模型
class LoginRequest(BaseModel):
    username: str
    password: str

class AgentRegistrationRequest(BaseModel):
    agent_type: str
    capabilities: List[str]
    agent_factory: Any  # 在实际应用中需要更具体的类型

class WorkflowCreateRequest(BaseModel):
    name: str
    description: str
    tasks: List[Dict[str, Any]]

5. 部署与运维

5.1 Docker容器化部署

5.1.1 Dockerfile

FROM python:3.9-slim

WORKDIR /app

# 安装系统依赖
RUN apt-get update && apt-get install -y \
    gcc \
    && rm -rf /var/lib/apt/lists/*

# 复制依赖文件
COPY requirements.txt .

# 安装Python依赖
RUN pip install --no-cache-dir -r requirements.txt

# 复制应用代码
COPY . .

# 暴露端口
EXPOSE 8000

# 启动命令
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

5.1.2 Docker Compose配置

version: '3.8'

services:
  multi-agent-system:
    build: .
    ports:
      - "8000:8000"
    environment:
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://user:password@postgres:5432/multiagent
    depends_on:
      - redis
      - postgres
    volumes:
      - ./logs:/app/logs
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=multiagent
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: unless-stopped

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - multi-agent-system
    restart: unless-stopped

volumes:
  redis_data:
  postgres_data:

5.2 监控与日志

5.2.1 系统监控配置

import logging
import sys
from datetime import datetime
from typing import Dict, Any
import psutil
import asyncio

class SystemMonitor:
    """系统监控"""

    def __init__(self):
        self.setup_logging()
        self.metrics = {}
        self.is_monitoring = False

    def setup_logging(self):
        """设置日志"""
        logging.basicConfig(
            level=logging.INFO,
            format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
            handlers=[
                logging.FileHandler('/app/logs/system.log'),
                logging.StreamHandler(sys.stdout)
            ]
        )
        self.logger = logging.getLogger(__name__)

    async def start_monitoring(self, interval: int = 60):
        """开始监控"""
        self.is_monitoring = True
        self.logger.info("系统监控已启动")

        while self.is_monitoring:
            try:
                await self.collect_system_metrics()
                await self.check_system_health()
                await asyncio.sleep(interval)
            except Exception as e:
                self.logger.error(f"监控错误: {e}")
                await asyncio.sleep(interval)

    async def collect_system_metrics(self):
        """收集系统指标"""
        # CPU使用率
        cpu_percent = psutil.cpu_percent(interval=1)

        # 内存使用情况
        memory = psutil.virtual_memory()

        # 磁盘使用情况
        disk = psutil.disk_usage('/')

        # 网络统计
        network = psutil.net_io_counters()

        self.metrics = {
            "timestamp": datetime.utcnow().isoformat(),
            "cpu_percent": cpu_percent,
            "memory": {
                "total": memory.total,
                "available": memory.available,
                "percent": memory.percent,
                "used": memory.used
            },
            "disk": {
                "total": disk.total,
                "used": disk.used,
                "free": disk.free,
                "percent": (disk.used / disk.total) * 100
            },
            "network": {
                "bytes_sent": network.bytes_sent,
                "bytes_recv": network.bytes_recv,
                "packets_sent": network.packets_sent,
                "packets_recv": network.packets_recv
            }
        }

        self.logger.info(f"系统指标: CPU {cpu_percent}%, 内存 {memory.percent}%, 磁盘 {self.metrics['disk']['percent']:.1f}%")

    async def check_system_health(self):
        """检查系统健康状态"""
        alerts = []

        # CPU告警
        if self.metrics["cpu_percent"] > 80:
            alerts.append(f"CPU使用率过高: {self.metrics['cpu_percent']}%")

        # 内存告警
        if self.metrics["memory"]["percent"] > 85:
            alerts.append(f"内存使用率过高: {self.metrics['memory']['percent']}%")

        # 磁盘告警
        if self.metrics["disk"]["percent"] > 90:
            alerts.append(f"磁盘使用率过高: {self.metrics['disk']['percent']:.1f}%")

        # 发送告警
        for alert in alerts:
            self.logger.warning(f"系统告警: {alert}")
            await self.send_alert(alert)

    async def send_alert(self, message: str):
        """发送告警"""
        # 这里可以集成邮件、Slack、钉钉等告警渠道
        print(f"[ALERT] {message}")

    def get_current_metrics(self) -> Dict[str, Any]:
        """获取当前指标"""
        return self.metrics

6. 实践项目

项目:构建企业级客服智能体系统

6.1.1 项目需求

  1. 多智能体协作:意图识别、知识检索、回答生成
  2. 工作流管理:客服流程自动化
  3. 监控告警:系统性能和服务质量监控
  4. 安全认证:用户认证和权限管理

6.1.2 实现框架

# 学员需要完成的项目代码框架
class CustomerServiceSystem(EnterpriseMultiAgentSystem):
    def __init__(self):
        super().__init__()
        self.setup_customer_service_agents()
        self.setup_customer_service_workflows()

    def setup_customer_service_agents(self):
        """设置客服智能体"""
        # TODO: 实现意图识别智能体
        # TODO: 实现知识检索智能体
        # TODO: 实现回答生成智能体
        pass

    def setup_customer_service_workflows(self):
        """设置客服工作流"""
        # TODO: 定义客服处理流程
        # TODO: 配置异常处理机制
        pass

    async def handle_customer_query(self, query: str, customer_id: str):
        """处理客户查询"""
        # TODO: 创建并执行客服工作流
        pass

总结

本课程涵盖了企业级多智能体系统的完整架构设计和实现,包括:

  1. 架构设计:分层架构、微服务模式、设计原则
  2. 核心组件:智能体管理、消息总线、工作流引擎
  3. 安全机制:认证授权、数据加密、安全存储
  4. 系统实现:完整的企业级系统集成
  5. 部署运维:容器化部署、监控告警、日志管理

下一课程将学习应用实践与部署运维的高级技术。