第四天:企业级系统架构设计与实现¶
学习目标¶
- 掌握企业级多智能体系统架构设计原则
- 学会设计和实现核心系统组件
- 掌握安全、监控和运维最佳实践
- 学会系统性能优化和故障处理
- 了解实际项目中的架构实现和最佳实践
参考项目¶
本课程深入讲解企业级多智能体系统的架构设计原则、核心组件实现和最佳实践,帮助学员构建可扩展、高可用的生产级系统。
💡 实际代码参考:完整的企业级架构实现可参考项目中的以下文件:
multi_agent_system/main.py- 多智能体系统主入口和整体架构multi_agent_system/src/agents/base_agent.py- BDI智能体基础架构multi_agent_system/src/communication/message_bus.py- 企业级消息总线multi_agent_system/src/workflows/langgraph_workflow.py- 工作流引擎架构multi_agent_system/src/monitoring/langsmith_integration.py- 监控系统集成src/examples/customer_service_system.py- 企业级客服系统实现multi_agent_system/src/agents/- 安全组件实现multi_agent_system/src/communication/- 基础设施组件
代码引用: 完整的企业级架构实现请参考 multi_agent_system/main.py
企业级架构的核心特性:
核心组件:
ArchitectureLayer: 架构层次管理(表示层、业务层、服务层、数据层、基础设施层)EnterpriseComponent: 企业级组件定义(依赖关系、可扩展性、可用性、安全级别)EnterpriseArchitectureManager: 架构管理器,支持组件注册和系统启动MultiAgentSystem: 多智能体系统主类,实现企业级架构
企业级特性:
- 分层架构设计,清晰的职责分离
- 组件化管理,支持动态注册和监控
- 异步启动机制,按依赖顺序启动组件
- 集成监控和安全策略
💡 实际代码参考:完整的企业级架构实现可参考项目中的以下文件:
main.py- 多智能体系统主入口和整体架构src/examples/customer_service_system.py- 企业级客服系统实现message_bus.py- 消息总线和通信中间件langgraph_workflow.py- 工作流引擎实现langsmith_integration.py- 监控系统集成
架构特色:本项目采用现代微服务架构,集成了BDI智能体模型、LangGraph工作流引擎、LangSmith监控平台,提供了完整的企业级多智能体解决方案。
1. 企业级架构设计原则¶
1.1 设计原则概述¶
1.1.1 核心设计原则¶
**代码引用**: 完整的企业级架构设计原则实现请参考 `multi_agent_system/main.py` 中的 `EnterpriseArchitecturalPrinciples` 和 `MultiAgentSystem` 类
企业级架构设计原则的核心特性:
**设计原则**:
- `EnterpriseArchitecturalPrinciples`: 定义了8大核心原则(可扩展性、可靠性、安全性、可维护性、性能、可观测性、灵活性、成本效率)
- `MultiAgentSystem`: 主系统类,实现企业级架构的完整生命周期管理
**系统初始化流程**:
1. 基础智能体架构初始化(BDI模型)
2. 专业化智能体初始化(业务特定智能体)
3. 通信中间件初始化(消息总线)
4. 工作流引擎初始化(LangGraph)
5. 监控系统初始化(LangSmith)
6. 示例应用初始化(客服系统)
**企业级特性**:
- 模块化设计,每个组件独立可替换
- 异步架构,支持高并发处理
- 全链路监控,集成LangSmith
- 容错设计,自动故障恢复
1.2 实际项目架构分析¶
1.2.1 项目架构层次¶
# 基于实际项目的架构层次分析
class ProjectArchitectureLayers:
"""项目架构层次"""
def __init__(self):
self.presentation_layer = {
"description": "表示层 - 用户接口和API",
"components": [
"main.py - 系统主入口",
"API接口 - RESTful API设计",
"管理界面 - 系统监控面板"
],
"technologies": ["FastAPI", "WebSocket", "HTTP/HTTPS"]
}
self.business_layer = {
"description": "业务层 - 智能体编排和业务逻辑",
"components": [
"customer_service_system.py - 客服业务逻辑",
"langgraph_workflow.py - 工作流编排",
"业务规则引擎 - 动态规则配置"
],
"technologies": ["LangGraph", "业务规则引擎", "工作流引擎"]
}
self.service_layer = {
"description": "服务层 - 微服务和中间件",
"components": [
"base_agent.py - BDI智能体服务",
"message_bus.py - 消息通信服务",
"langsmith_integration.py - 监控服务"
],
"technologies": ["异步消息队列", "服务发现", "负载均衡"]
}
self.data_layer = {
"description": "数据层 - 数据存储和管理",
"components": [
"智能体状态存储",
"消息历史存储",
"监控数据存储",
"配置数据存储"
],
"technologies": ["Redis", "PostgreSQL", "文件存储", "时序数据库"]
}
# 架构特性对比
class ArchitectureFeatures:
"""架构特性对比"""
@staticmethod
def get_feature_comparison():
return {
"传统单体架构": {
"优点": ["简单部署", "开发快速", "调试容易"],
"缺点": ["扩展困难", "技术栈固定", "故障影响全局"],
"适用场景": "小型项目、原型开发"
},
"微服务架构": {
"优点": ["独立扩展", "技术多样性", "故障隔离"],
"缺点": ["复杂度高", "网络开销", "数据一致性"],
"适用场景": "大型企业应用、高并发系统"
},
"智能体架构": {
"优点": ["自主决策", "动态协作", "智能适应"],
"缺点": ["行为不可预测", "调试困难", "性能开销"],
"适用场景": "复杂业务场景、智能化应用"
},
"本项目架构": {
"优点": ["结合三者优势", "企业级特性", "完整生态"],
"缺点": ["学习成本高", "初期复杂"],
"适用场景": "企业级AI应用、智能化业务系统"
}
}
1.2 分层架构模式¶
1.2.1 四层架构设计¶
class EnterpriseArchitecture:
"""企业级四层架构"""
def __init__(self):
self.presentation_layer = PresentationLayer()
self.business_layer = BusinessLayer()
self.service_layer = ServiceLayer()
self.data_layer = DataLayer()
class PresentationLayer:
"""表示层 - API网关、Web界面"""
def __init__(self):
self.api_gateway = "API网关"
self.web_interface = "Web管理界面"
self.mobile_app = "移动端应用"
class BusinessLayer:
"""业务层 - 智能体编排、业务逻辑"""
def __init__(self):
self.agent_orchestrator = "智能体编排器"
self.workflow_engine = "工作流引擎"
self.business_rules = "业务规则引擎"
class ServiceLayer:
"""服务层 - 微服务、中间件"""
def __init__(self):
self.agent_services = "智能体服务集群"
self.message_queue = "消息队列"
self.cache_service = "缓存服务"
self.auth_service = "认证授权服务"
class DataLayer:
"""数据层 - 数据库、存储"""
def __init__(self):
self.primary_database = "主数据库"
self.cache_database = "缓存数据库"
self.file_storage = "文件存储"
self.log_storage = "日志存储"
1.3 微服务架构模式¶
1.3.1 智能体微服务设计¶
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import asyncio
from typing import Dict, Any
class AgentMicroservice:
"""智能体微服务基类"""
def __init__(self, service_name: str, port: int):
self.service_name = service_name
self.port = port
self.app = FastAPI(title=f"{service_name} Service")
self.health_status = "healthy"
self._setup_routes()
def _setup_routes(self):
"""设置路由"""
@self.app.get("/health")
async def health_check():
return {"status": self.health_status, "service": self.service_name}
@self.app.post("/process")
async def process_request(request: ProcessRequest):
try:
result = await self.process_task(request.task, request.context)
return {"success": True, "result": result}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@abstractmethod
async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
"""处理任务的抽象方法"""
pass
class ProcessRequest(BaseModel):
task: Dict[str, Any]
context: Dict[str, Any] = {}
# 具体智能体服务实现
class AnalyzerService(AgentMicroservice):
"""分析器服务"""
def __init__(self):
super().__init__("analyzer", 8001)
async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
"""执行数据分析任务"""
# 模拟分析逻辑
await asyncio.sleep(0.1) # 模拟处理时间
return {
"analysis_type": task.get("type", "general"),
"insights": ["insight1", "insight2", "insight3"],
"confidence": 0.95,
"processing_time": 0.1
}
class ExecutorService(AgentMicroservice):
"""执行器服务"""
def __init__(self):
super().__init__("executor", 8002)
async def process_task(self, task: Dict[str, Any], context: Dict[str, Any]) -> Dict[str, Any]:
"""执行具体任务"""
# 模拟执行逻辑
await asyncio.sleep(0.2)
return {
"execution_status": "completed",
"actions_taken": task.get("actions", []),
"results": {"status": "success", "output": "Task completed successfully"}
}
2. 核心组件架构¶
2.1 智能体管理器¶
2.1.1 智能体生命周期管理¶
import asyncio
import uuid
from enum import Enum
from typing import Dict, List, Optional, Callable
from datetime import datetime
class AgentStatus(Enum):
INITIALIZING = "initializing"
READY = "ready"
BUSY = "busy"
ERROR = "error"
SHUTDOWN = "shutdown"
class AgentInfo:
def __init__(self, agent_id: str, agent_type: str, capabilities: List[str]):
self.agent_id = agent_id
self.agent_type = agent_type
self.capabilities = capabilities
self.status = AgentStatus.INITIALIZING
self.created_at = datetime.utcnow()
self.last_activity = datetime.utcnow()
self.task_count = 0
self.error_count = 0
class AgentManager:
"""智能体管理器"""
def __init__(self):
self.agents: Dict[str, AgentInfo] = {}
self.agent_instances: Dict[str, Any] = {}
self.load_balancer = LoadBalancer()
self.health_monitor = HealthMonitor()
async def register_agent(self, agent_type: str, capabilities: List[str],
agent_factory: Callable) -> str:
"""注册智能体"""
agent_id = str(uuid.uuid4())
# 创建智能体信息
agent_info = AgentInfo(agent_id, agent_type, capabilities)
self.agents[agent_id] = agent_info
try:
# 创建智能体实例
agent_instance = await agent_factory()
self.agent_instances[agent_id] = agent_instance
# 更新状态
agent_info.status = AgentStatus.READY
# 启动健康监控
asyncio.create_task(self.health_monitor.monitor_agent(agent_id))
return agent_id
except Exception as e:
agent_info.status = AgentStatus.ERROR
raise Exception(f"智能体注册失败: {e}")
async def assign_task(self, task: Dict[str, Any],
required_capabilities: List[str]) -> Optional[str]:
"""分配任务给合适的智能体"""
# 查找具备所需能力的智能体
suitable_agents = []
for agent_id, agent_info in self.agents.items():
if (agent_info.status == AgentStatus.READY and
all(cap in agent_info.capabilities for cap in required_capabilities)):
suitable_agents.append(agent_id)
if not suitable_agents:
return None
# 使用负载均衡器选择最优智能体
selected_agent_id = self.load_balancer.select_agent(suitable_agents)
# 更新智能体状态
agent_info = self.agents[selected_agent_id]
agent_info.status = AgentStatus.BUSY
agent_info.task_count += 1
agent_info.last_activity = datetime.utcnow()
return selected_agent_id
async def complete_task(self, agent_id: str, success: bool = True):
"""任务完成回调"""
if agent_id in self.agents:
agent_info = self.agents[agent_id]
agent_info.status = AgentStatus.READY
agent_info.last_activity = datetime.utcnow()
if not success:
agent_info.error_count += 1
def get_agent_statistics(self) -> Dict[str, Any]:
"""获取智能体统计信息"""
total_agents = len(self.agents)
status_counts = {}
for agent_info in self.agents.values():
status = agent_info.status.value
status_counts[status] = status_counts.get(status, 0) + 1
return {
"total_agents": total_agents,
"status_distribution": status_counts,
"total_tasks_processed": sum(agent.task_count for agent in self.agents.values()),
"total_errors": sum(agent.error_count for agent in self.agents.values())
}
class LoadBalancer:
"""负载均衡器"""
def __init__(self):
self.agent_loads: Dict[str, float] = {}
def select_agent(self, agent_ids: List[str]) -> str:
"""选择负载最低的智能体"""
if not agent_ids:
raise ValueError("没有可用的智能体")
# 简单的轮询策略
min_load = float('inf')
selected_agent = agent_ids[0]
for agent_id in agent_ids:
load = self.agent_loads.get(agent_id, 0)
if load < min_load:
min_load = load
selected_agent = agent_id
# 增加负载
self.agent_loads[selected_agent] = self.agent_loads.get(selected_agent, 0) + 1
return selected_agent
def update_agent_load(self, agent_id: str, load_delta: float):
"""更新智能体负载"""
current_load = self.agent_loads.get(agent_id, 0)
self.agent_loads[agent_id] = max(0, current_load + load_delta)
class HealthMonitor:
"""健康监控器"""
async def monitor_agent(self, agent_id: str):
"""监控智能体健康状态"""
while True:
try:
# 执行健康检查
is_healthy = await self._check_agent_health(agent_id)
if not is_healthy:
await self._handle_unhealthy_agent(agent_id)
await asyncio.sleep(30) # 30秒检查一次
except Exception as e:
print(f"健康监控错误 {agent_id}: {e}")
await asyncio.sleep(60) # 错误时延长检查间隔
async def _check_agent_health(self, agent_id: str) -> bool:
"""检查智能体健康状态"""
# 实现具体的健康检查逻辑
# 例如:检查响应时间、内存使用、错误率等
return True # 简化实现
async def _handle_unhealthy_agent(self, agent_id: str):
"""处理不健康的智能体"""
print(f"检测到不健康的智能体: {agent_id}")
# 实现恢复策略:重启、替换等
2.2 通信总线¶
2.2.1 企业级消息总线¶
import asyncio
import json
from typing import Dict, List, Callable, Any
from dataclasses import dataclass, asdict
from datetime import datetime
import aioredis
@dataclass
class Message:
id: str
sender: str
receiver: str
message_type: str
payload: Dict[str, Any]
timestamp: datetime
priority: int = 1
ttl: int = 3600 # 消息存活时间(秒)
class MessageBus:
"""企业级消息总线"""
def __init__(self, redis_url: str = "redis://localhost:6379"):
self.redis_url = redis_url
self.redis = None
self.subscribers: Dict[str, List[Callable]] = {}
self.message_handlers: Dict[str, Callable] = {}
self.is_running = False
async def initialize(self):
"""初始化消息总线"""
self.redis = await aioredis.from_url(self.redis_url)
self.is_running = True
# 启动消息处理循环
asyncio.create_task(self._message_processing_loop())
async def publish(self, message: Message) -> bool:
"""发布消息"""
try:
# 序列化消息
message_data = {
**asdict(message),
'timestamp': message.timestamp.isoformat()
}
# 发布到Redis
await self.redis.lpush(
f"queue:{message.receiver}",
json.dumps(message_data)
)
# 设置TTL
await self.redis.expire(f"queue:{message.receiver}", message.ttl)
# 发布通知
await self.redis.publish(
f"notify:{message.receiver}",
message.id
)
return True
except Exception as e:
print(f"消息发布失败: {e}")
return False
async def subscribe(self, agent_id: str, handler: Callable[[Message], None]):
"""订阅消息"""
if agent_id not in self.subscribers:
self.subscribers[agent_id] = []
self.subscribers[agent_id].append(handler)
# 启动订阅监听
asyncio.create_task(self._subscribe_to_notifications(agent_id))
async def _subscribe_to_notifications(self, agent_id: str):
"""监听消息通知"""
pubsub = self.redis.pubsub()
await pubsub.subscribe(f"notify:{agent_id}")
async for message in pubsub.listen():
if message['type'] == 'message':
# 处理新消息通知
await self._process_agent_messages(agent_id)
async def _process_agent_messages(self, agent_id: str):
"""处理智能体消息"""
queue_key = f"queue:{agent_id}"
while True:
# 从队列中获取消息
message_data = await self.redis.rpop(queue_key)
if not message_data:
break
try:
# 反序列化消息
message_dict = json.loads(message_data)
message_dict['timestamp'] = datetime.fromisoformat(message_dict['timestamp'])
message = Message(**message_dict)
# 调用处理器
if agent_id in self.subscribers:
for handler in self.subscribers[agent_id]:
try:
await handler(message)
except Exception as e:
print(f"消息处理错误: {e}")
except Exception as e:
print(f"消息反序列化错误: {e}")
async def _message_processing_loop(self):
"""消息处理主循环"""
while self.is_running:
try:
# 清理过期消息
await self._cleanup_expired_messages()
# 处理优先级消息
await self._process_priority_messages()
await asyncio.sleep(1)
except Exception as e:
print(f"消息处理循环错误: {e}")
await asyncio.sleep(5)
async def _cleanup_expired_messages(self):
"""清理过期消息"""
# 获取所有队列
queue_keys = await self.redis.keys("queue:*")
for queue_key in queue_keys:
# 检查队列TTL
ttl = await self.redis.ttl(queue_key)
if ttl == -1: # 没有设置TTL
await self.redis.expire(queue_key, 3600) # 设置默认TTL
async def _process_priority_messages(self):
"""处理优先级消息"""
# 实现优先级消息处理逻辑
pass
async def get_queue_stats(self, agent_id: str) -> Dict[str, Any]:
"""获取队列统计信息"""
queue_key = f"queue:{agent_id}"
queue_length = await self.redis.llen(queue_key)
ttl = await self.redis.ttl(queue_key)
return {
"agent_id": agent_id,
"queue_length": queue_length,
"ttl": ttl
}
2.3 工作流引擎¶
2.3.1 企业级工作流引擎¶
from enum import Enum
from typing import Dict, List, Any, Optional, Callable
import asyncio
import uuid
from datetime import datetime
class WorkflowStatus(Enum):
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
class TaskStatus(Enum):
WAITING = "waiting"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
SKIPPED = "skipped"
@dataclass
class WorkflowTask:
task_id: str
name: str
agent_type: str
required_capabilities: List[str]
inputs: Dict[str, Any]
outputs: Dict[str, Any] = None
status: TaskStatus = TaskStatus.WAITING
dependencies: List[str] = None
retry_count: int = 0
max_retries: int = 3
timeout: int = 300 # 5分钟超时
created_at: datetime = None
started_at: datetime = None
completed_at: datetime = None
error_message: str = None
@dataclass
class Workflow:
workflow_id: str
name: str
description: str
tasks: List[WorkflowTask]
status: WorkflowStatus = WorkflowStatus.PENDING
created_at: datetime = None
started_at: datetime = None
completed_at: datetime = None
context: Dict[str, Any] = None
class WorkflowEngine:
"""企业级工作流引擎"""
def __init__(self, agent_manager: AgentManager, message_bus: MessageBus):
self.agent_manager = agent_manager
self.message_bus = message_bus
self.workflows: Dict[str, Workflow] = {}
self.running_tasks: Dict[str, asyncio.Task] = {}
self.task_callbacks: Dict[str, Callable] = {}
async def create_workflow(self, name: str, description: str,
tasks: List[Dict[str, Any]]) -> str:
"""创建工作流"""
workflow_id = str(uuid.uuid4())
# 创建任务对象
workflow_tasks = []
for task_data in tasks:
task = WorkflowTask(
task_id=str(uuid.uuid4()),
name=task_data["name"],
agent_type=task_data["agent_type"],
required_capabilities=task_data.get("required_capabilities", []),
inputs=task_data.get("inputs", {}),
dependencies=task_data.get("dependencies", []),
max_retries=task_data.get("max_retries", 3),
timeout=task_data.get("timeout", 300),
created_at=datetime.utcnow()
)
workflow_tasks.append(task)
# 创建工作流
workflow = Workflow(
workflow_id=workflow_id,
name=name,
description=description,
tasks=workflow_tasks,
created_at=datetime.utcnow(),
context={}
)
self.workflows[workflow_id] = workflow
return workflow_id
async def start_workflow(self, workflow_id: str) -> bool:
"""启动工作流"""
if workflow_id not in self.workflows:
return False
workflow = self.workflows[workflow_id]
workflow.status = WorkflowStatus.RUNNING
workflow.started_at = datetime.utcnow()
# 启动工作流执行
asyncio.create_task(self._execute_workflow(workflow))
return True
async def _execute_workflow(self, workflow: Workflow):
"""执行工作流"""
try:
while True:
# 查找可执行的任务
ready_tasks = self._get_ready_tasks(workflow)
if not ready_tasks:
# 检查是否所有任务都完成
if self._is_workflow_completed(workflow):
workflow.status = WorkflowStatus.COMPLETED
workflow.completed_at = datetime.utcnow()
break
else:
# 等待正在执行的任务
await asyncio.sleep(1)
continue
# 并行执行准备好的任务
for task in ready_tasks:
asyncio.create_task(self._execute_task(workflow, task))
await asyncio.sleep(0.1) # 短暂等待
except Exception as e:
workflow.status = WorkflowStatus.FAILED
print(f"工作流执行失败 {workflow.workflow_id}: {e}")
def _get_ready_tasks(self, workflow: Workflow) -> List[WorkflowTask]:
"""获取准备执行的任务"""
ready_tasks = []
for task in workflow.tasks:
if task.status != TaskStatus.WAITING:
continue
# 检查依赖是否满足
if self._are_dependencies_satisfied(workflow, task):
ready_tasks.append(task)
return ready_tasks
def _are_dependencies_satisfied(self, workflow: Workflow, task: WorkflowTask) -> bool:
"""检查任务依赖是否满足"""
if not task.dependencies:
return True
task_dict = {t.task_id: t for t in workflow.tasks}
for dep_id in task.dependencies:
if dep_id not in task_dict:
return False
dep_task = task_dict[dep_id]
if dep_task.status != TaskStatus.COMPLETED:
return False
return True
async def _execute_task(self, workflow: Workflow, task: WorkflowTask):
"""执行单个任务"""
task.status = TaskStatus.RUNNING
task.started_at = datetime.utcnow()
try:
# 分配智能体
agent_id = await self.agent_manager.assign_task(
{"task": task.inputs, "task_id": task.task_id},
task.required_capabilities
)
if not agent_id:
raise Exception("没有可用的智能体")
# 执行任务(带超时)
result = await asyncio.wait_for(
self._run_task_on_agent(agent_id, task),
timeout=task.timeout
)
# 任务成功完成
task.outputs = result
task.status = TaskStatus.COMPLETED
task.completed_at = datetime.utcnow()
# 更新工作流上下文
workflow.context[task.task_id] = result
# 通知智能体管理器任务完成
await self.agent_manager.complete_task(agent_id, success=True)
except asyncio.TimeoutError:
task.error_message = "任务执行超时"
await self._handle_task_failure(workflow, task)
except Exception as e:
task.error_message = str(e)
await self._handle_task_failure(workflow, task)
async def _run_task_on_agent(self, agent_id: str, task: WorkflowTask) -> Dict[str, Any]:
"""在智能体上运行任务"""
# 获取智能体实例
agent_instance = self.agent_manager.agent_instances.get(agent_id)
if not agent_instance:
raise Exception(f"智能体实例不存在: {agent_id}")
# 执行任务
if hasattr(agent_instance, 'process_task'):
return await agent_instance.process_task(task.inputs)
else:
raise Exception("智能体不支持任务处理")
async def _handle_task_failure(self, workflow: Workflow, task: WorkflowTask):
"""处理任务失败"""
task.retry_count += 1
if task.retry_count <= task.max_retries:
# 重试任务
task.status = TaskStatus.WAITING
print(f"任务重试 {task.task_id} (第{task.retry_count}次)")
else:
# 任务最终失败
task.status = TaskStatus.FAILED
workflow.status = WorkflowStatus.FAILED
print(f"任务最终失败 {task.task_id}: {task.error_message}")
def _is_workflow_completed(self, workflow: Workflow) -> bool:
"""检查工作流是否完成"""
for task in workflow.tasks:
if task.status in [TaskStatus.WAITING, TaskStatus.RUNNING]:
return False
return True
def get_workflow_status(self, workflow_id: str) -> Optional[Dict[str, Any]]:
"""获取工作流状态"""
if workflow_id not in self.workflows:
return None
workflow = self.workflows[workflow_id]
task_stats = {}
for status in TaskStatus:
task_stats[status.value] = sum(
1 for task in workflow.tasks if task.status == status
)
return {
"workflow_id": workflow_id,
"name": workflow.name,
"status": workflow.status.value,
"created_at": workflow.created_at.isoformat() if workflow.created_at else None,
"started_at": workflow.started_at.isoformat() if workflow.started_at else None,
"completed_at": workflow.completed_at.isoformat() if workflow.completed_at else None,
"total_tasks": len(workflow.tasks),
"task_statistics": task_stats
}
3. 安全架构设计¶
3.1 认证与授权¶
3.1.1 JWT认证系统¶
import jwt
import bcrypt
from datetime import datetime, timedelta
from typing import Dict, Optional, List
from fastapi import HTTPException, Depends
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
class AuthenticationService:
"""认证服务"""
def __init__(self, secret_key: str, algorithm: str = "HS256"):
self.secret_key = secret_key
self.algorithm = algorithm
self.token_expiry = timedelta(hours=24)
self.refresh_token_expiry = timedelta(days=7)
def hash_password(self, password: str) -> str:
"""密码哈希"""
salt = bcrypt.gensalt()
return bcrypt.hashpw(password.encode('utf-8'), salt).decode('utf-8')
def verify_password(self, password: str, hashed_password: str) -> bool:
"""验证密码"""
return bcrypt.checkpw(password.encode('utf-8'), hashed_password.encode('utf-8'))
def create_access_token(self, user_id: str, roles: List[str]) -> str:
"""创建访问令牌"""
payload = {
"user_id": user_id,
"roles": roles,
"exp": datetime.utcnow() + self.token_expiry,
"iat": datetime.utcnow(),
"type": "access"
}
return jwt.encode(payload, self.secret_key, algorithm=self.algorithm)
def create_refresh_token(self, user_id: str) -> str:
"""创建刷新令牌"""
payload = {
"user_id": user_id,
"exp": datetime.utcnow() + self.refresh_token_expiry,
"iat": datetime.utcnow(),
"type": "refresh"
}
return jwt.encode(payload, self.secret_key, algorithm=self.algorithm)
def verify_token(self, token: str) -> Dict[str, any]:
"""验证令牌"""
try:
payload = jwt.decode(token, self.secret_key, algorithms=[self.algorithm])
return payload
except jwt.ExpiredSignatureError:
raise HTTPException(status_code=401, detail="令牌已过期")
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="无效令牌")
class AuthorizationService:
"""授权服务"""
def __init__(self):
self.permissions = {
"admin": ["*"], # 管理员拥有所有权限
"agent_manager": ["agent:create", "agent:read", "agent:update", "agent:delete"],
"workflow_manager": ["workflow:create", "workflow:read", "workflow:update", "workflow:delete"],
"monitor": ["system:read", "metrics:read"],
"user": ["agent:read", "workflow:read"]
}
def check_permission(self, user_roles: List[str], required_permission: str) -> bool:
"""检查权限"""
for role in user_roles:
role_permissions = self.permissions.get(role, [])
# 检查是否有通配符权限
if "*" in role_permissions:
return True
# 检查具体权限
if required_permission in role_permissions:
return True
# 检查模式匹配权限
for permission in role_permissions:
if self._match_permission(permission, required_permission):
return True
return False
def _match_permission(self, pattern: str, permission: str) -> bool:
"""权限模式匹配"""
if pattern.endswith("*"):
prefix = pattern[:-1]
return permission.startswith(prefix)
return pattern == permission
# FastAPI依赖注入
security = HTTPBearer()
auth_service = AuthenticationService("your-secret-key")
authz_service = AuthorizationService()
async def get_current_user(credentials: HTTPAuthorizationCredentials = Depends(security)):
"""获取当前用户"""
token = credentials.credentials
payload = auth_service.verify_token(token)
return payload
def require_permission(permission: str):
"""权限装饰器"""
def decorator(func):
async def wrapper(*args, **kwargs):
# 从依赖注入中获取用户信息
user = kwargs.get('current_user')
if not user:
raise HTTPException(status_code=401, detail="未认证")
user_roles = user.get('roles', [])
if not authz_service.check_permission(user_roles, permission):
raise HTTPException(status_code=403, detail="权限不足")
return await func(*args, **kwargs)
return wrapper
return decorator
3.2 数据加密与安全¶
3.2.1 数据加密服务¶
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
import base64
import os
from typing import Union
class EncryptionService:
"""数据加密服务"""
def __init__(self, password: str = None):
if password:
self.key = self._derive_key(password)
else:
self.key = Fernet.generate_key()
self.cipher = Fernet(self.key)
def _derive_key(self, password: str) -> bytes:
"""从密码派生密钥"""
password_bytes = password.encode()
salt = b'salt_' # 在生产环境中应使用随机盐
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=salt,
iterations=100000,
)
key = base64.urlsafe_b64encode(kdf.derive(password_bytes))
return key
def encrypt(self, data: Union[str, bytes]) -> str:
"""加密数据"""
if isinstance(data, str):
data = data.encode()
encrypted_data = self.cipher.encrypt(data)
return base64.urlsafe_b64encode(encrypted_data).decode()
def decrypt(self, encrypted_data: str) -> str:
"""解密数据"""
encrypted_bytes = base64.urlsafe_b64decode(encrypted_data.encode())
decrypted_data = self.cipher.decrypt(encrypted_bytes)
return decrypted_data.decode()
def encrypt_dict(self, data: Dict[str, Any]) -> Dict[str, str]:
"""加密字典数据"""
encrypted_dict = {}
for key, value in data.items():
encrypted_dict[key] = self.encrypt(str(value))
return encrypted_dict
def decrypt_dict(self, encrypted_dict: Dict[str, str]) -> Dict[str, str]:
"""解密字典数据"""
decrypted_dict = {}
for key, encrypted_value in encrypted_dict.items():
decrypted_dict[key] = self.decrypt(encrypted_value)
return decrypted_dict
class SecureDataStore:
"""安全数据存储"""
def __init__(self, encryption_service: EncryptionService):
self.encryption = encryption_service
self.data_store = {} # 在实际应用中应使用数据库
async def store_sensitive_data(self, key: str, data: Dict[str, Any]) -> bool:
"""存储敏感数据"""
try:
# 加密数据
encrypted_data = self.encryption.encrypt_dict(data)
# 存储加密后的数据
self.data_store[key] = {
"data": encrypted_data,
"created_at": datetime.utcnow().isoformat(),
"encrypted": True
}
return True
except Exception as e:
print(f"数据存储失败: {e}")
return False
async def retrieve_sensitive_data(self, key: str) -> Optional[Dict[str, Any]]:
"""检索敏感数据"""
try:
if key not in self.data_store:
return None
stored_data = self.data_store[key]
if stored_data.get("encrypted", False):
# 解密数据
decrypted_data = self.encryption.decrypt_dict(stored_data["data"])
return decrypted_data
else:
return stored_data["data"]
except Exception as e:
print(f"数据检索失败: {e}")
return None
4. 系统实现¶
4.1 完整系统集成¶
4.1.1 企业级多智能体系统¶
from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel
import asyncio
from typing import Dict, List, Any, Optional
class EnterpriseMultiAgentSystem:
"""企业级多智能体系统"""
def __init__(self):
self.app = FastAPI(title="Enterprise Multi-Agent System")
self.agent_manager = AgentManager()
self.message_bus = MessageBus()
self.workflow_engine = WorkflowEngine(self.agent_manager, self.message_bus)
self.auth_service = AuthenticationService("your-secret-key")
self.authz_service = AuthorizationService()
self.encryption_service = EncryptionService()
self.secure_store = SecureDataStore(self.encryption_service)
self._setup_routes()
def _setup_routes(self):
"""设置API路由"""
@self.app.post("/auth/login")
async def login(credentials: LoginRequest):
# 验证用户凭据(简化实现)
if credentials.username == "admin" and credentials.password == "password":
access_token = self.auth_service.create_access_token(
user_id="admin",
roles=["admin"]
)
refresh_token = self.auth_service.create_refresh_token("admin")
return {
"access_token": access_token,
"refresh_token": refresh_token,
"token_type": "bearer"
}
else:
raise HTTPException(status_code=401, detail="无效凭据")
@self.app.post("/agents/register")
async def register_agent(
request: AgentRegistrationRequest,
current_user: Dict = Depends(get_current_user)
):
# 检查权限
if not self.authz_service.check_permission(
current_user.get("roles", []),
"agent:create"
):
raise HTTPException(status_code=403, detail="权限不足")
# 注册智能体
agent_id = await self.agent_manager.register_agent(
request.agent_type,
request.capabilities,
request.agent_factory
)
return {"agent_id": agent_id, "status": "registered"}
@self.app.post("/workflows/create")
async def create_workflow(
request: WorkflowCreateRequest,
current_user: Dict = Depends(get_current_user)
):
# 检查权限
if not self.authz_service.check_permission(
current_user.get("roles", []),
"workflow:create"
):
raise HTTPException(status_code=403, detail="权限不足")
# 创建工作流
workflow_id = await self.workflow_engine.create_workflow(
request.name,
request.description,
request.tasks
)
return {"workflow_id": workflow_id, "status": "created"}
@self.app.post("/workflows/{workflow_id}/start")
async def start_workflow(
workflow_id: str,
current_user: Dict = Depends(get_current_user)
):
# 检查权限
if not self.authz_service.check_permission(
current_user.get("roles", []),
"workflow:update"
):
raise HTTPException(status_code=403, detail="权限不足")
# 启动工作流
success = await self.workflow_engine.start_workflow(workflow_id)
if success:
return {"status": "started"}
else:
raise HTTPException(status_code=404, detail="工作流不存在")
@self.app.get("/system/status")
async def get_system_status(
current_user: Dict = Depends(get_current_user)
):
# 检查权限
if not self.authz_service.check_permission(
current_user.get("roles", []),
"system:read"
):
raise HTTPException(status_code=403, detail="权限不足")
# 获取系统状态
agent_stats = self.agent_manager.get_agent_statistics()
return {
"system_status": "healthy",
"agent_statistics": agent_stats,
"timestamp": datetime.utcnow().isoformat()
}
async def initialize(self):
"""初始化系统"""
await self.message_bus.initialize()
print("企业级多智能体系统初始化完成")
async def shutdown(self):
"""关闭系统"""
self.message_bus.is_running = False
print("企业级多智能体系统已关闭")
# 请求模型
class LoginRequest(BaseModel):
username: str
password: str
class AgentRegistrationRequest(BaseModel):
agent_type: str
capabilities: List[str]
agent_factory: Any # 在实际应用中需要更具体的类型
class WorkflowCreateRequest(BaseModel):
name: str
description: str
tasks: List[Dict[str, Any]]
5. 部署与运维¶
5.1 Docker容器化部署¶
5.1.1 Dockerfile¶
FROM python:3.9-slim
WORKDIR /app
# 安装系统依赖
RUN apt-get update && apt-get install -y \
gcc \
&& rm -rf /var/lib/apt/lists/*
# 复制依赖文件
COPY requirements.txt .
# 安装Python依赖
RUN pip install --no-cache-dir -r requirements.txt
# 复制应用代码
COPY . .
# 暴露端口
EXPOSE 8000
# 启动命令
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
5.1.2 Docker Compose配置¶
version: '3.8'
services:
multi-agent-system:
build: .
ports:
- "8000:8000"
environment:
- REDIS_URL=redis://redis:6379
- DATABASE_URL=postgresql://user:password@postgres:5432/multiagent
depends_on:
- redis
- postgres
volumes:
- ./logs:/app/logs
restart: unless-stopped
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
restart: unless-stopped
postgres:
image: postgres:15-alpine
environment:
- POSTGRES_DB=multiagent
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
restart: unless-stopped
nginx:
image: nginx:alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./ssl:/etc/nginx/ssl
depends_on:
- multi-agent-system
restart: unless-stopped
volumes:
redis_data:
postgres_data:
5.2 监控与日志¶
5.2.1 系统监控配置¶
import logging
import sys
from datetime import datetime
from typing import Dict, Any
import psutil
import asyncio
class SystemMonitor:
"""系统监控"""
def __init__(self):
self.setup_logging()
self.metrics = {}
self.is_monitoring = False
def setup_logging(self):
"""设置日志"""
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('/app/logs/system.log'),
logging.StreamHandler(sys.stdout)
]
)
self.logger = logging.getLogger(__name__)
async def start_monitoring(self, interval: int = 60):
"""开始监控"""
self.is_monitoring = True
self.logger.info("系统监控已启动")
while self.is_monitoring:
try:
await self.collect_system_metrics()
await self.check_system_health()
await asyncio.sleep(interval)
except Exception as e:
self.logger.error(f"监控错误: {e}")
await asyncio.sleep(interval)
async def collect_system_metrics(self):
"""收集系统指标"""
# CPU使用率
cpu_percent = psutil.cpu_percent(interval=1)
# 内存使用情况
memory = psutil.virtual_memory()
# 磁盘使用情况
disk = psutil.disk_usage('/')
# 网络统计
network = psutil.net_io_counters()
self.metrics = {
"timestamp": datetime.utcnow().isoformat(),
"cpu_percent": cpu_percent,
"memory": {
"total": memory.total,
"available": memory.available,
"percent": memory.percent,
"used": memory.used
},
"disk": {
"total": disk.total,
"used": disk.used,
"free": disk.free,
"percent": (disk.used / disk.total) * 100
},
"network": {
"bytes_sent": network.bytes_sent,
"bytes_recv": network.bytes_recv,
"packets_sent": network.packets_sent,
"packets_recv": network.packets_recv
}
}
self.logger.info(f"系统指标: CPU {cpu_percent}%, 内存 {memory.percent}%, 磁盘 {self.metrics['disk']['percent']:.1f}%")
async def check_system_health(self):
"""检查系统健康状态"""
alerts = []
# CPU告警
if self.metrics["cpu_percent"] > 80:
alerts.append(f"CPU使用率过高: {self.metrics['cpu_percent']}%")
# 内存告警
if self.metrics["memory"]["percent"] > 85:
alerts.append(f"内存使用率过高: {self.metrics['memory']['percent']}%")
# 磁盘告警
if self.metrics["disk"]["percent"] > 90:
alerts.append(f"磁盘使用率过高: {self.metrics['disk']['percent']:.1f}%")
# 发送告警
for alert in alerts:
self.logger.warning(f"系统告警: {alert}")
await self.send_alert(alert)
async def send_alert(self, message: str):
"""发送告警"""
# 这里可以集成邮件、Slack、钉钉等告警渠道
print(f"[ALERT] {message}")
def get_current_metrics(self) -> Dict[str, Any]:
"""获取当前指标"""
return self.metrics
6. 实践项目¶
项目:构建企业级客服智能体系统¶
6.1.1 项目需求¶
- 多智能体协作:意图识别、知识检索、回答生成
- 工作流管理:客服流程自动化
- 监控告警:系统性能和服务质量监控
- 安全认证:用户认证和权限管理
6.1.2 实现框架¶
# 学员需要完成的项目代码框架
class CustomerServiceSystem(EnterpriseMultiAgentSystem):
def __init__(self):
super().__init__()
self.setup_customer_service_agents()
self.setup_customer_service_workflows()
def setup_customer_service_agents(self):
"""设置客服智能体"""
# TODO: 实现意图识别智能体
# TODO: 实现知识检索智能体
# TODO: 实现回答生成智能体
pass
def setup_customer_service_workflows(self):
"""设置客服工作流"""
# TODO: 定义客服处理流程
# TODO: 配置异常处理机制
pass
async def handle_customer_query(self, query: str, customer_id: str):
"""处理客户查询"""
# TODO: 创建并执行客服工作流
pass
总结¶
本课程涵盖了企业级多智能体系统的完整架构设计和实现,包括:
- 架构设计:分层架构、微服务模式、设计原则
- 核心组件:智能体管理、消息总线、工作流引擎
- 安全机制:认证授权、数据加密、安全存储
- 系统实现:完整的企业级系统集成
- 部署运维:容器化部署、监控告警、日志管理
下一课程将学习应用实践与部署运维的高级技术。