freedak/SBrainCO
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
64d2cd804323710f44fadca290587ede517a6112
SBrainCO/物化视图更新.md
T
freedakgmail a090fc7c19 fix: 修复API数据正确性问题 
- 移除3个API的operating_expense IS NOT NULL过滤，改用COALESCE
  - /overview/store-profit-ranking: 93家门店全部纳入排名
  - /overview/profit-opportunity: 所有有营收门店纳入机会计算
  - /overview/profit-trend: LEFT JOIN替代INNER JOIN
- 重建v_store_operating_expense_monthly视图，去除硬编码_april视图
  - sales CTE改用mv_store_site_profile_monthly
  - food CTE改用mv_store_theoretical_actual_cost_monthly
  - 支持任意月份查询
- 修复人工优化和能源优化opportunity为负数问题
- 新增scripts/refresh_materialized_views.sh自动化刷新脚本
- 新增本体.md/API.md/原始数据级导入处理.md/物化视图更新.md/API数据检查报告.md
2026-08-11 19:25:49 +08:00

11 KiB
Raw Blame History

物化视图更新手册

一、物化视图依赖链

数据更新后必须按依赖顺序刷新物化视图，否则下游数据不一致。

bill_records (原始表)
    │
    ▼ REFRESH
bill_fact (物化视图) ★ 第一层，最关键
    │
    ├─▶ mv_daily_revenue
    ├─▶ mv_store_risk_rating_monthly ★ 门店评估基础
    ├─▶ mv_dish_sku_abc_monthly
    ├─▶ mv_dish_sku_summary_monthly
    ├─▶ mv_dish_store_summary_monthly
    ├─▶ mv_dish_basket_monthly
    ├─▶ mv_dish_pair_summary_monthly
    ├─▶ mv_store_action_priority_deep_monthly
    ├─▶ mv_store_area_efficiency_monthly
    ├─▶ mv_store_benchmark_composite_monthly
    ├─▶ mv_store_category_mix_monthly
    ├─▶ mv_store_deep_diagnosis_monthly
    ├─▶ mv_store_member_opportunity_monthly
    ├─▶ mv_store_overlap_risk_monthly
    ├─▶ mv_store_platform_economics_monthly
    ├─▶ mv_store_site_profile_monthly
    ├─▶ mv_store_site_replication_monthly
    ├─▶ mv_store_repeat_summary_monthly
    ├─▶ mv_district_site_benchmark_monthly
    ├─▶ mv_site_segment_benchmark_monthly
    └─▶ mv_store_theoretical_actual_cost_monthly ★ 门店成本

inventory_cost_records (原始表)
    │
    ▼ REFRESH
mv_inventory_cost_classified_monthly

operating_expense_records (原始表)
    │
    ▼ VIEW (无需刷新)
v_operating_expense_unit_monthly
    │
    ▼ VIEW (无需刷新)
v_store_operating_expense_monthly (按 sales_store_code 聚合)
    │
    ▼ INSERT/UPDATE (需手动操作)
mv_store_operating_expense_monthly (物化表)

二、刷新命令清单

2.1 账单数据更新后（必刷）

-- ★ 第一优先级：bill_fact 是所有分析的基础
REFRESH MATERIALIZED VIEW analytics.bill_fact;

-- ★ 第二优先级：门店评估相关
REFRESH MATERIALIZED VIEW analytics.mv_store_risk_rating_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_theoretical_actual_cost_monthly;

-- 第三优先级：菜品分析
REFRESH MATERIALIZED VIEW analytics.mv_dish_sku_abc_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_sku_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_store_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_basket_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_pair_summary_monthly;

-- 第四优先级：门店深度分析
REFRESH MATERIALIZED VIEW analytics.mv_store_action_priority_deep_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_area_efficiency_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_benchmark_composite_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_category_mix_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_deep_diagnosis_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_member_opportunity_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_overlap_risk_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_platform_economics_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_site_profile_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_site_replication_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_repeat_summary_monthly;

-- 第五优先级：日度与区域
REFRESH MATERIALIZED VIEW analytics.mv_daily_revenue;
REFRESH MATERIALIZED VIEW analytics.mv_district_site_benchmark_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_site_segment_benchmark_monthly;

2.2 库存成本数据更新后

REFRESH MATERIALIZED VIEW analytics.mv_inventory_cost_classified_monthly;
-- 理论vs实际成本也依赖库存数据
REFRESH MATERIALIZED VIEW analytics.mv_store_theoretical_actual_cost_monthly;

2.3 费用数据更新后

-- v_operating_expense_unit_monthly 和 v_store_operating_expense_monthly 是普通视图，无需刷新
-- 但 mv_store_operating_expense_monthly 是物化表，需要手动更新：

-- 方法1：删除+插入（推荐，简单可靠）
DELETE FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '2026-04-01';
INSERT INTO analytics.mv_store_operating_expense_monthly
SELECT * FROM analytics.v_store_operating_expense_monthly WHERE report_month = '2026-04-01';

-- 方法2：全量重建
TRUNCATE analytics.mv_store_operating_expense_monthly;
INSERT INTO analytics.mv_store_operating_expense_monthly
SELECT * FROM analytics.v_store_operating_expense_monthly;

三、一键刷新脚本

#!/bin/bash
# refresh_all_mv.sh - 刷新所有物化视图
# 用法: bash refresh_all_mv.sh [month] (默认 2026-04)

MONTH=${1:-2026-04}
MONTH_START="${MONTH}-01"

psql -h localhost -p 5432 -U freedak -d bill_query <<EOF
-- 第一层
REFRESH MATERIALIZED VIEW analytics.bill_fact;

-- 第二层
REFRESH MATERIALIZED VIEW analytics.mv_store_risk_rating_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_theoretical_actual_cost_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_daily_revenue;

-- 第三层
REFRESH MATERIALIZED VIEW analytics.mv_dish_sku_abc_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_sku_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_store_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_basket_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_dish_pair_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_action_priority_deep_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_area_efficiency_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_benchmark_composite_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_category_mix_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_deep_diagnosis_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_member_opportunity_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_overlap_risk_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_platform_economics_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_site_profile_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_site_replication_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_store_repeat_summary_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_district_site_benchmark_monthly;
REFRESH MATERIALIZED VIEW analytics.mv_site_segment_benchmark_monthly;

-- 库存成本
REFRESH MATERIALIZED VIEW analytics.mv_inventory_cost_classified_monthly;

-- 费用物化表
DELETE FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '${MONTH_START}';
INSERT INTO analytics.mv_store_operating_expense_monthly
SELECT * FROM analytics.v_store_operating_expense_monthly WHERE report_month = '${MONTH_START}';

-- 验证
SELECT 'bill_fact' AS source, count(DISTINCT store_code) AS stores, round(sum(received_total)::numeric,2) AS received
FROM analytics.bill_fact WHERE closed_at >= '${MONTH_START}' AND closed_at < ('${MONTH_START}'::date + interval '1 month')
UNION ALL
SELECT 'risk_rating', count(*), round(sum(received)::numeric,2)
FROM analytics.mv_store_risk_rating_monthly WHERE month_start = '${MONTH_START}'
UNION ALL
SELECT 'theo_cost', count(*), round(sum(sales_received)::numeric,2)
FROM analytics.mv_store_theoretical_actual_cost_monthly WHERE month_start = '${MONTH_START}'
UNION ALL
SELECT 'op_expense', count(*), round(sum(received)::numeric,2)
FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '${MONTH_START}';
EOF

四、物化视图 vs 普通视图 vs 物化表

类型 刷新方式 示例 说明
物化视图 (Materialized View) REFRESH MATERIALIZED VIEW bill_fact, mv_store_risk_rating_monthly 数据快照，需手动刷新
普通视图 (View) 实时查询，无需刷新 v_store_operating_expense_monthly, v_operating_expense_account_monthly 每次查询实时计算
物化表 (Table) INSERT/UPDATE/DELETE mv_store_operating_expense_monthly 物理表，需手动写入

判断方法

-- 查看对象类型
SELECT relname, relkind 
FROM pg_class 
WHERE relname LIKE 'mv_%' OR relname LIKE 'v_%'
AND relnamespace = (SELECT oid FROM pg_namespace WHERE nspname = 'analytics');

-- relkind 含义:
-- 'm' = 物化视图 (需要 REFRESH)
-- 'v' = 普通视图 (无需刷新)
-- 'r' = 普通表 (需要 INSERT/UPDATE)

五、常见问题

5.1 数据不一致

症状: API返回的门店数少于实际门店数，或实收合计与 bill_fact 不一致

原因: 导入新数据后未刷新物化视图

修复: 执行上述刷新命令

5.2 费用未纳入门店评估

症状: 某门店在评估列表中缺失或费用为0

原因:

  1. operating_expense_store_mapping 中 sales_store_code 未设置
  2. include_in_operating_analysis 标志为 false
  3. mv_store_operating_expense_monthly 物化表未更新

修复:

-- 检查映射
SELECT * FROM operating_expense_store_mapping WHERE cost_unit_source_name ILIKE '%门店名%';

-- 更新映射
UPDATE operating_expense_store_mapping 
SET sales_store_code = '0022', include_in_operating_analysis = true 
WHERE cost_unit_source_name = '哈马尔罕大钟寺店';

-- 刷新物化表
DELETE FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '2026-04-01';
INSERT INTO analytics.mv_store_operating_expense_monthly
SELECT * FROM analytics.v_store_operating_expense_monthly WHERE report_month = '2026-04-01';

5.3 费用重复行

症状: 同一门店在 mv_store_operating_expense_monthly 中出现多行

原因: 多个费用单位映射到同一 sales_store_code，但视图未按 sales_store_code 聚合

修复: 确认 v_store_operating_expense_monthly 视图定义包含 expense_agg CTE，按 sales_store_code GROUP BY

5.4 固定月份物化视图

问题: *_april 系列物化视图硬编码2026年4月，不支持其他月份

影响: 5月数据需要创建新的 *_may 系列物化视图，或修改为通用月份

当前状态: v_store_operating_expense_monthly 视图中 sales/food/loc CTE 引用了 v_store_site_profile_april 和 v_store_theoretical_actual_cost_april，这些是4月专用视图

六、数据一致性验证

刷新后执行以下验证：

-- 1. 门店数一致性
SELECT 'bill_fact' AS src, count(DISTINCT store_code) AS stores
FROM analytics.bill_fact 
WHERE closed_at >= '2026-04-01' AND closed_at < '2026-05-01'
UNION ALL
SELECT 'risk_rating', count(*) 
FROM analytics.mv_store_risk_rating_monthly WHERE month_start = '2026-04-01'
UNION ALL
SELECT 'theo_cost', count(*) 
FROM analytics.mv_store_theoretical_actual_cost_monthly WHERE month_start = '2026-04-01'
UNION ALL
SELECT 'op_expense', count(*) 
FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '2026-04-01';

-- 2. 实收一致性
SELECT 'bill_fact' AS src, round(sum(received_total)::numeric,2) AS received
FROM analytics.bill_fact 
WHERE closed_at >= '2026-04-01' AND closed_at < '2026-05-01'
UNION ALL
SELECT 'risk_rating', round(sum(received)::numeric,2)
FROM analytics.mv_store_risk_rating_monthly WHERE month_start = '2026-04-01'
UNION ALL
SELECT 'theo_cost', round(sum(sales_received)::numeric,2)
FROM analytics.mv_store_theoretical_actual_cost_monthly WHERE month_start = '2026-04-01'
UNION ALL
SELECT 'op_expense', round(sum(received)::numeric,2)
FROM analytics.mv_store_operating_expense_monthly WHERE report_month = '2026-04-01';

-- 3. 差异门店排查
SELECT store_code, store_name, received
FROM analytics.mv_store_risk_rating_monthly 
WHERE month_start = '2026-04-01' AND received > 0
AND store_code NOT IN (
  SELECT DISTINCT store_code FROM analytics.mv_store_operating_expense_monthly 
  WHERE report_month = '2026-04-01'
);
Reference in New Issue View Git Blame Copy Permalink