數倉性能調優:大寬表關聯MERGE性能優化

来源:https://www.cnblogs.com/huaweiyun/archive/2023/07/04/17525697.html
-Advertisement-
Play Games

摘要:本文主要為大家講解在數倉性能調優過程中,關於大寬表關聯MERGE性能優化過程。 本文分享自華為雲社區《GaussDB(DWS)性能調優:大寬表關聯MERGE性能優化》,作者:譡里個檔。 【業務背景】 如下MERGE語句執行耗時長達2034s MERGE INTO sdifin.hah_ae_l ...


摘要:本文主要為大家講解在數倉性能調優過程中,關於大寬表關聯MERGE性能優化過程。

本文分享自華為雲社區《GaussDB(DWS)性能調優:大寬表關聯MERGE性能優化》,作者:譡里個檔。

【業務背景】

如下MERGE語句執行耗時長達2034s

MERGE INTO sdifin.hah_ae_line_sr_t_02_8663 Event_1u18olr USING (
 WITH Event_1ix1dzn AS (
 SELECT
 "sr38","sr39","sr40","sr41","sr42","sr43","sr44","sr45","sr46","sr47",
 "sr48","sr49","sr50","sr51","sr52","sr53","sr54","sr55","sr56","sr57",
 "sr58","sr59","sr60","unit_code","created_by","creation_date",
 "last_updated_by","last_update_date","ss_id","del_flag","crt_cycle_id",
 "last_upd_cycle_id","crt_job_instance_id","upd_job_instance_id",
 "dq_improve_flag","last_modified_date","ae_header_id","ae_line_num",
 "application_code","sr1","sr2","sr3","sr4","sr5","sr6","sr7","sr8",
 "sr9","sr10","sr11","sr12","sr13","sr14","sr15","sr16","sr17",
 "sr18","sr19","sr20","sr21","sr22","sr23","sr24","sr25","sr26",
 "sr27","sr28","sr29","sr30","sr31","sr32","sr33","sr34","sr35",
 "sr36","sr37" 
 FROM stgfin.dlt_hah_ae_line_sr_t_02_8663
 ),
    Event_1u18olr AS (
 SELECT 
 "ae_header_id","ae_line_num","application_code","sr1","sr2","sr3",
 "sr4","sr5","sr6","sr7","sr8","sr9","sr10","sr11","sr12","sr13",
 "sr14","sr15","sr16","sr17","sr18","sr19","sr20","sr21","sr22",
 "sr23","sr24","sr25","sr26","sr27","sr28","sr29","sr30","sr31",
 "sr32","sr33","sr34","sr35","sr36","sr37","sr38","sr39","sr40",
 "sr41","sr42","sr43","sr44","sr45","sr46","sr47","sr48","sr49",
 "sr50","sr51","sr52","sr53","sr54","sr55","sr56","sr57","sr58",
 "sr59","sr60","unit_code","created_by","creation_date",
 "last_updated_by","last_update_date","ss_id","del_flag",
 "crt_cycle_id",20230520000000 AS "last_upd_cycle_id",
 -1 AS "crt_job_instance_id",-1 AS "upd_job_instance_id",
 'N' AS "dq_improve_flag",sysdate() AS "last_modified_date" 
 FROM Event_1ix1dzn
 )
 SELECT 
 "ae_header_id","ae_line_num","application_code","sr1","sr2","sr3",
 "sr4","sr5","sr6","sr7","sr8","sr9","sr10","sr11","sr12","sr13",
 "sr14","sr15","sr16","sr17","sr18","sr19","sr20","sr21","sr22",
 "sr23","sr24","sr25","sr26","sr27","sr28","sr29","sr30","sr31",
 "sr32","sr33","sr34","sr35","sr36","sr37","sr38","sr39","sr40",
 "sr41","sr42","sr43","sr44","sr45","sr46","sr47","sr48","sr49",
 "sr50","sr51","sr52","sr53","sr54","sr55","sr56","sr57","sr58",
 "sr59","sr60","unit_code","created_by","creation_date",
 "last_updated_by","last_update_date","ss_id","del_flag",
 "crt_cycle_id","last_upd_cycle_id","crt_job_instance_id",
 "upd_job_instance_id","dq_improve_flag","last_modified_date" 
 FROM Event_1u18olr
) Event_1ix1dzn ON (Event_1u18olr."ae_header_id" = Event_1ix1dzn."ae_header_id" AND Event_1u18olr."ae_line_num" = Event_1ix1dzn."ae_line_num") 
WHEN MATCHED THEN UPDATE SET "application_code" = Event_1ix1dzn."application_code",
 "sr1" = Event_1ix1dzn."sr1",
 "sr2" = Event_1ix1dzn."sr2",
 "sr3" = Event_1ix1dzn."sr3",
 "sr4" = Event_1ix1dzn."sr4",
 "sr5" = Event_1ix1dzn."sr5",
 "sr6" = Event_1ix1dzn."sr6",
 "sr7" = Event_1ix1dzn."sr7",
 "sr8" = Event_1ix1dzn."sr8",
 "sr9" = Event_1ix1dzn."sr9",
 "sr10" = Event_1ix1dzn."sr10",
 "sr11" = Event_1ix1dzn."sr11",
 "sr12" = Event_1ix1dzn."sr12",
 "sr13" = Event_1ix1dzn."sr13",
 "sr14" = Event_1ix1dzn."sr14",
 "sr15" = Event_1ix1dzn."sr15",
 "sr16" = Event_1ix1dzn."sr16",
 "sr17" = Event_1ix1dzn."sr17",
 "sr18" = Event_1ix1dzn."sr18",
 "sr19" = Event_1ix1dzn."sr19",
 "sr20" = Event_1ix1dzn."sr20",
 "sr21" = Event_1ix1dzn."sr21",
 "sr22" = Event_1ix1dzn."sr22",
 "sr23" = Event_1ix1dzn."sr23",
 "sr24" = Event_1ix1dzn."sr24",
 "sr25" = Event_1ix1dzn."sr25",
 "sr26" = Event_1ix1dzn."sr26",
 "sr27" = Event_1ix1dzn."sr27",
 "sr28" = Event_1ix1dzn."sr28",
 "sr29" = Event_1ix1dzn."sr29",
 "sr30" = Event_1ix1dzn."sr30",
 "sr31" = Event_1ix1dzn."sr31",
 "sr32" = Event_1ix1dzn."sr32",
 "sr33" = Event_1ix1dzn."sr33",
 "sr34" = Event_1ix1dzn."sr34",
 "sr35" = Event_1ix1dzn."sr35",
 "sr36" = Event_1ix1dzn."sr36",
 "sr37" = Event_1ix1dzn."sr37",
 "sr38" = Event_1ix1dzn."sr38",
 "sr39" = Event_1ix1dzn."sr39",
 "sr40" = Event_1ix1dzn."sr40",
 "sr41" = Event_1ix1dzn."sr41",
 "sr42" = Event_1ix1dzn."sr42",
 "sr43" = Event_1ix1dzn."sr43",
 "sr44" = Event_1ix1dzn."sr44",
 "sr45" = Event_1ix1dzn."sr45",
 "sr46" = Event_1ix1dzn."sr46",
 "sr47" = Event_1ix1dzn."sr47",
 "sr48" = Event_1ix1dzn."sr48",
 "sr49" = Event_1ix1dzn."sr49",
 "sr50" = Event_1ix1dzn."sr50",
 "sr51" = Event_1ix1dzn."sr51",
 "sr52" = Event_1ix1dzn."sr52",
 "sr53" = Event_1ix1dzn."sr53",
 "sr54" = Event_1ix1dzn."sr54",
 "sr55" = Event_1ix1dzn."sr55",
 "sr56" = Event_1ix1dzn."sr56",
 "sr57" = Event_1ix1dzn."sr57",
 "sr58" = Event_1ix1dzn."sr58",
 "sr59" = Event_1ix1dzn."sr59",
 "sr60" = Event_1ix1dzn."sr60",
 "unit_code" = Event_1ix1dzn."unit_code",
 "created_by" = Event_1ix1dzn."created_by",
 "creation_date" = Event_1ix1dzn."creation_date",
 "last_updated_by" = Event_1ix1dzn."last_updated_by",
 "last_update_date" = Event_1ix1dzn."last_update_date",
 "ss_id" = Event_1ix1dzn."ss_id",
 "del_flag" = Event_1ix1dzn."del_flag",
 "crt_cycle_id" = Event_1ix1dzn."crt_cycle_id",
 "last_upd_cycle_id" = 20230520000000,
 "crt_job_instance_id" = -1,
 "upd_job_instance_id" = -1,
 "dq_improve_flag" = 'N',
 "last_modified_date" = sysdate() 
WHEN NOT MATCHED THEN INSERT("ae_header_id","ae_line_num","application_code","sr1","sr2","sr3","sr4","sr5","sr6","sr7
              
您的分享是我們最大的動力!

-Advertisement-
Play Games
更多相關文章
  • 我在不要更新挑戰中堅持了一年🎉🎉🎉,你也來試試吧(咕咕咕)! 好言歸正傳,本次更新帶來的是經典游戲掃雷,基於JavaFX實現。篇幅有限,文章主要介紹核心操作實現,不會列出所有代碼。需要完整源碼或是想預覽最終效果,可以點擊下方鏈接。後續會逐步更新細節實現方面的內容,將來吧反正(肯定不鴿!) 視頻 ...
  • ## 背景 隨著所在公司的發展,應用服務的規模不斷擴大,原有的垂直應用架構已無法滿足產品的發展,幾十個工程師在一個項目里並行開發不同的功能,開發效率不斷降低。 於是公司開始全面推進服務化進程,把團隊內的大部分工程師主要精力全部都集中到服務化中。服務化可以讓每個工程師僅在自己負責的子項目中進行開發,提 ...
  • 寫java時不管是我們自己new對象還是spring管理bean,儘管我們天天跟對象打交道,那麼對象的結構和記憶體佈局有多少人知道呢,這篇文章可帶你入門,瞭解java對象記憶體佈局。 本文涉及到JVM指針壓縮的知識點,不熟悉的小伙伴可以看前面寫過的一篇關於指針壓縮的文章。 [JVM之指針壓縮](http ...
  • 記憶體“泄露”是開發中常見的問題之一,它會導致應用程式占用越來越多的記憶體資源,最終可能導致系統性能下降甚至崩潰。軟體開發者需要瞭解在程式中出現記憶體泄露的情況,以避免軟體出現該的問題。 **什麼是記憶體“泄露”?** 記憶體泄露是申請了記憶體空間的變數一直在占用,無法釋放。比如申請了一塊記憶體空間,沒有回收一直 ...
  • 一般來說,單片機的時鐘電路是使用外部的無源晶振和負載電容組合實現連接到單片機的Xin和Xout引腳上,無源晶振自身無法振蕩,因此需要匹配外部諧振電路才可以輸出振動信號。 但是在實際電路設計中,也會在晶振兩端並聯一個電阻。這個電阻叫做反饋電阻。​ 那麼並聯的這個反饋電阻有什麼作用呢? 首先來看下時鐘電 ...
  • Linux下PAM認證詳解(以centos7為例) PAM簡介(Pluggable Authentication Modules,可插拔認證模塊) Sun公司於1995年開發的一種與認證相關的通用框架機制:PAM(可插拔認證模塊)是實現認證工作的一個模塊。 因為每個服務都用到不同的認證方式,所以就需 ...
  • 使用expdp/impdp導出導入數據時,遇到ORA-2000錯誤,如下所示: Processing object type SCHEMA_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANTProcessing object type SCHEMA_EXPORT/ ...
  • 文章中包含我所遇到的錯誤,進行了HDFS錯誤整改,以及後面有操作創建“遠程客戶端操作hdfs創建文件夾”,驗證環境是否配置成功的過程。 ...
一周排行
    -Advertisement-
    Play Games
  • 移動開發(一):使用.NET MAUI開發第一個安卓APP 對於工作多年的C#程式員來說,近來想嘗試開發一款安卓APP,考慮了很久最終選擇使用.NET MAUI這個微軟官方的框架來嘗試體驗開發安卓APP,畢竟是使用Visual Studio開發工具,使用起來也比較的順手,結合微軟官方的教程進行了安卓 ...
  • 前言 QuestPDF 是一個開源 .NET 庫,用於生成 PDF 文檔。使用了C# Fluent API方式可簡化開發、減少錯誤並提高工作效率。利用它可以輕鬆生成 PDF 報告、發票、導出文件等。 項目介紹 QuestPDF 是一個革命性的開源 .NET 庫,它徹底改變了我們生成 PDF 文檔的方 ...
  • 項目地址 項目後端地址: https://github.com/ZyPLJ/ZYTteeHole 項目前端頁面地址: ZyPLJ/TreeHoleVue (github.com) https://github.com/ZyPLJ/TreeHoleVue 目前項目測試訪問地址: http://tree ...
  • 話不多說,直接開乾 一.下載 1.官方鏈接下載: https://www.microsoft.com/zh-cn/sql-server/sql-server-downloads 2.在下載目錄中找到下麵這個小的安裝包 SQL2022-SSEI-Dev.exe,運行開始下載SQL server; 二. ...
  • 前言 隨著物聯網(IoT)技術的迅猛發展,MQTT(消息隊列遙測傳輸)協議憑藉其輕量級和高效性,已成為眾多物聯網應用的首選通信標準。 MQTTnet 作為一個高性能的 .NET 開源庫,為 .NET 平臺上的 MQTT 客戶端與伺服器開發提供了強大的支持。 本文將全面介紹 MQTTnet 的核心功能 ...
  • Serilog支持多種接收器用於日誌存儲,增強器用於添加屬性,LogContext管理動態屬性,支持多種輸出格式包括純文本、JSON及ExpressionTemplate。還提供了自定義格式化選項,適用於不同需求。 ...
  • 目錄簡介獲取 HTML 文檔解析 HTML 文檔測試參考文章 簡介 動態內容網站使用 JavaScript 腳本動態檢索和渲染數據,爬取信息時需要模擬瀏覽器行為,否則獲取到的源碼基本是空的。 本文使用的爬取步驟如下: 使用 Selenium 獲取渲染後的 HTML 文檔 使用 HtmlAgility ...
  • 1.前言 什麼是熱更新 游戲或者軟體更新時,無需重新下載客戶端進行安裝,而是在應用程式啟動的情況下,在內部進行資源或者代碼更新 Unity目前常用熱更新解決方案 HybridCLR,Xlua,ILRuntime等 Unity目前常用資源管理解決方案 AssetBundles,Addressable, ...
  • 本文章主要是在C# ASP.NET Core Web API框架實現向手機發送驗證碼簡訊功能。這裡我選擇是一個互億無線簡訊驗證碼平臺,其實像阿裡雲,騰訊雲上面也可以。 首先我們先去 互億無線 https://www.ihuyi.com/api/sms.html 去註冊一個賬號 註冊完成賬號後,它會送 ...
  • 通過以下方式可以高效,並保證數據同步的可靠性 1.API設計 使用RESTful設計,確保API端點明確,並使用適當的HTTP方法(如POST用於創建,PUT用於更新)。 設計清晰的請求和響應模型,以確保客戶端能夠理解預期格式。 2.數據驗證 在伺服器端進行嚴格的數據驗證,確保接收到的數據符合預期格 ...