對於數值模擬而言,無論是商軟或者開源軟體,並行計算都是非常重要的, 作為一名模擬工程師,如果想把自身數值模擬能力提升一個層次,需要對並行計算有很好的理解與應用 openfoam並行通信主要通過Pstream類完成 Pstream類,類如其名,parallel_stream,並行計算時使用的信息流 O ...
對於數值模擬而言,無論是商軟或者開源軟體,並行計算都是非常重要的,
作為一名模擬工程師,如果想把自身數值模擬能力提升一個層次,需要對並行計算有很好的理解與應用
openfoam並行通信主要通過Pstream類完成
Pstream類,類如其名,parallel_stream,並行計算時使用的信息流
Openfoam對其的介紹是:
Inter-processor communications stream.
處理器間交換信息流
類似的命名方法我們在c++文件讀取時說過,std有fstream類讀取寫入文件/二進位文件,比如說我們要讀取文件,會把讀取內容放入緩存區內進行操作
#include <iostream>
#include <fstream> // ifstream類需要包含的頭文件。
#include <string> // getline()函數需要包含的頭文件。
using namespace std;
int main()
{
string filename = R"(./test.txt)";
//ifstream fin(filename, ios::in);
ifstream fin;
fin.open(filename , ios::in);
// 判斷打開文件是否成功。
// 失敗的原因主要有:1)目錄不存在;2)文件不存在;3)沒有許可權,Linux平臺下很常見。
if (fin.is_open() == false)
{
cout << "打開文件" << filename << "失敗。\n"; return 0;
}
string buffer;
while (fin >> buffer)
{
cout << buffer << endl;
}
fin.close(); // 關閉文件,fin對象失效前會自動調用close()。
cout << "操作文件完成。\n";
}
類似的openfoam也有PstreamBuffers類進行並行通信緩衝
可以這樣使用:
PstreamBuffers pBuffers(Pstream::commsTypes::nonBlocking);
for (label proci = 0; proci < Pstream::nProcs(); proci++)
{
if (proci != Pstream::myProcNo())
{
someObject vals;
UOPstream str(proci, pBuffers);
str << vals;
}
}
pBuffers.finishedSends(); // no-op for blocking
for (label proci = 0; proci < Pstream::nProcs(); proci++)
{
if (proci != Pstream::myProcNo())
{
UIPstream str(proci, pBuffers);
someObject vals(str);
}
}
上面這個程式可以看到,先後使用UOPstream與UIPstream進行緩衝區的文件輸出與讀取,這就很像ofstream類與ifstream類,甚至命名方式上都有幾分相似,我們打開相應的繼承關係圖
二者分別服務於IPstream類與OPstream類,我們再打開今天文章的主角,Pstream類繼承關係圖
發現IPstream類與OPstream類是Pstream類的衍生類,Pstream類是其基礎
打開Pstream類的源碼:
點擊查看代碼
namespace Foam
{
/*---------------------------------------------------------------------------*\
Class Pstream Declaration
\*---------------------------------------------------------------------------*/
class Pstream
:
public UPstream
{
protected:
// Protected data
//- Transfer buffer
DynamicList<char> buf_;
public:
// Declare name of the class and its debug switch
ClassName("Pstream");
// Constructors
//- Construct given optional buffer size
Pstream
(
const commsTypes commsType,
const label bufSize = 0
)
:
UPstream(commsType),
buf_(0)
{
if (bufSize)
{
buf_.setCapacity(bufSize + 2*sizeof(scalar) + 1);
}
}
// Gather and scatter
//- Gather data. Apply bop to combine Value
// from different processors
template<class T, class BinaryOp>
static void gather
(
const List<commsStruct>& comms,
T& Value,
const BinaryOp& bop,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T, class BinaryOp>
static void gather
(
T& Value,
const BinaryOp& bop,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
//- Scatter data. Distribute without modification. Reverse of gather
template<class T>
static void scatter
(
const List<commsStruct>& comms,
T& Value,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T>
static void scatter
(
T& Value,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
// Combine variants. Inplace combine values from processors.
// (Uses construct from Istream instead of <<)
template<class T, class CombineOp>
static void combineGather
(
const List<commsStruct>& comms,
T& Value,
const CombineOp& cop,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T, class CombineOp>
static void combineGather
(
T& Value,
const CombineOp& cop,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
//- Scatter data. Reverse of combineGather
template<class T>
static void combineScatter
(
const List<commsStruct>& comms,
T& Value,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T>
static void combineScatter
(
T& Value,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
// Combine variants working on whole List at a time.
template<class T, class CombineOp>
static void listCombineGather
(
const List<commsStruct>& comms,
List<T>& Value,
const CombineOp& cop,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T, class CombineOp>
static void listCombineGather
(
List<T>& Value,
const CombineOp& cop,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
//- Scatter data. Reverse of combineGather
template<class T>
static void listCombineScatter
(
const List<commsStruct>& comms,
List<T>& Value,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T>
static void listCombineScatter
(
List<T>& Value,
const int tag = Pstream::msgType(),
const label comm = Pstream::worldComm
);
// Combine variants working on whole map at a time. Container needs to
// have iterators and find() defined.
template<class Container, class CombineOp>
static void mapCombineGather
(
const List<commsStruct>& comms,
Container& Values,
const CombineOp& cop,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class Container, class CombineOp>
static void mapCombineGather
(
Container& Values,
const CombineOp& cop,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
//- Scatter data. Reverse of combineGather
template<class Container>
static void mapCombineScatter
(
const List<commsStruct>& comms,
Container& Values,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class Container>
static void mapCombineScatter
(
Container& Values,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
// Gather/scatter keeping the individual processor data separate.
// Values is a List of size UPstream::nProcs() where
// Values[UPstream::myProcNo()] is the data for the current processor.
//- Gather data but keep individual values separate
template<class T>
static void gatherList
(
const List<commsStruct>& comms,
List<T>& Values,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T>
static void gatherList
(
List<T>& Values,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
//- Scatter data. Reverse of gatherList
template<class T>
static void scatterList
(
const List<commsStruct>& comms,
List<T>& Values,
const int tag,
const label comm
);
//- Like above but switches between linear/tree communication
template<class T>
static void scatterList
(
List<T>& Values,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
// Exchange
//- Helper: exchange contiguous data. Sends sendData, receives into
// recvData. If block=true will wait for all transfers to finish.
template<class Container, class T>
static void exchange
(
const UList<Container>& sendData,
const labelUList& recvSizes,
List<Container>& recvData,
const int tag = UPstream::msgType(),
const label comm = UPstream::worldComm,
const bool block = true
);
//- Helper: exchange sizes of sendData. sendData is the data per
// processor (in the communicator). Returns sizes of sendData
// on the sending processor.
template<class Container>
static void exchangeSizes
(
const Container& sendData,
labelList& sizes,
const label comm = UPstream::worldComm
);
//- Exchange contiguous data. Sends sendData, receives into
// recvData. Determines sizes to receive.
// If block=true will wait for all transfers to finish.
template<class Container, class T>
static void exchange
(
const UList<Container>& sendData,
List<Container>& recvData,
const int tag = UPstream::msgType(),
const label comm = UPstream::worldComm,
const bool block = true
);
};
我們看到Pstream類有一個構造函數,剩下的都是靜態成員函數,而這些成員函數就是並行通訊的工具箱
這裡多問一句,為什麼工具箱的函數都是靜態成員函數
為什麼這裡用靜態成員函數呢
用靜態成員可以變數實現多個對象間的數據共用,比全局變數更安全
這裡我詳細說下,舉個例子
Time mytime1;
mytime1.hour=2;
Time mytime2;
mytime2.hour=4;
這段程式中成員變數是跟著對象走的,他們的對象各自占用不同的記憶體地址,彼此互不影響
那我們想做類內的全局變數滿足相互通信需求,在不同對象mytime1和mytime2中共用一個副本,怎麼辦
這時static關鍵字就派上用場了,增加了static關鍵字或成員函數不隸屬整個對象,而隸屬於整個類
因為這個變數跟著類走,所以調用時用“類名::成員變數名”或“類名::成員變數函數”進行調用(當然也可用“對象名.靜態函數名”),表示明確的隸屬關係,不創建對象也可進行訪問編輯
在Pstream類調用工具箱中函數時,我們常見到這樣的調用方式,而且不創建Pstream對象也可進行調用
// 在head節點收集信息
Pstream::gatherList(nInternalFaces);
Pstream::gatherList(nBoundaries);
因為類的靜態成員脫離了與對象的關係,普通成員變數的記憶體分配是在對象初始化時完成的,對於靜態成員必須在程式的全局區進行清晰的初始化
全局區的初始化過程可由某個.cpp源文件的開頭的靜態成員函數完成,如下所示:
void Time::func(int testValue)
{
mystatic = testValue ;
}
或者在全局區這樣寫:
int Time::mystatic=10;
這樣能保證這個靜態成員變數能夠被正常使用。
此外靜態成員函數只能調用靜態成員變數,也沒有this指針可以使用
這裡上一張圖可能更方便理解
C++程式運行時,靜態變數和全局變數存儲在數據段,所以需要在全局區通過直接分配記憶體或者靜態函數進行分配記憶體
因而靜態成員的生命周期與程式運行周期相同,在程式中只有一份,無論創建對象與否,或者創建多少對象
說到這裡可能大家對Openfoam的並行通信多了一些理解,只要開始了並行計算那麼就可以通過Pstream類內的成員函數進行通信調用,在同樣的數據段副本上進行信息流溝通
接下來依次說下類中各個工具的使用
收發數據
Pstream::gather()與Pstream::scatter()分別有兩個重載,分別是收集以及散佈數據,不如後面Pstream::gatherList()與Pstream::scatterList()常用,這裡不細說了
Pstream::combineGather()、Pstream::combineScatter()重載情況與上同,用於就地集中收集或散佈的數據,不太常用
Pstream::listCombineGather()、Pstream::listCombineScatter()重載情況與上同,用於一次整合list容器中的變數
Pstream::mapCombineGather()、Pstream::mapCombineScatter()重載情況與上同,用於一次整合整個map容器中的變數
Pstream::gatherList()以及Pstream::scatterList()的第二個重載比較常用,
template<class T>
static void gatherList
(
List<T>& Values,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
template<class T>
static void scatterList
(
List<T>& Values,
const int tag = Pstream::msgType(),
const label comm = UPstream::worldComm
);
Pstream::gatherList()以及Pstream::scatterList()的輸入第一個參數是Values
這個Values需要自己整合下,Values是UPstream::nProcs()數量大小的List
List<label> nIternalFaces(Pstream::nProcs());
nIternalFaces[Pstream::myProcNo()] = mesh.Cf().size();//比如說看看每個節點分到了多少網格
Pstream::gatherList(nIternalFaces);//在頭結點收集數據
Pstream::scatterList()與之類似
Pstream::gatherList()以及Pstream::scatterList()的輸入第二個參數是Pstream::msgType(),預設為1,可以不輸入
int Foam::UPstream::msgType_(1);
Pstream::gatherList()以及Pstream::scatterList()的輸入第三個參數是Pstream::msgType(),預設為0,可以不輸入
Foam::label Foam::UPstream::worldComm(0);
交換數據
Pstream::exchange()有兩個重載,用於交換連續的數據,一般情況下等待其他所有傳輸完成再傳輸,可通過預設參數block()修改優先權
Pstream::exchangeSizes()用於交換數據的大小
下麵是Pstream類函數相互關係
結語
並行開發遠不止收發數據這麼簡單,還有很多類可說的,後續會一一進行介紹,並對openfoam並行計算進行優化
一起探索openfoam也是相當有趣的一件事,非常歡迎私信討論
指正的價值要比打賞更重要,下麵是個人聯繫方式,能結交到志同道合的朋友是我的榮幸