From: 011netservice@gmail.com
Date: 2023-07-19
Subject: MemoryMappedFile.txt

章節
	#### IPC, inter-process communication
	#### Sample1
	#### 關於 Shared Memory 的兩三事

歡迎來信交流, 訂購軟體需求.

以下 #### 標記段落, **** 標記常用(流程、設定、備忘)

#### IPC, inter-process communication
https://www.c-sharpcorner.com/article/inter-process-communication/

Introduction
 
In general, inter-process communication is a mechanism that allows processes to communicate with each other. The IPC methods vary depending on the operating systems we rely on.
 
There are two ways where a process can communicate,
	1. Shared memory
	2. Message passing 

IPC Mechanisms in windows
	1. Clipboard 
	2. Data copy
	3. DDE 
	4. File mapping 
	5. Mail slots 
	6. Pipes 
	7. RPC 
	8. Windows sockets

Selecting the correct IPC that matches your need depends on a few of the following questions,
	1. Should the application be able to communicate with other applications running on other computers on a network, or is it sufficient for the application to communicate only with applications on the local computer?
	2. Should the application be able to communicate with applications running on other computers that may be running under different operating systems?
	3. Should the application communicate with many different applications in a general way, such as allowing cut-and-paste operations with any other application, or should its communications requirements be limited to a restricted set of interactions with specific other applications?
	4. Is performance a critical aspect of the application? All IPC mechanisms include some amount of overhead.
	5. Should the application be a GUI application or a console application? Some IPC mechanisms require a GUI application.


Key Findings

A. Clipboard
	1. Both the applications should support the clipboard for those data formats that they understand.
	2. For example, a text editor or word processor should at least be able to produce and accept clipboard data in pure text format.

B. Data copy
	1. Data copy enables an application to send information to another application using the wm_copydata message.
	2. The data sent is read-only for the receiving application. It works only for the processes running on the same computer.

C. DDE
	1. It sends messages between applications that share data and uses shared memory to exchange data between applications.
	2. Applications can use the DDE protocol for one-time data transfers and for continuous exchanges.
	3. DDE is not as efficient as newer technologies. 
	However, you can still use DDE if other IPC mechanisms are not suitable or if you must interface with an existing application that only supports DDE.

D. File mapping
	1. Allows one-way and bi-directional communication between applications. It uses shared memory for data sharing.
	2. File mapping can be used only between processes on a local computer; it cannot be used over a network.
	3. Mail slots:
	4. Mail slots provide one-way communication. 
	   A process can be both a mail slot server and a mail slot client, so two-way communication is possible using multiple mail slots.
	5. Mail slots offer an easy way for applications to send and receive short messages. 
	They also provide the ability to broadcast messages across all computers in a network domain.

E. Pipes, ***
	1. Allows two-way communication. 
	   Two types of pipes are available. Named and anonymous pipes.
	2. Named pipes are used to transfer data between unrelated process where anonymous is used between processes on a same computer.
    https://columns.chicken-house.net/2019/06/15/netcli-pipeline/
    https://dotblogs.com.tw/supershowwei/2019/06/03/104149

F. RPC, ****
	1. RPC enables applications to call functions remotely. 
	   RPC operates between processes on a single computer or on different computers on a network.
	2. Applications that use RPC are able to communicate with applications running with other operating systems that support DCE (Distributed Computer Environment).

G. Windows Sockets
	1. Windows Sockets (Winsock) enables programmers to create advanced Internet, intranet, 
	   and other network-capable applications to transmit application data across the wire, 
	   independent of the network protocol being used.
	2. For example: peer to peer such as chat applications / client-server models.

Conclusion
 
The selection of an IPC completely depends on the process specifications. In general, shared memory will perform faster than the message passing because in shared memory it relies only a single time on the kernel for memory creation but in message passing It is time consuming as it is implemented through kernel intervention.
Each has its own pros and cons like shared memory can be used for local process and message passing can be used for remote process, etc.
 
Sharing the detailed benchmark of different IPC’s - IPC Benchmarks
https://github.com/LokeshVarman/ipc-bench

#### Sample1
https://learn.microsoft.com/en-us/dotnet/api/system.io.memorymappedfiles.memorymappedfile.createnew?view=net-6.0&redirectedfrom=MSDN#System_IO_MemoryMappedFiles_MemoryMappedFile_CreateNew_System_String_System_Int64_

The following example is composed of three separate processes (console applications) that write Boolean values to a memory-mapped file. The following sequence of actions occur:

Process A creates the memory-mapped file and writes a value to it.

Process B opens the memory-mapped file and writes a value to it.

Process C opens the memory-mapped file and writes a value to it.

Process A reads and displays the values from the memory-mapped file.

After Process A is finished with the memory-mapped file, the file is immediately reclaimed by garbage collection.

To run this example, do the following:

Compile the applications and open three Command windows.

In the first Command window, run Process A.

In the second Command window, run Process B.

Return to Process A and press ENTER.

In the third Command window, run Process C.

Return to Process A and press ENTER.

The output of Process A is as follows:
Start Process B and press ENTER to continue.  
Start Process C and press ENTER to continue.  
Process A says: True  
Process B says: False  
Process C says: True


Process A
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process A:
    static void Main(string[] args)
    {
        using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 10000))
        {
            bool mutexCreated;
            Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
            using (MemoryMappedViewStream stream = mmf.CreateViewStream())
            {
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(1);
            }
            mutex.ReleaseMutex();

            Console.WriteLine("Start Process B and press ENTER to continue.");
            Console.ReadLine();

            Console.WriteLine("Start Process C and press ENTER to continue.");
            Console.ReadLine();

            mutex.WaitOne();
            using (MemoryMappedViewStream stream = mmf.CreateViewStream())
            {
                BinaryReader reader = new BinaryReader(stream);
                Console.WriteLine("Process A says: {0}", reader.ReadBoolean());
                Console.WriteLine("Process B says: {0}", reader.ReadBoolean());
                Console.WriteLine("Process C says: {0}", reader.ReadBoolean());
            }
            mutex.ReleaseMutex();
        }
    }
}

Process B
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process B:
    static void Main(string[] args)
    {
        try
        {
            using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
            {

                Mutex mutex = Mutex.OpenExisting("testmapmutex");
                mutex.WaitOne();

                using (MemoryMappedViewStream stream = mmf.CreateViewStream(1, 0))
                {
                    BinaryWriter writer = new BinaryWriter(stream);
                    writer.Write(0);
                }
                mutex.ReleaseMutex();
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first.");
        }
    }
}

Process C
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;

class Program
{
    // Process C:
    static void Main(string[] args)
    {
        try
        {
            using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
            {

                Mutex mutex = Mutex.OpenExisting("testmapmutex");
                mutex.WaitOne();

                using (MemoryMappedViewStream stream = mmf.CreateViewStream(2, 0))
                {
                    BinaryWriter writer = new BinaryWriter(stream);
                    writer.Write(1);
                }
                mutex.ReleaseMutex();
            }
        }
        catch (FileNotFoundException)
        {
            Console.WriteLine("Memory-mapped file does not exist. Run Process A first, then B.");
        }
    }
}

#### 關於 Shared Memory 的兩三事
https://blog.darkthread.net/blog/about-shared-memory/

參與古老系統的搬遷工程，其中使用 Shared Memory 實現跨 Process 溝通（例如：ASP.NET 呼叫 Window Service），也因而被迫了解這門對 .NET 開發者偏冷門的技術，特筆記備忘。

【Shared Memory 是什麼?】
跨 Process 溝通有個術語，Interprocess Communictaion（IPC），在 Windows 平台有以下選擇：參考

Clipboard
程式 A 將内容貼進剪貼簿，程式 B 自剪貼簿取出内容。
COM
OLE 複合文件（Compound Document）讓 Word 文件可以內嵌 Excel 工作表，點兩下還能叫出 Excel 進行編輯， OLE 的基礎為 COM 元件技術。
Data Copy
程序 A 向程式 B 依約定的格式内容傳送 WM_COPYDATA 訊息
DDE
DDE 是一種允許不同應用程式交換不同格式資料的通訊協定，可視為剪貼簿的沿伸，除了一次性抛轉，還能持續傳輸資料。（效能相對差，已不建議使用）
File Mapping
File Mapping 意指將檔案模擬成 Process 中的一塊記憶體，當多個應用程式間透過共用 File Mapping 交換資料，稱之為 Named Shared Memory，在各種 IPC 方法中效能最佳，但必須透過 Mutex 等同步機制防止讀寫衝突。
Mailslots
單向溝通，Mailslot Client 送訊息給 Mailslot Server，訊息在 Server 讀取後删除，支援跨機器傳送，還可一對多廣播。（廣播訊息長度限制 400 bytes，一對一傳輸時訊息長度則由 Mailslot Server 建立時決定）
Pipes
雙向傳輸，分為 Anonymous Pipe 及 Named Pipe。Anonymous Pipe 一般用於父程序與子程序間的標準輸入/輸出導向，雙向溝通要建兩條 Pipe，不能跨網路且限於有從屬關係的 Process；Named Pipe 則可用於任意 Process 間交換資料，並支援跨網路 Process 間傳輸。
RPC
Remote Procedure Call(RPC) 允許應用程式呼叫其他應用程式提供的函式功能，並可跨網路呼叫。Windows RPC 符合 ISO DCE 標準，支援跨作業系統系統整合。
Windows Sockets
基於 TCP/IP 或其他網路協定制訂的抽象通訊介面，底層透過網路連線進行資料交換。
Shared Memory 是 C/C++ 開發者常用的資料交換方式（ Google 可以查到很多在 Linux 用 Shared Memory 實現 IPC 的範例），故 C/C++ 開發者在 Windows 平台也常選擇它做為溝通管道。

【Shared Memory 實作練習】
雖然用的人較少，但 .NET 內建 System.IO.MemoryMappedFiles 命名空間，要玩 Shared Memory 不是難事，幾乎跟操作檔案沒什麼兩樣，只要有 FileStream 相關操作經驗很快就上手，參考 MSDN 範例，我寫了小程式練習。

我寫了兩隻程式，ProcessA 透過 MemoryMappedFile.CreateNew() 建立大小為 1024 Bytes 的空間，與另一隻 ProcessB 練習傳接球。由於 1024 Bytes 兩隻程式共用，我將前 512 規劃為 ProcessA 寫入 ProcessB 讀取，後 512 則是 ProcessB 寫 ProcessA 讀，程式中使用 CreateViewStream 傳入起始位址及長度指向自己專屬的區域。為了避免 ProcessA 及 ProcessB 存取 MemoryMappedFile 時出現讀寫衝突，我使用 Mutex 鎖定控管單一時間只有一個 Process 可以存取 MemoryMappedFile。測試過程為 ProcessA 建立 MemoryMappedFile，寫入訊息字串 –> ProcessB 讀取訊息字串並寫入回應字串 –> ProcessA 讀取回應字串，結束。

ProcessA 程式如下：
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Linq;
using System.Text;
using System.Threading;
 
namespace ProcessA
{
    class Program
    {
        static void Main(string[] args)
        {
            //REF: https://msdn.microsoft.com/en-us/library/dd267552(v=vs.110).aspx
            using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("DARKTHREAD", 1024))
            {
                bool mutexCreated;
                Mutex mutex = new Mutex(true, "DarkthreadSharedMem", out mutexCreated);
                using (var stream = mmf.CreateViewStream()) {
                    byte[] msg = Encoding.UTF8.GetBytes("Hello, World!");
                    using (BinaryWriter bw = new BinaryWriter(stream))
                    {
                        bw.Write(msg.Length); //先寫Length
                        bw.Write(msg); //再寫byte[]
                    }
                }
                mutex.ReleaseMutex();
                Console.Write("操作 Process B 進行讀取及回應，完成後按Enter");
                Console.ReadLine();
 
                mutex.WaitOne();
                using (MemoryMappedViewStream stream = mmf.CreateViewStream(512, 512))
                {
                    using (var br = new BinaryReader(stream))
                    {
                        //先讀取長度，再讀取内容
                        var len = br.ReadInt32();
                        var msg = Encoding.UTF8.GetString(br.ReadBytes(len), 0, len);
                        Console.WriteLine($"回應＝{msg}");
                    }
                }
                mutex.ReleaseMutex();
                Console.ReadLine();
            }
        }
    }
}

Process B
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Linq;
using System.Text;
using System.Threading;
 
namespace ProcessB
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                Console.Write("按 Enter 開始讀取及回應…");
                Console.ReadLine();
                using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("DARKTHREAD"))
                {
                    Mutex mutex = Mutex.OpenExisting("DarkthreadSharedMem");
                    mutex.WaitOne();                    
                    using (MemoryMappedViewStream stream = mmf.CreateViewStream(0, 0))
                    {
                        using (var br = new BinaryReader(stream))
                        {
                            //先讀取長度，再讀取内容
                            var len = br.ReadInt32();
                            var word = Encoding.UTF8.GetString(br.ReadBytes(len), 0, len);
                            Console.WriteLine($"訊息＝{word}");
                        }
                    }
                    using (MemoryMappedViewStream stream = mmf.CreateViewStream(512, 512))
                    {
                        using (var bw = new BinaryWriter(stream))
                        {
                            var msg = Encoding.UTF8.GetBytes("朕知道了");
                            bw.Write(msg.Length);
                            bw.Write(msg);
                        }
                    }
                    mutex.ReleaseMutex();
                }
                Console.ReadLine();
            }
            catch (FileNotFoundException)
            {
                Console.WriteLine("Memory-mapped file does not exist.");
            }
        }
    }
}


【補充技巧】
1. 如何檢視 Windows 目前已開啟的 MemoryMappedFile?
SystemInternals 有個 AccessChk 工具能列出 Windows 所有可存取的檔案、資料夾、Registry、物件以及 Windows 服務。而 MemoryMappedFile 屬於一種 Windows 物件，使用以下指令可列出所有物件並存檔
accesschk -osv > e:\objList.txt
在其中尋找 MemoryMappedFile 名稱，若存在可看到類似以下記錄：
\Sessions\1\BaseNamedObjects\DARKTHREAD
  Type: Section
  Medium Mandatory Level (Default) [No-Write-Up]
  RW NT AUTHORITY\SYSTEM
    SECTION_ALL_ACCESS
  RW DOMAIN\UserName
    SECTION_ALL_ACCESS
  RW DOMAIN\UserName-S-1-5-5-0-954410
    SECTION_ALL_ACCESS

2. MemoryMappedFile 預設是開在使用者的 Session 中，預設無法跨 Session 使用。
例如：兩個分屬不同 AppPool 的 ASP.NET 若執行身分不同，即使 MemoryMappedFile 名稱相同也是各自一份，故運用時需確認溝通雙方使用的執行身分相同。

3. 若要跨不同執行身分溝通，MemoryMappedFile 可命名為 "Global\Filename"（注意 Global 大小寫有別，我踩到誤寫為GLOBAL 路徑無效的雷），如此可跨執行身分存取。
但需要注意，Session 0 （Windows Service）以外的 Process 需要具有 SeCreateGlobalPrivilege 權限才能建立 Global\… MemoryMappedFile。（MSDN文件）
關於 Session 0，可參考對岸 MVP 的這篇文章 - 穿透Session 0 隔离（一） 裡面有蠻詳細的介紹。