From: 011netservice@gmail.com
Date: 2024-03-04, 週1
Subject: CodeHelper\cs\KeyWord\Marshal.txt

#### 摘要
1. Marshaled call 會同時被好幾個 thread 執行!


#### Marshalling ByValTStr as UTF8
Marshalling ByValTStr as UTF8
https://stackoverflow.com/questions/61593410/marshalling-byvaltstr-as-utf8

I am working with a C DLL and have trouble with marshalling strings using P/Invoke.

The DLL has a struct as follows:

typedef struct
{
    char sAddress[256];
    BYTE byUseRtsp;
    WORD wPort;
}INFO,*LPINFO;
My C# struct looks like this:

[StructLayout(LayoutKind.Sequential)]
public struct INFO
{
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
    public string sAddress;

    public byte byUseRtsp;

    public short wPort;
}
The string marshalling for sAddress works with ASCII text, but the DLL uses UTF-8 encoding throughout. So as soon as multi-byte characters are used the marshalling garbles the text. Using CharSet.Unicode doesn't work here as that tells the marshaller to encode/decode strings as UTF-16 on Windows. I need a CharSet.Utf8 which unfortunately doesn't exist.

I do have a workaround, but it's ugly and want to avoid if possible. The workaround is to replace:

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
    public string sAddress;
with:

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 256)]
    public byte[] sAddress;
and then just re-write my code to use Encoding.UTF8.GetBytes/String() methods to get the string values. I will also need to handle null-terminators myself with this method.

Is there a better way of doing this?

answer:
Using the built-in options for marshalling what you have done is probably as good as it gets. You would want to write some helper methods to manage this for you, but I'm sure you are alive to that.

Beyond this you could use a custom marshaler. 
I'm not aware that there is a great body of work available on this topic. 
One of the more complete happens to be this question of mine: 
How do I write a custom marshaler which allows data to flow from native to managed?

https://stackoverflow.com/questions/18498452/how-do-i-write-a-custom-marshaler-which-allows-data-to-flow-from-native-to-manag
How do I write a custom marshaler which allows data to flow from native to managed?

In attempting to write a custom marshaler related to this question (P/Invoke from C to C# without knowing size of array),
 I have come across something I cannot understand. 
 
 This is the first ever custom marshaler that I have written so no doubt I'm missing something obvious due to my ignorance.

Here's my C# code:
using System;
using System.Runtime.InteropServices;
using System.Text;

namespace CustomMarshaler
{
    public class MyCustomMarshaler : ICustomMarshaler
    {
        static MyCustomMarshaler static_instance;

        public IntPtr MarshalManagedToNative(object managedObj)
        {
            if (managedObj == null)
                return IntPtr.Zero;
            if (!(managedObj is int[]))
                throw new MarshalDirectiveException("VariableLengthArrayMarshaler must be used on an int array.");

            int[] arr = (int[])managedObj;
            int size = sizeof(int) + arr.Length * sizeof(int);
            IntPtr pNativeData = Marshal.AllocHGlobal(size);
            Marshal.WriteInt32(pNativeData, arr.Length);
            Marshal.Copy(arr, 0, pNativeData + sizeof(int), arr.Length);
            return pNativeData;
        }

        public object MarshalNativeToManaged(IntPtr pNativeData)
        {
            int len = Marshal.ReadInt32(pNativeData);
            int[] arr = new int[len];
            Marshal.Copy(pNativeData + sizeof(int), arr, 0, len);
            return arr;
        }

        public void CleanUpNativeData(IntPtr pNativeData)
        {
            Marshal.FreeHGlobal(pNativeData);
        }

        public void CleanUpManagedData(object managedObj)
        {
        }

        public int GetNativeDataSize()
        {
            return -1;
        }

        public static ICustomMarshaler GetInstance(string cookie)
        {
            if (static_instance == null)
            {
                return static_instance = new MyCustomMarshaler();
            }
            return static_instance;
        }
    }
    class Program
    {
        [DllImport(@"MyLib.dll")]
        private static extern void Foo(
            [In, Out, MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(MyCustomMarshaler))]
            int[] arr
        );

        static void Main(string[] args)
        {
            int[] colorTable = new int[] { 1, 2, 3, 6, 12 };
            Foo(colorTable);
            foreach (int value in colorTable)
                Console.WriteLine(value);
        }
    }
}

On the other side is a trivial native DLL, written in Delphi as it happens.
library MyLib;

procedure Foo(P: PInteger); stdcall;
var
  i, len: Integer;
begin
  len := P^;
  Writeln(len);
  for i := 1 to len do begin
    inc(P);
    Writeln(P^);
    inc(P^);
  end;
end;

exports
  Foo;

begin
end.


The idea is that the array is passed to the DLL which then prints out the length field, and the values of the array. 
The native code also increments each value of the array by 1.

So, I expect to see this output:
5
1
2
3
6
12
2
3
4
7
13

But unfortunately I see this output:
5
1
2
3
6
12
1
2
3
6
12

Under the debugger I can see that MarshalNativeToManaged is executing, and that the values that it returns have been incremented. 
But these incremented values don't find there way back into the object that is passed to Foo.

What do I need to do to fix this?

answer:
I had a similar problem many years ago and found that there was very little documentation on Custom Marshaling. 
I suspect using ICustomMarshaler never really took off since it can always be done using manual marshaling in the course of your regular code. 
And so there was never really a need for any documentation of advanced custom marshaling scenarios.

Anyway, through a variety of sources and much trial and error I think I teased out a practical understanding of how most of Custom Marshaling works.

In your case, you have set up the ManagedToNative method correctly for [In] marshaling and the NativeToManaged method correctly for most [Out] marshaling but [In, Out] marshaling is actually a bit trickier. 
[In, Out] marshaling is actually in-place marshaling. 
So on the way back out you must marshal the data back to the same instance that was provided in the [In] side of the operation.

There are a number of small variations on this depending on whether using reference or value types, whether the call is a normal pInvoke call or a callback on a delegate, etc. 
But thinking about what needs to end up where is the key.

The following variation on your code works the way you want it to (and it seems to works the same way for .Net 2.0 and up):

        //This must be thread static since, in theory, the marshaled
    //call could be executed simultaneously on two or more threads.
    // [ThreadStatic]: Custom attribute to indicate that the field should be treated as a static relative to a thread.
	
    [ThreadStatic] int[] marshaledObject;
    public IntPtr MarshalManagedToNative(object managedObj)
    {
        if (managedObj == null)
            return IntPtr.Zero;
        if (!(managedObj is int[]))
            throw new MarshalDirectiveException("VariableLengthArrayMarshaler must be used on an int array.");

        //This is called on the way in so we must keep a reference to 
        //the original object so we can marshal to it on the way out.
        marshaledObject = (int[])managedObj;
        int size = sizeof(int) + marshaledObject.Length * sizeof(int);
        IntPtr pNativeData = Marshal.AllocHGlobal(size);
        Marshal.WriteInt32(pNativeData, marshaledObject.Length);
        Marshal.Copy(marshaledObject, 0, (IntPtr)(pNativeData.ToInt64() + sizeof(int)), marshaledObject.Length);
        return pNativeData;
    }

    public object MarshalNativeToManaged(IntPtr pNativeData)
    {
        if (marshaledObject == null)
            throw new MarshalDirectiveException("This marshaler can only be used for in-place ([In. Out]) marshaling.");

        int len = Marshal.ReadInt32(pNativeData);
        if (marshaledObject.Length != len)
            throw new MarshalDirectiveException("The size of the array cannot be changed when using in-place marshaling.");

        Marshal.Copy((IntPtr)(pNativeData.ToInt64() + sizeof(int)), marshaledObject, 0, len);

        //Reset to null for next call;
        marshalledObject = null;

        return marshaledObject;
    }
	
Thanks for this. I'll try it out later. Is there really no alternative to that thread static variable? – 
David Heffernan
 Sep 6, 2013 at 18:23
 
The runtime will call the same instance of your marshaler for every call to the pInvoked method. 
So you can drop the ThreadStatic attribute only if you can guarantee that the pInvoked method(s) that use the Custom Marshaler will never be called simultaneously on multiple threads. 
Either through another type of synchronization or just because of the nature of the code. – 
Stephen Martin
 Sep 6, 2013 at 18:34
 






#### 參考 CodeHelper\cs\KeyWord\Marshal 資料夾 有這兩個 sample:
https://learn.microsoft.com/en-us/dotnet/framework/interop/marshalling-strings
Marshalling Strings

https://learn.microsoft.com/en-us/dotnet/standard/native-interop/customize-struct-marshalling
Customize structure marshalling

#### Marshaling pointer to an array of strings
https://stackoverflow.com/questions/1323797/marshaling-pointer-to-an-array-of-strings
Marshaling pointer to an array of strings

[StructLayout(LayoutKind.Sequential)] 
public struct UnmanagedStruct 
{ 
    [MarshalAs(UnmanagedType.ByValArray, SizeConst=100)] 
    public IntPtr[] listOfStrings; 
}

for (int i = 0; i < 100; ++i)
{
    if (listOfstrings[i] != IntPtr.Zero)
        Console.WriteLine(Marshal.PtrToStringAnsi(listOfStrings[i]));
}    


#### 提示: 轉成 ASCII 再 Array.Copy 進 Marshal 欄位 byte[] 或 char[]
https://stackoverflow.com/questions/9663265/marshalling-exception-with-char-arrays
Marshalling exception with char arrays
Question:
I have some issues with marshalling script giving an exception. 
I have C++ structs that I try to mimic in C# for testing a system we have. 
The C++ struct looks like this:

#pragma pack(1)
typedef struct
{
  ACE_UINT32 result;
  ACE_UINT32 command;
  ACE_TCHAR information[2001];
  ACE_UINT16 informationLength; ///< Length of the variable information.
} MsgStructType;
#pragma pack()

in C# I declare the struct as follows:
[StructLayout(LayoutKind.Explicit, Pack = 1)]
struct MsgStruct
{
  [FieldOffset(0)]
  public uint result;
  [FieldOffset(4)]
  public uint command;
  [FieldOffset(8)]
  public Byte[] information;
  [FieldOffset(2009)]
  public ushort informationLength;
}

I use the following methods to serialize and deserialize the message.
public static T DeserializeMsg<T>(Byte[] data) where T : struct
{
  int objsize = Marshal.SizeOf(typeof(T));
  IntPtr buff = Marshal.AllocHGlobal(objsize);
  Marshal.Copy(data, 0, buff, objsize);
  T retStruct = (T)Marshal.PtrToStructure(buff, typeof(T));
  Marshal.FreeHGlobal(buff);
  return retStruct;
}

public static Byte[] SerializeMessage<T>(T msg) where T : struct
{
  int objsize = Marshal.SizeOf(typeof(T));
  Byte[] ret = new Byte[objsize];
  IntPtr buff = Marshal.AllocHGlobal(objsize);
  Marshal.StructureToPtr(msg, buff, true);
  Marshal.Copy(buff, ret, 0, objsize);
  Marshal.FreeHGlobal(buff);
  return ret;
}

I manage to serialize the message, send it on udp to the same application as received and the size of data seems to be correct. 
Problem I get is when I try to Deserialize the message. 
I get the following error code:
  ... System.Runtime.Interop.SafeArray.TypeMismatchException...

Perhaps the method to use Byte[] is incomplete but it is the exact same class I use for serializeing and unserializeing the data. 
Only difference is that I go throught udp between.

Some trial and error made me come to the insight that the definition of byte[] or char[] seems to be the issue.

[StructLayout(LayoutKind.Explicit, Pack = 1)]
struct MsgStruct
{
  [FieldOffset(0)]
  public uint result;
  [FieldOffset(4)]
  public uint command;
  [FieldOffset(8)]
//  public Byte[] information;
//  [FieldOffset(2009)]
  public ushort informationLength;
}
This one can be transferred without problems between the systems. 
So I guess its the byte / char array I need to work on to declare correct.


answer:
If you write just public Byte[] information;, 
it's marshaled as if it was a pointer to an array, but that's not what you have here. 
What you need to do is to specify [MarshalAs(UnmanagedType.ByValArray)], 
so that it's interpreted as an array that's directly in the structure:

[FieldOffset(8)]
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 2001)]
public byte[] information;

And if you do that, I think you won't need LayoutKind.Explicit anymore:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct MsgStruct
{
    public uint result;
    public uint command;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 2001)]
    public byte[] information;
    public ushort informationLength;
}


EDIT: The information array has to be exactly the declared length, even you want to put in less than that much bytes. 
So, with your example, it could look like this:

var msg = new MsgStruct();
msg.information = new byte[2001];
var information = Encoding.ASCII.GetBytes("Kalle"); // 轉成 ASCII 再 Array.Copy 進 Marshal 欄位 byte[] 或 char[]
Array.Copy(information, msg.information, information.Length);
var bytes = SerializeMessage(msg);