Enable marshal methods support by default (#7351)

Context: 5271f3e
Context: e1af958
Context: 186a9fc
Context: 903ba37
Context: a760281
Context: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration

Complete the LLVM Marshal Methods effort sketched out in e1af958.

LLVM Marshal Methods are only supported in .NET Android, *not*
Xamarin.Android.

A *Marshal Method* is a JNI Callable C function (pointer) which has
[parameter types and return types which comply with the JNI ABI][0].
[`generator`][1] emits marshal methods as part of the binding, which
are turned into Delegate instances at runtime as part of
[Java Type Registration][2].

*LLVM Marshal Methods* turn this runtime operation -- looking up
`generator`-emitted marshal methods and registering those methods
with Java -- into a *build-time* operation, using LLVM-IR to generate
[JNI Native Method Names][3] which will then be contained within
`libxamarin-app.so`.  LLVM Marshal Methods will also *remove* the
previous Reflection-based infrastructure from relevant types.

LLVM Marshal Methods are *enabled by default* for ***Release***
configuration builds in .NET 8, and disabled by default for Debug
builds. The new `$(AndroidEnableMarshalMethods)` MSBuild property
explicitly controls whether or not LLVM Marshal Methods are used.

LLVM Marshal Methods are *not* available in Classic Xamarin.Android.


~~ Build Phase: Scanning for Compatible Types ~~

During the application build, all `Java.Lang.Object` and
`Java.Lang.Throwable` subclasses are scanned as part of
[Java Callable Wrapper generation][4], looking for "un-bound"
(user-written) types which override `abstract` or `virtual`
methods, or implement interface members.  This is done to emit
Java Callable Wrappers, Java code which "mirrors" the C# code with
an appropriate base class, interface implementation list, and
Java `native` method declarations for "virtual" member overrides.

This scanning process is updated for LLVM Marshal Methods to classify
each type to see if it requires the legacy Delegate-based
registration mechanism, as constructs such as
`[Java.Interop.ExportAttribute]` cannot (yet) be used with
LLVM Marshal Methods.


~~ Build Phase: Java Callable Wrapper Generation ~~

For example, given the C# type:

    // C#
    public partial class MainActivity : Activity {
        protected override void OnCreate (Bundle? state) => …
    }

Then the resulting Java Callable Wrapper *without* LLVM Marshal
Methods enabled will be:

    // Java + No LLVM Marshal Methods
    public /* partial */ class MainActivity extends Activity {
        static {
            String __md_methods =
                "n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler\n";
            mono.android.Runtime.register ("Example.MainActivity, ExampleAssembly", MainActivity.class, __md_methods);
        }
        public void onCreate (android.os.Bundle p0) {n_onCreate(p0);}
        private native void n_onCreate (android.os.Bundle p0);
    }

When LLVM Marshal Methods are enabled, the Java Callable Wrapper
has no static constructor, nor any call to `Runtime.register()`.


~~ Build Phase: Marshal Method Wrapper ~~

Consider the binding infrastructure code that `generator` emits for
`Android.App.Activity.OnCreate()`:

	namespace Android.App {
	    public partial class Activity {
	        static Delegate? cb_onCreate_Landroid_os_Bundle_;
	#pragma warning disable 0169
	        static Delegate GetOnCreate_Landroid_os_Bundle_Handler ()
	        {
	            if (cb_onCreate_Landroid_os_Bundle_ == null)
	                cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_);
	            return cb_onCreate_Landroid_os_Bundle_;
	        }

	        static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
	        {
	            var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
	            var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer);
	            __this.OnCreate (savedInstanceState);
	        }
	#pragma warning restore 0169

	        [Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")]
	        protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
	        {
	            const string __id = "onCreate.(Landroid/os/Bundle;)V";
	            try {
	                JniArgumentValue* __args = stackalloc JniArgumentValue [1];
	                __args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
	                _members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
	            } finally {
	                global::System.GC.KeepAlive (savedInstanceState);
	            }
	        }
	    }
	}

When LLVM Marshal Methods are enabled, the following IL
transformations are performed:

  * The `static Delegate? cb_…` field is removed.
  * The `static Delegate Get…Handler()` method is removed.
  * A new `static … n_…_mm_wrapper()` method is added.

The `n_…_mm_wrapper()` method is responsible for exception marshaling
and for `bool` marshaling.  The `n_…_mm_wrapper()` method has the
[`UnmanagedCallersOnlyAttribute`][5], and works by calling the
existing `n_…()` method:

	namespace Android.App {
	    public partial class Activity {

	        // Added
	        [UnmanagedCallersOnly]
	        static void n_OnCreate_Landroid_os_Bundle__mm_wrapper (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
	        {
	            try {
	                n_OnCreate_Landroid_os_Bundle_ (jnienv, native__this, native_savedInstanceState);
	            }
	            catch (Exception __e) {
	                Android.Runtime.AndroidEnvironmentInternal.UnhandledException (__e);
	            }
	        }
	    }
	}


~~ Build Phase: LLVM-IR Marshal Method Generation ~~

For each Java `native` method declaration contained in Java Callable
Wrappers which support LLVM Marshal Methods, LLVM-IR is used to
generate the JNI Native Method with the `Java_…` symbol name:

	using android_app_activity_on_create_bundle_fn = void (*) (JNIEnv *env, jclass klass, jobject savedInstanceState);
	static android_app_activity_on_create_bundle_fn android_app_activity_on_create_bundle = nullptr;

	extern "C" JNIEXPORT void
	JNICALL Java_helloandroid_MainActivity_n_1onCreate__Landroid_os_Bundle_2 (JNIEnv *env, jclass klass, jobject savedInstanceState) noexcept
	{
	  if (android_app_activity_on_create_bundle == nullptr) {
	    get_function_pointer (
	        16,                                                               // mono image index; computed at build time
	        0,                                                                // class index; computed at build time
	        0x0600055B,                                                       // method token; computed at build time
	        reinterpret_cast<void*&>(android_app_activity_on_create_bundle)   // target pointer
	    );
	  }

	  android_app_activity_on_create_bundle (env, klass, savedInstanceState);
	}


~~ Other Changes ~~

The new `Android.Runtime.JNIEnvInit` type was split out of the
`Android.Runtime.JNIEnv` type to further reduce startup overhead, as
there are fewer fields to initialize.

The `Mono.Android.Runtime.dll` assembly is added because the
Marshal Method Wrapper needs to be able to invoke what *was*
`AndroidEnvironment.UnhandledException()`, *while also* updating
`Mono.Android.dll`!  `Mono.Android.Runtime.dll` allows the marshal
method wrappers to reliably use
`Android.Runtime.AndroidEnvironmentInternal.UnhandledException()`,
which will *never* be changed by the marshal method wrapper
infrastructure.


~~ Results ~~

Marshal methods make application startup around 3.2% faster (the
bigger the app the  more performance gains), with a bit room for
future improvements (by eliminating wrapper methods and other
optimizations):

[.NET Podcasts][6] app test results:

| Before  | After   | Δ        | Notes                                          |
| ------- | ------- | -------- | ---------------------------------------------- |
| 868.500 | 840.400 | -3.24% ✓ | preload disabled; 32-bit build; no compression |
| 863.700 | 837.600 | -3.02% ✓ | preload disabled; 64-bit build; no compression |
| 872.500 | 850.100 | -2.57% ✓ | preload enabled; 64-bit build                  |
| 877.000 | 854.800 | -2.53% ✓ | preload disabled; 64-bit build                 |
| 859.300 | 839.800 | -2.27% ✓ | preload enabled; 64-bit build; no compression  |
| 871.700 | 853.100 | -2.13% ✓ | preload enabled; 32-bit build                  |
| 860.600 | 842.300 | -2.13% ✓ | preload enabled; 32-bit build; no compression  |
| 869.500 | 852.500 | -1.96% ✓ | preload disabled; 32-bit build                 |

Maui Hello World app test results:

| Before  | After   | Δ        | Notes                                          |
| ------- | ------- | -------- | ---------------------------------------------- |
| 374.800 | 365.500 | -2.48% ✓ | preload disabled; 64-bit build                 |
| 374.100 | 365.600 | -2.27% ✓ | preload disabled; 32-bit build                 |
| 369.100 | 364.400 | -1.27% ✓ | preload enabled; 32-bit build                  |
| 364.300 | 360.600 | -1.02% ✓ | preload enabled; 32-bit build; no compression  |
| 368.900 | 365.400 | -0.95% ✓ | preload enabled; 64-bit build                  |
| 362.500 | 359.400 | -0.86% ✓ | preload disabled; 32-bit build; no compression |
| 361.100 | 361.600 | +0.14% ✗ | preload enabled; 64-bit build; no compression  |
| 359.200 | 368.000 | +2.39% ✗ | preload disabled; 64-bit build; no compression |


[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#native_method_arguments
[1]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#generator
[2]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
[3]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#resolving_native_method_names
[4]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-callable-wrapper-generator
[5]: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-7.0
[6]: https://github.com/microsoft/dotnet-podcasts/tree/net7.0

Author: grendello

Directory.Build.props

@@ -27,10 +27,6 @@
     <DisableTransitiveFrameworkReferenceDownloads>true</DisableTransitiveFrameworkReferenceDownloads>
   </PropertyGroup>
 
-  <PropertyGroup Condition=" '$(MSBuildRuntimeType)' == 'Core' ">
-    <_EnableMarshalMethods>NoThanks</_EnableMarshalMethods> <!-- set to YesPlease to enable -->
-  </PropertyGroup>
-
   <PropertyGroup>
     <ProductVersion>13.1.99</ProductVersion>
     <!-- NuGet package version numbers. See Documentation/guides/OneDotNet.md.

Documentation/guides/building-apps/build-properties.md

@@ -424,6 +424,15 @@ not wish to run those checks.
 
 Added in Xamarin.Android 9.4.
 
+## AndroidEnableMarshalMethods
+
+A bool property, not available in the classic Xamarin.Android
+releases.
+
+Enable or disable generation of [marshal
+methods](../../internals/JavaJNI_Interop.md). Defaults to `True` for
+`Release` builds and to `False` for `Debug` builds.
+
 ## AndroidEnableMultiDex
 
 A boolean property that

Documentation/guides/internals/JavaJNI_Interop.md

@@ -92,6 +92,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Mono.Android", "src\Mono.An
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Mono.Android.Export", "src\Mono.Android.Export\Mono.Android.Export.csproj", "{B8105878-D423-4159-A3E7-028298281EC6}"
 EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Mono.Android.Runtime", "src\Mono.Android.Runtime\Mono.Android.Runtime.csproj", "{43564FB3-0F79-4FF4-A2B0-B1637072FF01}"
+EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Android.Build.BaseTasks", "external\xamarin-android-tools\src\Microsoft.Android.Build.BaseTasks\Microsoft.Android.Build.BaseTasks.csproj", "{3DE17662-DCD6-4F49-AF06-D39AACC8649A}"
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Xamarin.Android.Tools.AndroidSdk", "external\xamarin-android-tools\src\Xamarin.Android.Tools.AndroidSdk\Xamarin.Android.Tools.AndroidSdk.csproj", "{E34BCFA0-CAA4-412C-AA1C-75DB8D67D157}"
@@ -296,6 +298,12 @@ Global
 		{B8105878-D423-4159-A3E7-028298281EC6}.Debug|AnyCPU.Build.0 = Debug|Any CPU
 		{B8105878-D423-4159-A3E7-028298281EC6}.Release|AnyCPU.ActiveCfg = Release|Any CPU
 		{B8105878-D423-4159-A3E7-028298281EC6}.Release|AnyCPU.Build.0 = Release|Any CPU
+
+		{43564FB3-0F79-4FF4-A2B0-B1637072FF01}.Debug|AnyCPU.ActiveCfg = Debug|Any CPU
+		{43564FB3-0F79-4FF4-A2B0-B1637072FF01}.Debug|AnyCPU.Build.0 = Debug|Any CPU
+		{43564FB3-0F79-4FF4-A2B0-B1637072FF01}.Release|AnyCPU.ActiveCfg = Release|Any CPU
+		{43564FB3-0F79-4FF4-A2B0-B1637072FF01}.Release|AnyCPU.Build.0 = Release|Any CPU
+
 		{3DE17662-DCD6-4F49-AF06-D39AACC8649A}.Debug|AnyCPU.ActiveCfg = Debug|Any CPU
 		{3DE17662-DCD6-4F49-AF06-D39AACC8649A}.Debug|AnyCPU.Build.0 = Debug|Any CPU
 		{3DE17662-DCD6-4F49-AF06-D39AACC8649A}.Release|AnyCPU.ActiveCfg = Release|Any CPU
@@ -456,6 +464,7 @@ Global
 		{73DF9E10-E933-4222-B8E1-F4536FFF9FAD} = {864062D3-A415-4A6F-9324-5820237BA058}
 		{66CF299A-CE95-4131-BCD8-DB66E30C4BF7} = {04E3E11E-B47D-4599-8AFC-50515A95E715}
 		{B8105878-D423-4159-A3E7-028298281EC6} = {04E3E11E-B47D-4599-8AFC-50515A95E715}
+		{43564FB3-0F79-4FF4-A2B0-B1637072FF01} = {04E3E11E-B47D-4599-8AFC-50515A95E715}
 		{3DE17662-DCD6-4F49-AF06-D39AACC8649A} = {04E3E11E-B47D-4599-8AFC-50515A95E715}
 		{E34BCFA0-CAA4-412C-AA1C-75DB8D67D157} = {04E3E11E-B47D-4599-8AFC-50515A95E715}
 		{1E5501E8-49C1-4659-838D-CC9720C5208F} = {CAB438D8-B0F5-4AF0-BEBD-9E2ADBD7B483}

Xamarin.Android.sln

@@ -62,7 +62,10 @@ steps:
   displayName: 'Copy binlogs'
   inputs:
     sourceFolder: ${{ parameters.designerSourcePath }}/Xamarin.Designer.Android
-    contents: '**/*.binlog'
+    contents: |
+      **/*.binlog
+      **/hs*.log
+      **/hs*.mdmp
     targetFolder: $(Build.ArtifactStagingDirectory)/designer-binlogs
     overWrite: true
     flattenFolders: true

build-tools/automation/yaml-templates/run-designer-tests.yml

@@ -33,6 +33,7 @@ by projects that use the Microsoft.Android framework in .NET 6+.
     <ItemGroup>
       <_AndroidRefPackAssemblies Include="$(JavaInteropSourceDirectory)\bin\$(Configuration)-net7.0\ref\Java.Interop.dll" />
       <_AndroidRefPackAssemblies Include="$(_MonoAndroidNETDefaultOutDir)ref\Mono.Android.dll" />
+      <_AndroidRefPackAssemblies Include="$(_MonoAndroidNETDefaultOutDir)ref\Mono.Android.Runtime.dll" />
       <!-- Always include stable Mono.Android.Export.dll -->
       <_AndroidRefPackAssemblies Include="$(_MonoAndroidNETOutputRoot)$(AndroidLatestStableApiLevel)\ref\Mono.Android.Export.dll" />
       <FrameworkListFileClass Include="@(_AndroidRefPackAssemblies->'%(Filename)%(Extension)')" Profile="Android" />

build-tools/create-packs/Microsoft.Android.Ref.proj

@@ -35,6 +35,7 @@ projects that use the Microsoft.Android framework in .NET 6+.
     <ItemGroup>
       <_AndroidRuntimePackAssemblies Include="$(_MonoAndroidNETDefaultOutDir)Java.Interop.dll" />
       <_AndroidRuntimePackAssemblies Include="$(_MonoAndroidNETDefaultOutDir)Mono.Android.dll" />
+      <_AndroidRuntimePackAssemblies Include="$(_MonoAndroidNETDefaultOutDir)Mono.Android.Runtime.dll" />
       <!-- Always include stable Mono.Android.Export.dll -->
       <_AndroidRuntimePackAssemblies Include="$(_MonoAndroidNETOutputRoot)$(AndroidLatestStableApiLevel)\Mono.Android.Export.dll" />
       <_AndroidRuntimePackAssets Include="$(MicrosoftAndroidSdkOutDir)lib\$(AndroidRID)\libmono-android.debug.so" />

build-tools/create-packs/Microsoft.Android.Runtime.proj

@@ -23,7 +23,6 @@
     />
     <ItemGroup>
       <_JavaSources Include="$(IntermediateOutputPath)jcw\src\**\*.java" />
-      <_JavaSources Include="$(JavaInteropSourceDirectory)\src\Java.Interop\java\com\xamarin\**\*.java" />
     </ItemGroup>
     <WriteLinesToFile
         File="$(IntermediateOutputPath)jcw/classes.txt"

build-tools/scripts/JavaCallableWrappers.targets

@@ -103,7 +103,7 @@
         WriteOutputAsMessage="True"
     />
     <Xamarin.Android.Tools.BootstrapTasks.Adb
-        Arguments="$(_AdbTarget) shell setprop debug.mono.log timing"
+        Arguments="$(_AdbTarget) shell setprop debug.mono.log default,assembly,timing=bare"
         IgnoreExitCode="True"
         ToolExe="$(AdbToolExe)"
         ToolPath="$(AdbToolPath)"

build-tools/scripts/TestApks.targets

@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="utf-8"?>
+<linker>
+	<assembly fullname="Mono.Android.Runtime" preserve="all" />
+</linker>