In general, after a .NET assembly is loaded into a program, its type information and data such as native code will remain in memory. The .NET runtime cannot reclaim them if we want to implement plug-in hot loading (such as Razor or A hot update of the Aspx template) can cause a memory leak. In the past, we could use the AppDomain mechanism of the .NET Framework, either using an interpreter (with some performance penalty) or restarting the program after a certain number of compilations (Asp.NET’s numRecompilesBeforeAppRestart) to avoid memory leaks.

Because .NET Core does not support the dynamic creation and uninstallation of AppDomains like the .NET Framework, there is no good way to implement plugin hot loading. The good news is that .NET Core has supported Collectible Assembly since 3.0. We can create a recyclable AssemblyLoadContext that can be used to load and unload assemblies.

This article will introduce how to use the .NET Core 3.0 AssemblyLoadContext to implement plug-in hot loading through a sample program of about 180 lines. The program uses Roslyn to implement dynamic compilation. The final result is that the plug-in code can be automatically updated to the running program. And will not cause a memory leak.

Complete source code and folder structure

First, let’s take a look at the complete source code and folder structure. The source code is divided into two parts, one is the host, responsible for compiling and loading the plugin, and the other is the plugin, which will be explained in detail later in the source code.

Folder structure:

  • Pluginexample (top folder)
    • Host (hosted project)
      • Program.cs (hosted code)
      • Host.csproj (hosted project file)
    • Guest (plugin’s code folder)
      • Plugin.cs (plugin code)
      • Bin (the folder where the plugin compilation results are saved)
        • MyPlugin.dll (plug-in compiled DLL file)

Content of Program.cs:

using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.Loader;
using System.Threading;

namespace Common
{
    public interface IPlugin : IDisposable
    {
        string GetMessage();
    }
}

namespace Host
{
    using Common;

    internal class PluginController : IPlugin
    {
        private List<Assembly> _defaultAssemblies;
        private AssemblyLoadContext _context;
        private string _pluginName;
        private string _pluginDirectory;
        private volatile IPlugin _instance;
        private volatile bool _changed;
        private object _reloadLock;
        private FileSystemWatcher _watcher;

        public PluginController(string pluginName, string pluginDirectory)
        {
            _defaultAssemblies = AssemblyLoadContext.Default.Assemblies
                .Where(assembly => !assembly.IsDynamic)
                .ToList();
            _pluginName = pluginName;
            _pluginDirectory = pluginDirectory;
            _reloadLock = new object();
            ListenFileChanges();
        }

        private void ListenFileChanges()
        {
            Action<string> onFileChanged = path =>
            {
                if (Path.GetExtension(path).ToLower() == ".cs")
                    _changed = true;
            };
            _watcher = new FileSystemWatcher();
            _watcher.Path = _pluginDirectory;
            _watcher.IncludeSubdirectories = true;
            _watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName;
            _watcher.Changed += (sender, e) => onFileChanged(e.FullPath);
            _watcher.Created += (sender, e) => onFileChanged(e.FullPath);
            _watcher.Deleted += (sender, e) => onFileChanged(e.FullPath);
            _watcher.Renamed += (sender, e) => { onFileChanged(e.FullPath); onFileChanged(e.OldFullPath); };
            _watcher.EnableRaisingEvents = true;
        }

        private void UnloadPlugin()
        {
            _instance?.Dispose();
            _instance = null;
            _context?.Unload();
            _context = null;
        }

        private Assembly CompilePlugin()
        {
            var binDirectory = Path.Combine(_pluginDirectory, "bin");
            var dllPath = Path.Combine(binDirectory, $"{_pluginName}.dll");
            if (!Directory.Exists(binDirectory))
                Directory.CreateDirectory(binDirectory);
            if (File.Exists(dllPath))
            {
                File.Delete($"{dllPath}.old");
                File.Move(dllPath, $"{dllPath}.old");
            }

            var sourceFiles = Directory.EnumerateFiles(
                _pluginDirectory, "*.cs", SearchOption.AllDirectories);
            var compilationOptions = new CSharpCompilationOptions(
                OutputKind.DynamicallyLinkedLibrary,
                optimizationLevel: OptimizationLevel.Debug);
            var references = _defaultAssemblies
                .Select(assembly => assembly.Location)
                .Where(path => !string.IsNullOrEmpty(path) && File.Exists(path))
                .Select(path => MetadataReference.CreateFromFile(path))
                .ToList();
            var syntaxTrees = sourceFiles
                .Select(p => CSharpSyntaxTree.ParseText(File.ReadAllText(p)))
                .ToList();
            var compilation = CSharpCompilation.Create(_pluginName)
                .WithOptions(compilationOptions)
                .AddReferences(references)
                .AddSyntaxTrees(syntaxTrees);

            var emitResult = compilation.Emit(dllPath);
            if (!emitResult.Success)
            {
                throw new InvalidOperationException(string.Join("\r\n",
                    emitResult.Diagnostics.Where(d => d.WarningLevel == 0)));
            }
            //return _context.LoadFromAssemblyPath(Path.GetFullPath(dllPath));
            using (var stream = File.OpenRead(dllPath))
            {
                var assembly = _context.LoadFromStream(stream);
                return assembly;
            }
        }

        private IPlugin GetInstance()
        {
            var instance = _instance;
            if (instance != null && !_changed)
                return instance;

            lock (_reloadLock)
            {
                instance = _instance;
                if (instance != null && !_changed)
                    return instance;

                UnloadPlugin();
                _context = new AssemblyLoadContext(
                    name: $"Plugin-{_pluginName}", isCollectible: true);

                var assembly = CompilePlugin();
                var pluginType = assembly.GetTypes()
                    .First(t => typeof(IPlugin).IsAssignableFrom(t));
                instance = (IPlugin)Activator.CreateInstance(pluginType);

                _instance = instance;
                _changed = false;
            }

            return instance;
        }

        public string GetMessage()
        {
            return GetInstance().GetMessage();
        }

        public void Dispose()
        {
            UnloadPlugin();
            _watcher?.Dispose();
            _watcher = null;
        }
    }

    internal class Program
    {
        static void Main(string[] args)
        {
            using (var controller = new PluginController("MyPlugin", "../guest"))
            {
                bool keepRunning = true;
                Console.CancelKeyPress += (sender, e) => {
                    e.Cancel = true;
                    keepRunning = false;
                };
                while (keepRunning)
                {
                    try
                    {
                        Console.WriteLine(controller.GetMessage());
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine($"{ex.GetType()}: {ex.Message}");
                    }
                    Thread.Sleep(1000);
                }
            }
        }
    }
}

The contents of host.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.3.1" />
  </ItemGroup>

</Project>

Content of Plugin.cs:

using System;
using Common;

namespace Guest
{
    public class MyPlugin : IPlugin
    {
        public MyPlugin()
        {
            Console.WriteLine("MyPlugin loaded");
        }

        public string GetMessage()
        {
            return "Hello 1";
        }

        public void Dispose()
        {
            Console.WriteLine("MyPlugin unloaded");
        }
    }
}

Running the sample program

Into the pluginexample/hostnext run dotnet runto start the program host, then change automatically compile and load the plug, plug detect host file and recompile the program will load at the time of the change. You can modify after running pluginexample/guest/Plugin.csin Hello 1to Hello 2, then you can see output similar to the following:

MyPlugin loaded
Hello 1
Hello 1
Hello 1
MyPlugin unloaded
MyPlugin loaded
Hello 2
Hello 2

We can see that the program automatically updates and executes the modified code. If you are interested, you can test what happens when the plugin code syntax error occurs.

Source code explanation

Next is a detailed explanation of the various parts of the source code of the host:

IPlugin interface

public interface IPlugin : IDisposable
{
    string GetMessage();
}

This is the implementation interface required by the plugin project. After compiling the plugin, the host project will look for the type of IPlugin in the assembly. Create an instance of this type and use it. When the plugin is created, the constructor will be called. When the plugin is uninstalled, the Dispose method will be called. If you have used the .NET Framework’s AppDomain mechanism, you may want to ask for Marshalling. The answer is no, the .NET Core’s recyclable assembly will be loaded into the current AppDomain. Recycling relies on GC cleanup. The advantage is that it is easy to use and The efficiency is high. The downside is that there is a delay in GC cleanup. As long as there is an instance of the plugin type that is not being recycled, the data used by the plugin assembly will remain, resulting in a memory leak.

PluginController type

internal class PluginController : IPlugin
{
    private List<Assembly> _defaultAssemblies;
    private AssemblyLoadContext _context;
    private string _pluginName;
    private string _pluginDirectory;
    private volatile IPlugin _instance;
    private volatile bool _changed;
    private object _reloadLock;
    private FileSystemWatcher _watcher;

This is the proxy class for the management plugin, which internally is responsible for compiling and loading the plugin, and forwards method calls to the IPlugin interface to the implementation of the plugin. Class members include a list of assemblies in the default AssemblyLoadContext, a _defaultAssembliescustom AssemblyLoadContext for loading plugins _context, plugin names and folders, plugin implementations _instance, markup plugin files have been changed _changed, prevent multiple threads from compiling plugins at the same time _reloadLock, and monitoring plugins The file changes _watcher.

Constructor for PluginController

public PluginController(string pluginName, string pluginDirectory)
{
    _defaultAssemblies = AssemblyLoadContext.Default.Assemblies
        .Where(assembly => !assembly.IsDynamic)
        .ToList();
    _pluginName = pluginName;
    _pluginDirectory = pluginDirectory;
    _reloadLock = new object();
    ListenFileChanges();
}

Constructor from AssemblyLoadContext.Default.Assembliesobtaining the default program AssemblyLoadContext in the set list, including the host assembly, System.Runtime, etc., the list is used when compiling plug-in Roslyn, indicate which plug-ins need to reference the assembly at compile time. After the call will be ListenFileChangeswhether the listener plug-in file has changed.

PluginController.ListenFileChanges

private void ListenFileChanges()
{
    Action<string> onFileChanged = path =>
    {
        if (Path.GetExtension(path).ToLower() == ".cs")
            _changed = true;
    };
    _watcher = new FileSystemWatcher();
    _watcher.Path = _pluginDirectory;
    _watcher.IncludeSubdirectories = true;
    _watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.FileName;
    _watcher.Changed += (sender, e) => onFileChanged(e.FullPath);
    _watcher.Created += (sender, e) => onFileChanged(e.FullPath);
    _watcher.Deleted += (sender, e) => onFileChanged(e.FullPath);
    _watcher.Renamed += (sender, e) => { onFileChanged(e.FullPath); onFileChanged(e.OldFullPath); };
    _watcher.EnableRaisingEvents = true;
}

This method creates FileSystemWatcher, whether the file listener plugin folder there is a change, if there is change and change is the C # source code (.cs extension) is set _changedmember to true, the members of the tag plugin file has been changed, the next visit plug-in A reload is triggered when the instance is called.

You may have questions about why the plugin is not reloaded immediately after the file is changed. One reason is that the file saver implementation of some file editors may cause the changed event to fire several times in succession. Delayed triggering can avoid multiple compilations. Another reason Exceptions that occur during compilation can be passed to the thread that accesses the plugin instance for debugging and debugging (although ExceptionDispatchInfo can be used).

PluginController.UnloadPlugin

private void UnloadPlugin()
{
    _instance?.Dispose();
    _instance = null;
    _context?.Unload();
    _context = null;
}

This method will unload loaded plug-in, first call IPlugin.Disposenotification plug-in is being unloaded, if the plug-in creates a new thread can Disposestop the thread method to avoid leaks, and then call AssemblyLoadContext.Unload.NET Core runtime allows unloading assembly this context loaded assembly The data is reclaimed after the GC detects that all types of instances have been recycled (see the link at the beginning of the article).

PluginController.CompilePlugin

private Assembly CompilePlugin()
{
    var binDirectory = Path.Combine(_pluginDirectory, "bin");
    var dllPath = Path.Combine(binDirectory, $"{_pluginName}.dll");
    if (!Directory.Exists(binDirectory))
        Directory.CreateDirectory(binDirectory);
    if (File.Exists(dllPath))
    {
        File.Delete($"{dllPath}.old");
        File.Move(dllPath, $"{dllPath}.old");
    }

    var sourceFiles = Directory.EnumerateFiles(
        _pluginDirectory, "*.cs", SearchOption.AllDirectories);
    var compilationOptions = new CSharpCompilationOptions(
        OutputKind.DynamicallyLinkedLibrary,
        optimizationLevel: OptimizationLevel.Debug);
    var references = _defaultAssemblies
        .Select(assembly => assembly.Location)
        .Where(path => !string.IsNullOrEmpty(path) && File.Exists(path))
        .Select(path => MetadataReference.CreateFromFile(path))
        .ToList();
    var syntaxTrees = sourceFiles
        .Select(p => CSharpSyntaxTree.ParseText(File.ReadAllText(p)))
        .ToList();
    var compilation = CSharpCompilation.Create(_pluginName)
        .WithOptions(compilationOptions)
        .AddReferences(references)
        .AddSyntaxTrees(syntaxTrees);

    var emitResult = compilation.Emit(dllPath);
    if (!emitResult.Success)
    {
        throw new InvalidOperationException(string.Join("\r\n",
            emitResult.Diagnostics.Where(d => d.WarningLevel == 0)));
    }
    //return _context.LoadFromAssemblyPath(Path.GetFullPath(dllPath));
    using (var stream = File.OpenRead(dllPath))
    {
        var assembly = _context.LoadFromStream(stream);
        return assembly;
    }
}

This method will call Roslyn to compile the plugin code into the DLL and load the compiled DLL using the custom AssemblyLoadContext. First of all, it needs to delete the original DLL file. Because there is a delay in uninstalling the assembly, the original DLL file will probably be deleted and prompted to be used on the Windows system, so it needs to be renamed and deleted next time. Then it looks for all the C # source code plug-in folder, with CSharpSyntaxTreeresolve them, and with CSharpCompilationcompiling reference when compiling the set list is a program set list default AssemblyLoadContext constructor acquired (including host the assembly, so that the plugin The code can use the IPlugin interface). After the compilation is successful, the compiled Assembly DLL is loaded with a custom AssemblyLoadContext to support the uninstallation.

This code should be noted that there are two parts, the first part is not going to throw an exception when Roslyn compilation fails, you need to determine compiled emitResult.Successand from emitResult.Diagnosticsthe error message; the second part is to load the plug-in assembly must be used AssemblyLoadContext.LoadFromStreamfrom memory data loading, if AssemblyLoadContext.LoadFromAssemblyPaththe next time will still return to the assembly for the first time to load when loading from the same path, which may be 3.0 .NET Core implementation issue and may fix in a future release.

PluginController.GetInstance

private IPlugin GetInstance()
{
    var instance = _instance;
    if (instance != null && !_changed)
        return instance;

    lock (_reloadLock)
    {
        instance = _instance;
        if (instance != null && !_changed)
            return instance;

        UnloadPlugin();
        _context = new AssemblyLoadContext(
            name: $"Plugin-{_pluginName}", isCollectible: true);

        var assembly = CompilePlugin();
        var pluginType = assembly.GetTypes()
            .First(t => typeof(IPlugin).IsAssignableFrom(t));
        instance = (IPlugin)Activator.CreateInstance(pluginType);

        _instance = instance;
        _changed = false;
    }

    return instance;
}


This method is the method to get the latest plugin instance. If the plugin instance has been created and the file has not changed, return the existing instance, otherwise uninstall the original plugin, recompile the plugin, load and generate the instance. Note that the AssemblyLoadContext type is abstract in netstandard (including 2.1) and cannot be created directly. Only netcoreapp3.0 can be created directly (currently only .NET Core 3.0 supports this mechanism). If you need to support recyclable, you need to set it when creating. The isCollectible parameter is true because support for recyclability causes the GC to do some extra work when scanning objects, so it is not enabled by default.

PluginController.GetMessage

public string GetMessage()
{
    return GetInstance().GetMessage();
}

This method is a proxy method that gets the latest plugin instance and forwards the call parameters and results. If there are other methods for IPlugin, you can write it like this method.

PluginController.Dispose

public void Dispose()
{
    UnloadPlugin();
    _watcher?.Dispose();
    _watcher = null;
}

This method supports the active release of the PluginController, which will uninstall the loaded plugin and stop listening to the plugin file. Because the PluginController does not directly manage unmanaged resources, and
the destruction of the AssemblyLoadContext

triggers the unload, the PluginController does not need to provide a destructor.

Main function code

static void Main(string[] args)
{
    using (var controller = new PluginController("MyPlugin", "../guest"))
    {
        bool keepRunning = true;
        Console.CancelKeyPress += (sender, e) => {
            e.Cancel = true;
            keepRunning = false;
        };
        while (keepRunning)
        {
            try
            {
                Console.WriteLine(controller.GetMessage());
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{ex.GetType()}: {ex.Message}");
            }
            Thread.Sleep(1000);
        }
    }
}

The main function creates the PluginController instance and specifies the above guest folder as the plugin folder, then calls the GetMessage method every 1 second, so that when the plugin code changes, we can observe it from the console output, if the plugin code contains A syntax error will throw an exception when called, and the program will continue to run and retry compiling and loading on the next call.

Written at the end

This article is the end of this article, in this article we saw a simple .NET Core 3.0 plug-in hot-loading implementation, this implementation still has a lot of improvements, such as how to manage multiple plug-ins, how to restart the host program After avoiding recompiling all plugins, how to compile the plugin code, etc. If you are interested in solving them, do a plugin system embedding into your project, or write a new framework.

Regarding ZKWeb, 3.0 will use the mechanism described in this article to implement plug-in hot loading, but since I have already withdrawn from the IT industry, all development is done in a spare free time, so basically there will be no big updates, ZKWeb is more Will be used as a framework for implementation reference. In addition, I am writing the HTTP framework cpv-framework in C++ , with a focus on performance (more than twice the throughput of .NET Core 3.0 and flat with actix-web), which has not yet been officially released.

Regarding books, the publisher agreed to November but has not yet let me see the revised manuscript (although I will answer it when I ask), so it is very likely that I will continue to postpone, sorry for the students who are looking forward to publishing, the book is currently Still based on .NET Core 2.2 instead of .NET Core 3.0.

Orignal link:https://www.cnblogs.com/zkweb/p/11630228.html