It turns out this optimization brings in a breaking change: `$MyInvocation` is different comparing to before the optimization change. I tried to fix the breaking change, but couldn't without introducing more hacky code. Given that, that PR should be reverted.
* Update 'Microsoft.PowerShell.CoreCLR.Eventing' to resolve conflicts
* Add reference to 'System.Diagnostics.EventLog' to build Microsoft.PowerShell.Commands.Diagnostics on Unix
* Stop compiling Get/New-WinEvent on Unix
- Reusing the same 2-dimensional integer array for processing line comments in the same Tokenizer.
- Use ArrayPool<int> in PatternPositionVisitor to avoid creating transient int[].
.NET Core changes to return "shortest roundtrippable string" by default for the ToString() method of double and float types. This results in ToString() for double/float values sometimes return a string in 17-digit/9-digit precision format. This PR updated the double/float-to-string conversion in PowerShell to continue using the old precision specifier before the change in .NET Core 3.0.
Update the task-based async APIs added to PowerShell to not use the `aysnc/await` keywords, but to return a `Task` object directly.
There is nothing to continue on after the `Task.Factory.FromAsync` call in those methods, so there is not need to use `aysnc` and `await` keywords, which turns the method into a state machine class unnecessarily.
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
Update `New-ReferenceAssembly` and `New-UnifiedNugetPackage` to generate reference assembly for `Microsoft.PowerShell.Commands.Utility.dll` and properly deploy it for `Microsoft.PowerShell.Commands.Utility` NuGet package and `Microsoft.PowerShell.SDK` NuGet package.
An incremental step to fix, eventually, #8121
We have the public API `JsonObject.ConvertFromJson` to convert from JSON string in the PowerShell context. It would be good to have a public API for conversion to JSON. This PR refactors the `ConvertTo-Json` cmdlet to move the core implementation to `JsonObject.ConvertToJson`, and make `ConvertTo-Json` call that public method.
This would help the Azure Function PowerShell worker. Currently, we depends on [calling the cmdlet](https://github.com/Azure/azure-functions-powershell-worker/blob/729710d2595b4b96abd4fa5a5547fe880e98bfa6/src/PowerShell/PowerShellManager.cs#L198-L205) to convert object to JSON which is expensive. Once we have the public method `JsonObject.ConvertToJson` exposed, we can call the API directly to avoid a command invocation.
# Conflicts:
# test/Test.Common.props
We have the public API `JsonObject.ConvertFromJson` to convert from JSON string in the PowerShell context. It would be good to have a public API for conversion to JSON. This PR refactors the `ConvertTo-Json` cmdlet to move the core implementation to `JsonObject.ConvertToJson`, and make `ConvertTo-Json` call that public method.
This would help the Azure Function PowerShell worker. Currently, we depends on [calling the cmdlet](https://github.com/Azure/azure-functions-powershell-worker/blob/729710d2595b4b96abd4fa5a5547fe880e98bfa6/src/PowerShell/PowerShellManager.cs#L198-L205) to convert object to JSON which is expensive. Once we have the public method `JsonObject.ConvertToJson` exposed, we can call the API directly to avoid a command invocation.
The last two commits to this file "broke" both "Developing and contributing"(link is missing) and "Support" (don't know what it didn't format correctly.
This commit fixes both cases.
Major changes are:
- Rename `s_wasSystemPolicyDebugPolicy` to `s_allowDebugOverridePolicy` to make it less confusing. Also slightly refactor `HelperSecurity.psm1` and `ConstrainedLanguageDebugger.Tests.ps1` to remove unneeded code. There is no functional change in this commit.
- Remove the unneeded static property `IsInbox`, as PowerShell Core won't be shipped in-box with Windows in the foreseeable feature. Even if we do in future, we won't be needing it because Windows PowerShell will probably be gone by that time.
- Update 'BindRunspace' to avoid getting all commands and unneeded method calls.
- Avoid creating a `IsSafeValueVisitor` every time when `IsScriptBlockInFactASafeHashtable` runs.
Major changes are as follows:
- Avoid `SecuritySupport.IsProductBinary` and unnecessary AMSI/suspicious code scan at startup time
- Update `CompiledScriptBlockData.IsProductCode` to avoid unnecessary calls to `IsProductBinary`, which attempts to retrieve catalog signature of the target file.
- Update `PerformSecurityChecks` to skip AMSI and suspicious code scan for the `.psd1` file that contains a safe `HashtableAst` only.
- Use customized `ReadOnlyBag` instead of `ImmutableHashSet` so that we can avoid loading the `System.Collections.Immutable.dll` completely.
- Replace `SHA1` with `CRC32` when generating module analysis cache file name
- This remove the loading of `System.Security.Cryptography.Algorithms.dll` at startup
- Move `ConvertFrom-SddlString` to C# to remove the `Utility.psm1` file.
- Crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation
- Even pwsh with crossgen assemblies spends a lot time in jitting at the startup, about `191.6ms` comparing with `24.7ms` for Windows PowerShell.
- Jitting `Microsoft.ApplicationInsights.dll` takes about `51.6ms`.
- By crossgen `Microsoft.ApplicationInsights.dll` and enable tiered compilation, the jitting time drops to about `98.9ms`.
The root cause is that `OutputBuffer` is not cleaned up (set to `null`) in `Stop` and `BeginStop/EndStop` when the powershell instance owns the `OutputBuffer` object. Since the pipeline has been intentionally stopped by the caller, the `OutputBuffer` should be set to `null` as well when it's owned by the PowerShell instance, just like how it's cleaned up in `EndInvoke`.
The type System.IntPtr has an explicit cast operator defined that takes a pointer type System.void* parameter. Now we handle that type properly in GetPSMethodProjectedType.
Support calling methods with ByRef-like type parameters in PowerShell, as long as the argument specified for the parameter can be implicitly/explicitly converted to the ByRef-like type.
We cannot create an instance of a ByRef-like type in PowerShell, but there are types that can be implicitly or explicitly converted to ByRef-like types, such as `T[] -> Span<T>`, `T[] -> ReadOnlySpan<T>`, `String -> ReadOnlySpan<T>`. `ArraySegment<T> -> Span<T>`, `ArraySegment<T> -> ReadOnlySpan<T>`. With changes in this PR, we can call methods with ByRef-like parameter types by providing the arguments of the types that can be cast to the target ByRef-like type.
**What is enabled?**
1. Invoking methods that have ByRef-like parameters, but the return type is not ByRef-like.
2. Invoking constructors with ByRef-like parameters via `[target-type]::new` syntax, but the `target-type` is not ByRef-like.
3. Accessing indexers that have ByRef-like parameters, but the return type is not ByRef-like.
ByRef-like types are supposed to be used on stack only, so we need to fail gracefully when accessing properties, calling methods, or creating objects related to ByRef-like types.
A comment from the existing `Utils.IsComObject` mentions that: "We can't use System.Runtime.InteropServices.Marshal.IsComObject(obj) since it doesn't work in partial trust."
Partial trust is a concept of Code Access Security (CAS)/Security-Transparent. CAS and Security-Transparent are no longer supported in .NET Core, so we should use `Marshal.IsComObject` instead.
- Make switch-statement report correct error position when it fails to evaluate the condition.
- Make for-statement report correct error position when it fails to evaluate the initializer.
- For the condition of `if/for/while/do-while/do-until` statements, the sequence point update is either duplicate in some cases (for `if/for/while`) which causes debugger to stop at the condition twice before moving forward, or missing (for `do-while/do-until`) which causes debugger to skip the condition. They are fixed.
- `ConvertFrom-Markdown` is used for converting a markdown document or string to a MarkdownInfo object. It can optionally return an HTML or a VT100 encoded string in addition to an AST of the markdown document.
- `Show-Markdown` is used to either display the VT100 encoded string on the console or redirect the HTML string to the browser.
- `Set/Get-MarkdownOption` cmdlets are used to view or set markdown rendering options.
* The major refactoring changes are:
- In `ModuleIntrisic.cs`, remove unneeded Windows-PowerShell-only code.
- In `ModuleUtils.cs`
- use the new API `Directory.GetDirectories(string path, string searchPattern, EnumerationOptions enumerationOptions)` and `Directory.GetFiles(string path, string searchPattern, EnumerationOptions enumerationOptions)` to enumerate files and sub-directories within a directory path.
- remove the unused parameter `bool force` from `GetDefaultAvailableModuleFiles(bool force, bool isForAutoDiscovery, ExecutionContext context)`
- refactor the method `GetModuleVersionsFromAbsolutePath`. Add more comments and rename the method name.
- In `ModuleCmdletBase.cs`, refactor the method `GetModuleForNonRootedPaths` to `GetModuleForNames` to simply its implementation.
- In `PSModuleInfo.cs`
- group the declarations of `Declared*Exports` fields together
- rename `_detected*Exports` fields to `Detected*Exports` to group them together. They are internal fields and used outside `PSModuleInfo`.
* There is some perf improvement after this refactoring change:
- For `Get-Module -ListAvailable`, there is about 36% speed improvement for 94 default modules.
- For `Get-Module -ListAvailable -All`, there is about 14% speed improvement for totally 600 module files.
- For `Get-Module <name> -ListAvailable -List`, there is over 17x speed improvement for finding 13 modules from 600 modules. This is because we now filter names using the module file before creating a `PSMdouleInfo` object.
The fix causes a problem in debugging. When stepping over the scripts in debugging mode, the debugger will stop at the switch statement condition expression twice at the very beginning -- one for evaluating the condition, and one for iterating the condition.
Dongbo Wang