This PR fixes the logging issue on Linux where logging is initialized before `-settingsFile` is parsed causing custom log settings to be ignored. (see ConsoleHost.cs and ManagedEntrance.cs)
The PR also includes basic logging tests for Linux and MacOS. PSSyslog.psm1 contains the functions to retrieve selected logged items (based on PowerShell's log id and a timestamp) and Logging.Tests.ps1 contains tests for Linux and MacOS.
Based on standard practices, we need to have a copyright and license notice at the top of each source file. Removed existing copyrights and updated/added copyright notices for .h, .cpp, .cs, .ps1, and .psm1 files.
Updated module manifests for consistency to have Author = "PowerShell" and Company = "Microsoft Corporation". Removed multiple line breaks.
Separate PR coming to update contribution document for new source files: #6140
Manually reviewed each change.
Fix#6073
- Remove previous `UseAlwaysAnalytic` workaround in `SysLogProvider.Log`
- Update Script Block logging to always log to the operational channel with `UseAlwaysOperational`
- Fix `PSChannel` on Linux to use a bitmask
- Handle `UseAlwaysOperational` and `UseAlwaysAnalytic` keywords but adding to `_keywordFilter` if the associated channels were selected in the configuration
This PR is divided into the following areas:
1. Add/Rename the PowerShell/Windows ETW manifest to the repo and change both the provider id (guid) and name. See #4939.
The manifest is at tools/resxgen/PowerShell-Core-Instrumentation.man
2. Generate a resx file containing the string resources needed for logging ETW events to syslog.
This is accomplished by tools\resxgen\resxgen.psm1 and resxgen.ps1. The tool generates two files
A resx file containing string resources for each message string from the manifest. This is generated in the System.Management.Automation\gen directory.
A C# class (EventResource) that provides the mapping between the integer event id and the associated string resource name. The file is generated in the System.Management.Automation\CoreCLR directory with a compile-time condition of UNIX
NOTE: The EventResource.cs class generated by resgen is explicitly ignored in the csproj file; it is not used.
3. SMA\utils\tracing\PSSysLogProvider.cs
Implements the abstract LogProvider class and is the syslog equivalent of PSEtwLogProvider. The class contains a number of logical methods for logging lifecycle, health, and normal events.
4. SMA\utils\tracing\SysLogProvider.cs
This is the Syslog equivalent of ETW's log provider class and implements a Log method versus ETW's EventWrite. It is also responsible for resolving event ids to resource names and performing the Syslog call. There is a large comment block in the class XML doc that describes the types of log output it produces.
5. SMA\utils\tracing\PSEtwLog.cs
PowerShell's current implementation is tightly coupled to this class; with code calling it directly for all events. To simplify integration of syslog, I updated the class to create an instance of PSSysLogProvider on Linux and removed the Linux-specific stub file.
6. SMA\engine\PropertyAccessor.cs
This class provides a wrapper around PowerShellProperties.json and has been extended to have Unix-specific assessors for configuring logging. Note that the file is expected to be in the $PSHOME directory to ensure SxS.
Currently, there are four configuration properties:
- LogIdentity - the string identifier for the source application. This defaults to 'powershell' and can be configured to enable distinguishing between side-by-side installations.
- LogLevel - configures the tracing level (log level). Informational is the defauilt.
- LogChannels - used to enable operational and analytic logging. Operational is the default.
- LogKeywords - used to configure enabling tracing by keyword. All keywords other than `UseAlwaysAnalytic` are enabled by default.
NOTE: This will likely change. PowerShell sometimes confuses the analytic channel with this keyword and sends logging to the wrong channel. Once this is cleared up, `UseAlwaysAnalytic` and `UseAlwaysOperational` keywords will likely be removed.
Additional Notes:
1. The current implementation writes directly to syslog and writing to a separate log file is still pending.
2. The generated class and resx are not part of the build; instead, it is expected that Resxgen should be run when events are added to the manifest. To fully automate the process, resxgen will need to be updated to generate the other dependent enums such as 'PSChannel', 'PSEventId', 'PSTask', 'PSOpcode', etc. You will see parsing logic in resxgen.psm1 to prepare for that but it is not enabled at this point.
4. Documentation is pending that documents the format of the syslog output as well as associated configuration.
5. As mentioned at the start, tests are pending
Using the assembly name to hint at the source of the classes was
problematic in multiple ways.
This change stores the actual filename in an attribute on the assembly.
So for a given type, one can get the assembly this way:
[SomeType].Assembly.GetCustomAttributes() |
? { $_ -is [System.Management.Automation.DynamicClassImplementationAssemblyAttribute] } |
% { $_.ScriptFile }
Added the following to the Get-Service command:
•BinaryPathName (Was suggested as ServicePath )
•Description
•UserName (Was suggested as LogOnAs )
•DelayedAutoStart
•ServiceStartupType
The ServiceStartupType was also added, providing the user access to use services like in services.msc , having Automatic , Automatic (Delayed Start) , Manual and Disabled as options (Also includes InvalidValue as an initial value for ServiceStartupType )
This unifies file encoding across the inbox cmdlets to be UTF-8 without a BOM for all platforms. This is a breaking change as cmdlets on windows have a number of different encodings. This supports better interoperability with tradition Linux shells as we are using the same encoding.
Validate that files are created with UTF-8 encoding without BOM
Update tests to validate Encoding parameter to new type and create new tests for
parameter type validation.
[Breaking Change] The '-Encoding Byte' has been removed from the filesystem provider cmdlets. A new parameter '-AsByteStream' is now added to indicate that a byte stream is required as input, or output will be a stream of bytes.
For PSCore 6, we are only supporting InitialSessionState. The RunspaceConfiguration APIs were already made internal. This PR removes all the code related to RunspaceConfiguration. This also means that some public APIs have changed. Was deciding between leaving the RunspaceConfiguration parameters and throwing Unsupported, but thought it was better to have it a compile-time error. This should simplify the code base.
* Get-Verb: add descriptions and prefixes
* Add description and alias prefix tests
* Make the new classes internal
* Convert alias prefix to static method
* Move verb descriptions to a resource file
* Shorten function name
* Shorten alias prefix function name
* Change tests to look at counts
* Add uniqueness test
* Add missing alias prefixes
* Fix name collision
* Change alias prefix for Compare
* Rename method
* Remove unnecessary string allocation
* Make tests easier to diagnose
- add ending newline to files missing it using:
`awk -F: '$2 ~ / no line terminators/ {print $1}' ~/text-files.txt | xargs -I{} sh -c 'printf "\n" >> "$1"' - {}`
- update .gitattributes to enforce autocrlf on all text files
- added <copyright> opening element where it was missing
removed file attribute pointing to wrong filename
- fix mis-encoded character to apostrophe
- replace incorrect encoding of copyright symbol with (c)
- updated file hashes in the test
* [Feature]Remove-Service added to Management module
* [Feature]Capitalization
* [Feature]Added test cases for Remove-Service, Removed return value
* [Feature]Documentation from copy-paste code corrected to reflect the new function
* [Feature]Erroraction to be sure that an error is thrown in testcase
* [Feature]Removed direct reference to module in Pester test
* [Feature]Removed extra line in test
* [Feature]Use FullyQualifiedErrorId in Remove-Service test without a valid servicename
* [Feature]Exposed Remove-Service
* [Feature]Consistent casing, Named Arguments & Remove incorrect exception on cleanup
* [Feature]Remove-Service test should fail if the service was not removed
* [Feature]Cleanup comments & add Remove-Service for CI
* [Feature]Remove-Service in CI set in alphabetic order
* [Feature]Use ParameterSetName instead of _ParameterSetName and rewrite test which used the function class directly
* Revert "[Feature]Use ParameterSetName instead of _ParameterSetName and rewrite test which used the function class directly"
This reverts commit da41f7deb811f32538e0dc7f0d811646ceb8acf8.
* [Feature]Remove _ParameterSetName check & added test for pipeline input in
Set-Service
* AlternateStreams support relies on pinvoke to win32 APIs which don't work on non-Windows and produces errors about not loading a DLL in the case of copy-item
* Address PR feedback, remove -Stream parameter for Unix added new -stream tests
* if/def out alternate stream syntax checks not applicable to Unix
To improve code clarity and give better error messages when using ValidateRange, you can now specify:
```
[ValidateRange(ValidateRangeKind.Positive)]
```
instead of
```
[ValidateRange(1, 2147483647)]
```
Valid kinds are:
Positive
Negative
NonPostiive
NonNegative
Make `System.Management.Automation references` to package `System.Security.Permissions`, which brings back `SecurityZone` and `System.Security.Policy.Zone`.
* address static analysis issues
- remoteRunspaces is not used TFS:10618323
- not checking return value of ResumeThread() TFS:10618344
* fixed magic number use
* changed magic number to Uint.MaxValue
Remove the code that spins up our own assembly load context. Keep the code that registers our `Resolve` method to the default loader's `Resolving` event, so that we can continue to do special assembly resolution as needed (such as the GAC probing logic needed for consuming FullCLR PS modules).
Essentially, the assembly `Microsoft.PowerShell.CoreCLR.AssemblyLoadContext.dll` is not needed anymore, the remaining code should be moved to S.M.A.dll. However, that will break DSC and other native hosts that are hosting powershell. So this assembly is kept for now.
* Caching the parent id of the current process
In certain scenarios, like a domain joined machine with an executionpolicy set via GPO, a lot of checks are done for the ExecutionPolicy, and this is in the hot path.
* Caching HasGPScriptParent.
It is expensive to determine if a script was run as a result of applying a group policy. This does not change during the lifetime of a process.
This only affect the scenario when an execution policy has been set by a group policy.
* Using available Microsoft.PowerShell.ProcessCodeMethods.GetParentPid
This change moves powershell to .NET Core 2.0. Major changes are:
1. PowerShell assemblies are now targeting `netcoreapp2.0`. We are using `microsoft.netcore.app-2.0.0-preview1-001913-00`, which is from dotnet-core build 4/4/17. We cannot target `netstandard2.0` because the packages `System.Reflection.Emit` and `System.Reflection.Emit.Lightweight`, which are needed for powershell class, cannot be referenced when targeting `netstandard2.0`.
2. Refactor code to remove most CLR stub types and extension types.
3. Update build scripts to enable CI builds. The `-cache` section is specified to depend on `appveyor.yml`, so the cache will be invalidated if `appveyor.yml` is changed.
4. Ship `netcoreapp` reference assemblies with powershell to fix the issues in `Add-Type` (#2764). By default `Add-Type` will reference all those reference assemblies when compiling C# code. If `-ReferenceAssembly` is specified, then we search reference assemblies first, then the framework runtime assemblies, and lastly the loaded assemblies (possibly a third-party one that was already loaded).
5. `dotnet publish` generates executable on Unix platforms, but doesn't set "x" permission and thus it cannot execute. Currently, the "x" permission is set in the build script, `dotnet/cli` issue [#6286](https://github.com/dotnet/cli/issues/6286) is tracking this.
6. Replace the use of some APIs with the ones that take `SecureString`.
7. osx.10.12 is required to update to `netcoreapp2.0` because `dotnet-cli` 2.0.0-preview only works on osx.10.12.
8. Add dependency to `System.ValueTuple` to work around a ambiguous type identity issue in coreclr. The issue is tracked by `dotnet/corefx` [#17797](https://github.com/dotnet/corefx/issues/17797). When moving to newer version of `netcoreapp2.0`, we need to verify if this dependency is still needed.
- Fix incorrect handling of 'GetCurrentProcess()' API return value in 'IsWow64()' function (MainEntry.cpp)
- Remove unused INVALID_HANDLE_VALUE field from 'ProcessNativeMethods' class (Process.cs)
There are cases like using PowerShell via Puppet where the account being
used does not have a home directory. Updated PowerShell to use a process
specific temporary directory if HOME, CONFIG, CACHE, and DATA directories
are not available. Temporary directory is removed when last runspace
is disposed.
Based on the conclusion in discussion #3248, this PR fix "OEM" and "Default" encoding in powershell.
1. OEM -- We need an internal fix because .Net Core has no "OEM" encoding.
The fix gives the following PowerShell Core behavior on "OEM" encoding:
- on Windows - as Windows PowerShell based on GetOEMCP() and legacy encodings (System.Text.Encoding.CodePages)
- on Unix - the same as default encoding in PowerShell Core
2. Default -- We need internal fix because CoreFX `Encoding.default` for all platforms is UTF8.
Until we complete the `Encoding RFC` we should restore behavior as in Windows PowerShell.
The fix gives the following PowerShell Core behavior on "Default" encoding:
- on Windows - as Windows PowerShell based on GetACP() and legacy encodings (System.Text.Encoding.CodePages)
- on Unix - UTF-8 without BOM (We are expecting that the same will be [in CoreFX](https://github.com/dotnet/coreclr/issues/10643))
Dan Travison