Powershell’s equivalent to Bash’s process substitution

powershell

Bash has <(..) for process substitution. What is Powershell's equivalent?

I know there is $(...), but it returns a string, while <(..) returns a file the outer command can read from, which is what it expects.

I'm also not looking for a pipe based solution, but something I can stick in the middle of the command line.

Best Answer

This answer is NOT for you, if you:
- rarely, if ever, need to use external CLIs (which is generally worth striving for - PowerShell-native commands play much better together and have no need for such a feature).
- aren't familiar with Bash's process substitution.
This answer IS for you, if you:
- frequently use external CLIs (whether out of habit or due to lack of (good) PowerShell-native alternatives), especially while writing scripts.
- are used to and appreciate what Bash's process substitution can do.
- Update: Now that PowerShell is supported on Unix platforms too, this feature is of increasing interest - see this feature request on GitHub, which suggests that PowerShell implement a feature akin to process substitution.

In the Unix world, in Bash/Ksh/Zsh, a process substitution is offers treating command output as if it were a temporary file that cleans up after itself; e.g. cat <(echo 'hello'), where cat sees the output from the echo command as the path of a temporary file containing the command output.

While PowerShell-native commands have no real need for such a feature, it can be handy when dealing with external CLIs.

Emulating the feature in PowerShell is cumbersome, but may be worth it, if you find yourself needing it often.

Picture a function named cf that accepts a script block, executes the block and writes its output to a temp. file created on demand, and returns the temp. file's path; e.g.:

 findstr.exe "Windows" (cf { Get-ChildItem c:\ }) # findstr sees the temp. file's path.

This is a simple example that doesn't illustrate the need for such a feature well. Perhaps a more convincing scenario is the use of psftp.exe for SFTP transfers: its batch (automated) use requires providing an input file containing the desired commands, whereas such commands can easily be created as a string on the fly.

So as to be as widely compatible with external utilities as possible, the temp. file should use UTF-8 encoding without a BOM (byte-order mark) by default, although you can request a UTF-8 BOM with -BOM, if needed.

Unfortunately, the automatic cleanup aspect of process substitutions cannot be directly emulated, so an explicit cleanup call is needed; cleanup is performed by calling cf without arguments:

  • For interactive use, you can automate the cleanup by adding the cleanup call to your prompt function as follows (the prompt function returns the prompt string, but can also be used to perform behind-the-scenes commands every time the prompt is displayed, similar to Bash's $PROMPT_COMMAND variable); for availability in any interactive session, add the following as well as the definition of cf below to your PowerShell profile:

    "function prompt { cf 4>`$null; $((get-item function:prompt).definition) }" |
  Invoke-Expression

  • For use in scripts, to ensure that cleanup is performed, the block that uses cf - potentially the whole script - needs to be wrapped in a try / finally block, in which cf without arguments is called for cleanup:

# Example
try {

  # Pass the output from `Get-ChildItem` via a temporary file.
  findstr.exe "Windows" (cf { Get-ChildItem c:\ })

  # cf() will reuse the existing temp. file for additional invocations.
  # Invoking it without parameters will delete the temp. file.

} finally {
  cf  # Clean up the temp. file.
}

Here's the implementation: advanced function ConvertTo-TempFile and its succinct alias, cf:

Note: The use of New-Module, which requires PSv3+, to define the function via a dynamic module ensures that there can be no variable conflicts between the function parameters and variables referenced inside the script block passed. 

$null = New-Module {  # Load as dynamic module
  # Define a succinct alias.
  set-alias cf ConvertTo-TempFile
  function ConvertTo-TempFile {
    [CmdletBinding(DefaultParameterSetName='Cleanup')]
    param(
        [Parameter(ParameterSetName='Standard', Mandatory=$true, Position=0)]
        [ScriptBlock] $ScriptBlock
      , [Parameter(ParameterSetName='Standard', Position=1)]
        [string] $LiteralPath
      , [Parameter(ParameterSetName='Standard')]
        [string] $Extension
      , [Parameter(ParameterSetName='Standard')]
        [switch] $BOM
    )

    $prevFilePath = Test-Path variable:__cttfFilePath
    if ($PSCmdlet.ParameterSetName -eq 'Cleanup') {
      if ($prevFilePath) { 
        Write-Verbose "Removing temp. file: $__cttfFilePath"
        Remove-Item -ErrorAction SilentlyContinue $__cttfFilePath
        Remove-Variable -Scope Script  __cttfFilePath
      } else {
        Write-Verbose "Nothing to clean up."
      }
    } else { # script block specified
      if ($Extension -and $Extension -notlike '.*') { $Extension = ".$Extension" }
      if ($LiteralPath) {
        # Since we'll be using a .NET framework classes directly, 
        # we must sync .NET's notion of the current dir. with PowerShell's.
        [Environment]::CurrentDirectory = $pwd
        if ([System.IO.Directory]::Exists($LiteralPath)) { 
          $script:__cttfFilePath = [IO.Path]::Combine($LiteralPath, [IO.Path]::GetRandomFileName() + $Extension)
          Write-Verbose "Creating file with random name in specified folder: '$__cttfFilePath'."
        } else { # presumptive path to a *file* specified
          if (-not [System.IO.Directory]::Exists((Split-Path $LiteralPath))) {
            Throw "Output folder '$(Split-Path $LiteralPath)' must exist."
          }
          $script:__cttfFilePath = $LiteralPath
          Write-Verbose "Using explicitly specified file path: '$__cttfFilePath'."
        }
      } else { # Create temp. file in the user's temporary folder.
        if (-not $prevFilePath) { 
          if ($Extension) {
            $script:__cttfFilePath = [IO.Path]::Combine([IO.Path]::GetTempPath(), [IO.Path]::GetRandomFileName() + $Extension)
          } else {
            $script:__cttfFilePath = [IO.Path]::GetTempFilename() 
          }
          Write-Verbose "Creating temp. file: $__cttfFilePath"
        } else {
          Write-Verbose "Reusing temp. file: $__cttfFilePath"      
        }
      }
      if (-not $BOM) { # UTF8 file *without* BOM
        # Note: Out-File, sadly, doesn't support creating UTF8-encoded files 
        #       *without a BOM*, so we must use the .NET framework.
        #       [IO.StreamWriter] by default writes UTF-8 files without a BOM.
        $sw = New-Object IO.StreamWriter $__cttfFilePath
        try {
            . $ScriptBlock | Out-String -Stream | % { $sw.WriteLine($_) }
        } finally { $sw.Close() }
      } else { # UTF8 file *with* BOM
        . $ScriptBlock | Out-File -Encoding utf8 $__cttfFilePath
      }
      return $__cttfFilePath
    }
  }
}

Note the ability to optionally specify an output [file] path and/or filename extension.

Related Topic