PowerShell tips and tricks – Multipart/form-data requests

Mario Majcica January 13, 2016 20 Comments

Introduction

If you ever came across a need to invoke a request that needs to oblige to Multipart/form-data standard in PowerShell, you probably got to know quite quickly that none of commonly used cmdlets do support it.
Both, Invoke-WebRequest and Invoke-RestMethod are unaware on how to format the request body in order to comply to Multipart/form-data standard. In order to succeed in your intent, you probably Googled this out, and the best you could find are a couple of questions on StackOverflow that are showing on how to manually set the message body and then invoke your call with one of above mentioned cmdlets. I did it too. Still I was not satisfied by the answers I found. Although what proposed could be written in a slightly better way, still it was not optimal. First of all the memory footprint that it has. In case you are transmitting a large amount of data, all of the objects are loaded into memory.
After trying this approach, I dug into the .Net framework and found out that all that we need is right there. True, it is only available on .Net 4.5, and if that is not an obstacle, I would advise you to follow that approach. HttpClient and connected classes do have all of the necessary to support Multipart/form-data standard. Initially I created a prototype in a form of a small C# application and once succeed I translated all of that into PowerShell.

In this post I will show you both, PowerShell approach by formatting the request body and transmitting via Invoke-WebRequest cmdlet, as using an approach that is less resource intensive and is based on HttpClient .Net class.

Lets start.

Throw everything in

Multipart/form-data standard requires the message body to follow a certain structure. You can read more about it in the RFC 2388.
Aside the body structure, there is a concept of a boundary. Boundary is nothing else than an unique string that will be used for delimitation purposes inside the message body. It will be specified in the request header and used inside the message body to circumvent files that we intend to transmit.

As I mentioned earlier, this approach constructs the body string in a variable and once ready invokes the Invoke-WebRequest cmdlet to execute the call.

function Invoke-MultipartFormDataUpload
{
    [CmdletBinding()]
    PARAM
    (
        [string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$InFile,
        [string]$ContentType,
        [Uri][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Uri,
        [System.Management.Automation.PSCredential]$Credential
    )
    BEGIN
    {
        if (-not (Test-Path $InFile))
        {
            $errorMessage = ("File {0} missing or unable to read." -f $InFile)
            $exception =  New-Object System.Exception $errorMessage
			$errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'MultipartFormDataUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $InFile
			$PSCmdlet.ThrowTerminatingError($errorRecord)
        }

        if (-not $ContentType)
        {
            Add-Type -AssemblyName System.Web

            $mimeType = [System.Web.MimeMapping]::GetMimeMapping($InFile)
            
            if ($mimeType)
            {
                $ContentType = $mimeType
            }
            else
            {
                $ContentType = "application/octet-stream"
            }
        }
    }
    PROCESS
    {
		$fileName = Split-Path $InFile -leaf
		$boundary = [guid]::NewGuid().ToString()
		
    	$fileBin = [System.IO.File]::ReadAllBytes($InFile)
	    $enc = [System.Text.Encoding]::GetEncoding("iso-8859-1")

	    $template = @'
--{0}
Content-Disposition: form-data; name="fileData"; filename="{1}"
Content-Type: {2}

{3}
--{0}--

'@

        $body = $template -f $boundary, $fileName, $ContentType, $enc.GetString($fileBin)

		try
		{
			return Invoke-WebRequest -Uri $Uri `
									 -Method Post `
									 -ContentType "multipart/form-data; boundary=$boundary" `
									 -Body $body `
									 -Credential $Credential
		}
		catch [Exception]
		{
			$PSCmdlet.ThrowTerminatingError($_)
		}
    }
    END { }
}

As you can see, we are preparing “by hand” the correct message body and executing Invoke-WebRequest in order to send our file. It is not efficient at all as it encodes the whole file as a string and keeps it in memory. Consider transmitting large files and make yourself an idea on the memory footprint of this approach. Aside that, it is just not elegant!

Wait a moment, what a waste

Previously shown method is as said, not elegant and waists resources. A better approach is to let the HttpClient class handle everything. It is way more efficient as it uses streams in order to read our file and transfers it to http StreamContent. It is less resource intensive and a more elegant solution.

Let’s see now how our refactored Invoke-MultipartFormDataUpload cmdlet looks like:

function Invoke-MultipartFormDataUpload
{
    [CmdletBinding()]
    PARAM
    (
        [string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$InFile,
        [string]$ContentType,
        [Uri][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Uri,
        [System.Management.Automation.PSCredential]$Credential
    )
    BEGIN
    {
        if (-not (Test-Path $InFile))
        {
            $errorMessage = ("File {0} missing or unable to read." -f $InFile)
            $exception =  New-Object System.Exception $errorMessage
			$errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'MultipartFormDataUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $InFile
			$PSCmdlet.ThrowTerminatingError($errorRecord)
        }

        if (-not $ContentType)
        {
            Add-Type -AssemblyName System.Web

            $mimeType = [System.Web.MimeMapping]::GetMimeMapping($InFile)
            
            if ($mimeType)
            {
                $ContentType = $mimeType
            }
            else
            {
                $ContentType = "application/octet-stream"
            }
        }
    }
    PROCESS
    {
        Add-Type -AssemblyName System.Net.Http

		$httpClientHandler = New-Object System.Net.Http.HttpClientHandler

        if ($Credential)
        {
		    $networkCredential = New-Object System.Net.NetworkCredential @($Credential.UserName, $Credential.Password)
		    $httpClientHandler.Credentials = $networkCredential
        }

        $httpClient = New-Object System.Net.Http.Httpclient $httpClientHandler

        $packageFileStream = New-Object System.IO.FileStream @($InFile, [System.IO.FileMode]::Open)
        
		$contentDispositionHeaderValue = New-Object System.Net.Http.Headers.ContentDispositionHeaderValue "form-data"
	    $contentDispositionHeaderValue.Name = "fileData"
		$contentDispositionHeaderValue.FileName = (Split-Path $InFile -leaf)

        $streamContent = New-Object System.Net.Http.StreamContent $packageFileStream
        $streamContent.Headers.ContentDisposition = $contentDispositionHeaderValue
        $streamContent.Headers.ContentType = New-Object System.Net.Http.Headers.MediaTypeHeaderValue $ContentType
        
        $content = New-Object System.Net.Http.MultipartFormDataContent
        $content.Add($streamContent)

        try
        {
			$response = $httpClient.PostAsync($Uri, $content).Result

			if (!$response.IsSuccessStatusCode)
			{
				$responseBody = $response.Content.ReadAsStringAsync().Result
				$errorMessage = "Status code {0}. Reason {1}. Server reported the following message: {2}." -f $response.StatusCode, $response.ReasonPhrase, $responseBody

				throw [System.Net.Http.HttpRequestException] $errorMessage
			}

			return $response.Content.ReadAsStringAsync().Result
        }
        catch [Exception]
        {
			$PSCmdlet.ThrowTerminatingError($_)
        }
        finally
        {
            if($null -ne $httpClient)
            {
                $httpClient.Dispose()
            }

            if($null -ne $response)
            {
                $response.Dispose()
            }
        }
    }
    END { }
}

Conclusion

There is still space for improvement, however in my case it is good enough and I will not continue extending this cmdlet. If you think this is not enough or good enough, please extend it and let me know. As an idea, you may make possible passing multiple files via the pipeline and execute the upload only once, in the END step. In that way it may be more efficient and flexible. Also supporting some other common parameters as Proxy would be beneficial to others. If any let me know, I would be happy to hear from you in comments.