Introduction
If you ever came across a need to invoke a request that needs to oblige to Multipart/form-data standard in PowerShell, you probably got to know quite quickly that none of commonly used cmdlets do support it.
Both, Invoke-WebRequest and Invoke-RestMethod are unaware on how to format the request body in order to comply to Multipart/form-data standard. In order to succeed in your intent, you probably Googled this out, and the best you could find are a couple of questions on StackOverflow that are showing on how to manually set the message body and then invoke your call with one of above mentioned cmdlets. I did it too. Still I was not satisfied by the answers I found. Although what proposed could be written in a slightly better way, still it was not optimal. First of all the memory footprint that it has. In case you are transmitting a large amount of data, all of the objects are loaded into memory.
After trying this approach, I dug into the .Net framework and found out that all that we need is right there. True, it is only available on .Net 4.5, and if that is not an obstacle, I would advise you to follow that approach. HttpClient and connected classes do have all of the necessary to support Multipart/form-data standard. Initially I created a prototype in a form of a small C# application and once succeed I translated all of that into PowerShell.
In this post I will show you both, PowerShell approach by formatting the request body and transmitting via Invoke-WebRequest cmdlet, as using an approach that is less resource intensive and is based on HttpClient .Net class.
Lets start.
Throw everything in
Multipart/form-data standard requires the message body to follow a certain structure. You can read more about it in the RFC 2388.
Aside the body structure, there is a concept of a boundary. Boundary is nothing else than an unique string that will be used for delimitation purposes inside the message body. It will be specified in the request header and used inside the message body to circumvent files that we intend to transmit.
As I mentioned earlier, this approach constructs the body string in a variable and once ready invokes the Invoke-WebRequest cmdlet to execute the call.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
function Invoke-MultipartFormDataUpload { [CmdletBinding()] PARAM ( [string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$InFile, [string]$ContentType, [Uri][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Uri, [System.Management.Automation.PSCredential]$Credential ) BEGIN { if (-not (Test-Path $InFile)) { $errorMessage = ("File {0} missing or unable to read." -f $InFile) $exception = New-Object System.Exception $errorMessage $errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'MultipartFormDataUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $InFile $PSCmdlet.ThrowTerminatingError($errorRecord) } if (-not $ContentType) { Add-Type -AssemblyName System.Web $mimeType = [System.Web.MimeMapping]::GetMimeMapping($InFile) if ($mimeType) { $ContentType = $mimeType } else { $ContentType = "application/octet-stream" } } } PROCESS { $fileName = Split-Path $InFile -leaf $boundary = [guid]::NewGuid().ToString() $fileBin = [System.IO.File]::ReadAllBytes($InFile) $enc = [System.Text.Encoding]::GetEncoding("iso-8859-1") $template = @' --{0} Content-Disposition: form-data; name="fileData"; filename="{1}" Content-Type: {2} {3} --{0}-- '@ $body = $template -f $boundary, $fileName, $ContentType, $enc.GetString($fileBin) try { return Invoke-WebRequest -Uri $Uri ` -Method Post ` -ContentType "multipart/form-data; boundary=$boundary" ` -Body $body ` -Credential $Credential } catch [Exception] { $PSCmdlet.ThrowTerminatingError($_) } } END { } } |
As you can see, we are preparing “by hand” the correct message body and executing Invoke-WebRequest in order to send our file. It is not efficient at all as it encodes the whole file as a string and keeps it in memory. Consider transmitting large files and make yourself an idea on the memory footprint of this approach. Aside that, it is just not elegant!
Wait a moment, what a waste
Previously shown method is as said, not elegant and waists resources. A better approach is to let the HttpClient class handle everything. It is way more efficient as it uses streams in order to read our file and transfers it to http StreamContent. It is less resource intensive and a more elegant solution.
Let’s see now how our refactored Invoke-MultipartFormDataUpload cmdlet looks like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 |
function Invoke-MultipartFormDataUpload { [CmdletBinding()] PARAM ( [string][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$InFile, [string]$ContentType, [Uri][parameter(Mandatory = $true)][ValidateNotNullOrEmpty()]$Uri, [System.Management.Automation.PSCredential]$Credential ) BEGIN { if (-not (Test-Path $InFile)) { $errorMessage = ("File {0} missing or unable to read." -f $InFile) $exception = New-Object System.Exception $errorMessage $errorRecord = New-Object System.Management.Automation.ErrorRecord $exception, 'MultipartFormDataUpload', ([System.Management.Automation.ErrorCategory]::InvalidArgument), $InFile $PSCmdlet.ThrowTerminatingError($errorRecord) } if (-not $ContentType) { Add-Type -AssemblyName System.Web $mimeType = [System.Web.MimeMapping]::GetMimeMapping($InFile) if ($mimeType) { $ContentType = $mimeType } else { $ContentType = "application/octet-stream" } } } PROCESS { Add-Type -AssemblyName System.Net.Http $httpClientHandler = New-Object System.Net.Http.HttpClientHandler if ($Credential) { $networkCredential = New-Object System.Net.NetworkCredential @($Credential.UserName, $Credential.Password) $httpClientHandler.Credentials = $networkCredential } $httpClient = New-Object System.Net.Http.Httpclient $httpClientHandler $packageFileStream = New-Object System.IO.FileStream @($InFile, [System.IO.FileMode]::Open) $contentDispositionHeaderValue = New-Object System.Net.Http.Headers.ContentDispositionHeaderValue "form-data" $contentDispositionHeaderValue.Name = "fileData" $contentDispositionHeaderValue.FileName = (Split-Path $InFile -leaf) $streamContent = New-Object System.Net.Http.StreamContent $packageFileStream $streamContent.Headers.ContentDisposition = $contentDispositionHeaderValue $streamContent.Headers.ContentType = New-Object System.Net.Http.Headers.MediaTypeHeaderValue $ContentType $content = New-Object System.Net.Http.MultipartFormDataContent $content.Add($streamContent) try { $response = $httpClient.PostAsync($Uri, $content).Result if (!$response.IsSuccessStatusCode) { $responseBody = $response.Content.ReadAsStringAsync().Result $errorMessage = "Status code {0}. Reason {1}. Server reported the following message: {2}." -f $response.StatusCode, $response.ReasonPhrase, $responseBody throw [System.Net.Http.HttpRequestException] $errorMessage } return $response.Content.ReadAsStringAsync().Result } catch [Exception] { $PSCmdlet.ThrowTerminatingError($_) } finally { if($null -ne $httpClient) { $httpClient.Dispose() } if($null -ne $response) { $response.Dispose() } } } END { } } |
Conclusion
There is still space for improvement, however in my case it is good enough and I will not continue extending this cmdlet. If you think this is not enough or good enough, please extend it and let me know. As an idea, you may make possible passing multiple files via the pipeline and execute the upload only once, in the END step. In that way it may be more efficient and flexible. Also supporting some other common parameters as Proxy would be beneficial to others. If any let me know, I would be happy to hear from you in comments.
Thank you for trying! Amazing issue you are trying to solve!
When I tried your function, I received:
Invoke-MultipartFormDataUpload : You cannot call a method on a null-valued expression.
At line:1 char:1
Also:
– not sure about the credential part for basic auth
– unsure of how to add more headers using this approach
I understand you won’t work on it anymore, but you invested the time to write, so I will invest the time to provide some feedback. Maybe it will help, maybe not.
Thank you!
Hi,
How did you tried to invoke it? In this post you can find an example of usage and invokation of this cmdlet, http://blog.majcica.com/2016/01/15/uploading-xl-deploy-dar-package-via-powershell/.
If credentials are passed, pre-authentication is made and yes, it supports basic auth.
What headers you are interested in, request headers, or content headers?
Cheers
Hi Mario,
could you please explain how to add a custom to the main http command when using the .net edition? 🙂
Thanks!
Hi Maigel,
what do you mean with a custom? What are you trying to achieve?
Cheers,
Mario
If you want to pass parameters as well as files to the webserver you can do (added because I struggled a lot with it:
$multiContent = New-Object System.Net.Http.MultipartFormDataContent #$boundary
ForEach($keyvaluepair in $parameters.GetEnumerator())
{
$stringcontent = New-Object System.Net.Http.StringContent $keyvaluepair.value
$name ='”{0}”‘ -f $keyvaluepair.key
$multiContent.Add($stringcontent,$name);
}
$multiContent.Add($streamContent,'”File”‘,'”test.txt”‘);
Great, thanks for your help!
For people that are trying to use the Tableau REST Api in PowerShell, the below code snipped works if you have a twb file without a datasource. If you get “bad request” then most probably you have Data Sources in the twb file that cannot be found on the Tableau Server.
Add-Type -AssemblyName System.Net.Http[System.Net.Http.Httpclient]$httpClient=New-Object -TypeName System.Net.Http.Httpclient
$httpClient.DefaultRequestHeaders.Clear()
$httpClient.DefaultRequestHeaders.Add("X-Tableau-Auth",$strToken)
$httpclient.DefaultRequestHeaders.ExpectContinue = $false
[System.Net.Http.MultipartFormDataContent]$MPcontent = New-Object -TypeName System.Net.Http.MultipartFormDataContent -ArgumentList ($guidBS.ToString())
[System.Net.Http.StringContent]$stringContent=New-Object -TypeName System.Net.Http.StringContent -ArgumentList ($tmpXML.InnerXml.ToString())
[System.Net.Http.Headers.ContentDispositionHeaderValue]$contentDispositionHeaderValue=New-Object -TypeName System.Net.Http.Headers.ContentDispositionHeaderValue -ArgumentList "form-data"
$contentDispositionHeaderValue.Name="
"request_payload""$stringContent.Headers.ContentDisposition = $contentDispositionHeaderValue
$stringContent.Headers.ContentType=[System.Net.Http.Headers.MediaTypeHeaderValue]::Parse("text/xml")
$MPcontent.Add($stringContent)
[Byte[]]$BytArray=[System.IO.File]::ReadAllBytes($strFileName)
[System.IO.memorystream]$memStream=New-Object -TypeName System.IO.MemoryStream -ArgumentList (,$BytArray)
$memstream.Seek(0,[System.IO.SeekOrigin]::Begin)
[System.Net.Http.StreamContent]$streamContent=New-Object -TypeName System.Net.Http.StreamContent -ArgumentList $memStream
$streamContent.Headers.ContentDisposition=New-Object -TypeName System.Net.Http.Headers.ContentDispositionHeaderValue -ArgumentList "form-data"
$streamContent.Headers.ContentDisposition.Name="
"tableau_workbook""$streamContent.Headers.ContentDisposition.FileName="
""+(Split-Path -Path $strFileName -leaf)+"""$streamContent.Headers.ContentType=[System.Net.Http.Headers.MediaTypeHeaderValue]::Parse("application/octet-stream")
$MPcontent.Add($streamContent)
$MPcontent.Headers.Clear()
$MPcontent.Headers.TryAddWithoutValidation("Content-Type", "multipart/mixed; boundary=$($guidBS.ToString())")
$resp=$httpClient.PostAsync("https://your.server/api/2.2/sites/d0356794-bb9d-4c5c-b43d-ec384a2baf5a/workbooks?overwrite=true",$MPcontent)
$resp.Wait()
Write-Host $resp.Result.ToString()