Get-CloudManagementTools

This PowerShell script can be used to download some of the common cloud (Azure and AWS) management tools available on Windows Platform.

Earlier I had seen Download and Install SharePoint 2013 Prerequisites on Windows Server 2012.  However this technet script uses Start-BitsTransfer cmdlet from BitsTransfer module. I was toying with similar idea while working on my last workshop (Cloud Automation Using PowerShell) preparation. Since some of the URLs do redirect the Start-BitsTransfer wasn’t working for me initially.

I started with simple Invoke-WebRequest cmdlet and added second variation to handle redirected URLs and pass it to Start-BitsTransfer. So let’s go through the script and output for the both the approaches.

I am using Downloads directory to store the files. For demo I have created CloudTools folder inside. I already have AWS Tools for Windows PowerShell downloaded in the folder.

I am already into the working directory where the script is saved.

Started the script execution and you can notice it finds AWS Tools For Windows PowerShell and skips the download for it.

On my Windows Server 2012 R2 VM it created a CloudTools folder which was missing before download starts.

# The folder location where the downloads will be saved
$DestinationFolder = "$ENV:homedrive$env:homepath\Downloads\CloudTools"
If (!(Test-Path $DestinationFolder)){
 New-Item $DestinationFolder -ItemType Directory -Force
}

Download is in progress…

Before this, I have created hash table with Download URL and File Name After Downloaded

# Specify download url's for various cloud management tools. Do not change unless Microsoft changes the downloads themselves in future
$Downloads = @{
 # Latest Azure PowerShell
 # Github: https://github.com/Azure/azure-powershell/releases/latest
 # WebPI: https://www.microsoft.com/web/handlers/webpi.ashx/getinstaller/WindowsAzurePowershellGet.3f.3f.3fnew.appids
 "https://aka.ms/azure-powershellget2" = "$DestinationFolder\Azure-PowerShell.msi"; 
 # Latest Azure Storage Explorer
 "https://go.microsoft.com/fwlink/?LinkId=708343" = "$DestinationFolder\StorageExplorer.exe";
 # Latest AZCopy
 "http://aka.ms/downloadazcopy" = "$DestinationFolder\MicrosoftAzureStorageTools.msi";
 # Latest Azure CLI
 "http://aka.ms/webpi-azure-cli" = "$DestinationFolder\Azure-cli.msi";
 # Azure Storage Emulator
 "https://go.microsoft.com/fwlink/?LinkId=717179&clcid=0x409" = "$DestinationFolder\MicrosoftAzureStorageEmulator.msi";
 # Latest AWS Tools For Windows PowerShell
 "http://sdk-for-net.amazonwebservices.com/latest/AWSToolsAndSDKForNet.msi" = "$DestinationFolder\AWSToolsAndSDKForNet.msi";
 }

Approach 1 – using Invoke-WebRequest cmdlet ,

Then I have a function which loop through the elements in downloads hash table using Invoke-WebRequest cmdlet,

function DownloadFiles(){ 
 Write-Host ""
 Write-Host "====================================================================="
 Write-Host " Downloading Cloud Management Tools for Azure & AWS" 
 Write-Host "====================================================================="
 
 $ReturnCode = 0

$Downloads.GetEnumerator() | ForEach-Object { 
 $DownloadURL = $_.get_key()
 $Filespec = $_.get_value()
 # Get the file name based on the portion of the file path after the last slash 
 $FilePath = Split-Path $Filespec
 $FileName = Split-Path $Filespec -Leaf
 Write-Host "DOWNLOADING: $FileName"
 Write-Host " FROM: $DownloadURL"
 Write-Host " TO: $FilePath"
 
 Try 
 { 
 # Check if file already exists 
 If (!(Test-Path "$Filespec")) 
 { 
 # Begin download 
 Invoke-WebRequest -Uri $DownloadURL -OutFile "$Filespec" -ErrorVariable err
 If ($err) {Throw ""} 
 Write-Host " STATUS: Downloaded"
 Write-Host
 } 
 Else 
 { 
 Write-Host " STATUS: Already exists. Skipping." 
 Write-Host
 } 
 } 
 Catch 
 { 
 $ReturnCode = -1
 Write-Warning " AN ERROR OCCURRED DOWNLOADING `'$FileName`'" 
 Write-Error $_
 Break 
 }
} 
 return $ReturnCode 
}

and the downloads completed.

Approach 2 – using Start-BitsTransfer cmdlet,

First a function Get-RedirectedUrl to get redirected URL

# This function gets the end URL to avoid redirection in URL breaking the Start-BitsTransfer
# Courtesy: http://www.powershellmagazine.com/2013/01/29/pstip-retrieve-a-redirected-url/
Function Get-RedirectedUrl {
 Param (
 [Parameter(Mandatory=$true)]
 [String]$url
 )
 $request = [System.Net.WebRequest]::Create($url)
 $request.AllowAutoRedirect=$true
 try{
 $response=$request.GetResponse()
 $response.ResponseUri.AbsoluteUri
 $response.Close()
 }
 catch{
 “ERROR: $_”
 }
}

Then I have a function which loop through the elements in downloads hash table using Start-BitsTransfer cmdlet,

# Import Required Modules: BITS is used for file transfer
Import-Module BitsTransfer

function DownloadFiles(){ 
 Write-Host ""
 Write-Host "====================================================================="
 Write-Host " Downloading Cloud Management Tools for Azure & AWS" 
 Write-Host "====================================================================="
 
 $ReturnCode = 0

$Downloads.GetEnumerator() | ForEach-Object { 
 $DownloadURL = Get-RedirectedUrl -URL $_.get_key()
 $Filespec = $_.get_value()
 # Get the file name based on the portion of the file path after the last slash 
 $FilePath = Split-Path $Filespec
 $FileName = Split-Path $Filespec -Leaf
 Write-Host "DOWNLOADING: $FileName"
 Write-Host " FROM: $DownloadURL"
 Write-Host " TO: $FilePath"
 
 Try 
 { 
 # Check if file already exists 
 If (!(Test-Path "$Filespec")) 
 { 
 # Begin download 
 Start-BitsTransfer -Source $DownloadURL -Destination "$Filespec" -DisplayName "Downloading `'$FileName`' to $FilePath" -Priority High -Description "From $DownloadURL..." -ErrorVariable err 
 If ($err) {Throw ""} 
 Write-Host " STATUS: Downloaded"
 Write-Host
 } 
 Else 
 { 
 Write-Host " STATUS: Already exists. Skipping." 
 Write-Host
 } 
 } 
 Catch 
 { 
 $ReturnCode = -1
 Write-Warning " AN ERROR OCCURRED DOWNLOADING `'$FileName`'" 
 Write-Error $_
 Break 
 }

} 
 return $ReturnCode 
}

while downloading the progress bar looks different,

I had deleted 2 files and ran the script again and following is the output,

If you want to know more about BITSTransfer & PowerShell you can refer this How to copy files with BITS using PowerShell

You can try both variation scripts by downloading from my (cloudscripts) github. I will be updating it later for more tools.

That’s it for now.

AWS PowerShell and PowerShell Profile

In my last post Login to Azure PowerShell Options talked about Azure. In this one I will share about AWS.

Before we begin it would be good to get familiar with basic AWS PowerShell and connectivity using PowerShell. Following resources are helpful.

• Setting up the AWS Tools for Windows PowerShell
• Using AWS Credentials
• Handling Credentials with AWS Tools for Windows PowerShell

Goal

I want to start using AWS Cmdlets as soon as I have PowerShell launched.

Solution I am using,

Few checks before updating Default Windows PowerShell Profile for Current User.

To find cmdlets which work with the AWS credentials

PS C:\> Get-Command *aws*cred*

To find cmdlets which work with the AWS defaults

PS C:\> Get-Command *aws*defa*

Just to ensure there are not configurations loaded by PowerShell Current User Profile

PS C:\> Get-Variable *aws*

I am expecting “StoredAWSCredentials” and “StoredAWSRegion” to be loaded and available and there is no file available at C:\Users\{UserName}\AppData\Local\AWSToolkit

To set credentials I have added following 5 lines into my PowerShell profile.

$awRegion = 'ap-south-1'
$awAdmin = Import-Csv -Path "C:\Users\{username}\OneDrive\Documents\Cloud\MyAWS\credentials.csv"
Set-DefaultAWSRegion $awRegion
Set-AWSCredentials -AccessKey $awAdmin.'Access Key Id' -SecretKey $awAdmin.'Secret Access Key' -StoreAs "MSAWPS"
Initialize-AWSDefaults -ProfileName "MSAWPS"

I have a credentials csv file in which access key and secret key for my admin account is stored. I am importing that from my onedrive path and using it in Set-AWSCredentials command.

Now launch PowerShell as Administrator and notice few changes. First there a file called RegisteredAccounts.json created at

C:\Users\{UserName}\AppData\Local\AWSToolkit

Open it in notepad or your favorite tool. I am using Visual Studio Code. The file looks like below,

This time some AWS configurations are loaded by PowerShell Current User Profile

PS C:\> Get-Variable *aws*

You can verify that credentials are loaded and available for use.

PS C:\> Get-AWSCredentials -ListProfiles

You can directly start using AWS cmdlets. Notice I don’t have to supply the credentials or profile as it’s already set and initialized as default.

PS C:\> Get-IAMUser

PS C:\> Get-IAMUsers

That’s it for now.