How do I practice PowerShell?

“How do I stay in touch with PowerShell practice?”
Yes, this is one question is surely asked in all my PowerShell batches.
It’s obvious that people get overwhelmed with capabilities of PowerShell and wonder how do I keep practising. The biggest question is where do I start?

Well!!! PowerShell can answer that, PowerShell can solve that for you with following steps,

  1. Update-Help
  2. $PROFILE (Set up your PowerShell profile)
  3. Get-Help (Automated)

Lets try to understand more about the solution.

Step 1 – Update-Help

  • Update-Help cmdlet downloads the newest help files for Windows PowerShell modules and installs them on your computer.
  • You do not have to restart Windows PowerShell to make the change effective.
  • With this you will also get several about_* topics downloaded on your computer at C:\Windows\System32\WindowsPowerShell\v1.0\en-US and inside modules where latest help manuals are available.
  • This will lay the foundation for further steps.

Step 2 – $PROFILE

  • The $Profile automatic variable stores the paths to the Windows PowerShell profiles that are available in the current session.
  • A Windows PowerShell profile is a script that runs when Windows PowerShell starts.
  • You can use the profile as a logon script to customise the environment.
  • You can add commands, aliases, functions, variables, snap-ins, modules, and Windows PowerShell drives.
  • You can also add other session-specific elements to your profile so they are available in every session without having to import or re-create them.
  • Windows PowerShell supports several profiles for users and host programs. However, it does not create the profiles for you by default.
  • The Windows PowerShell console supports the following basic profile files. The profiles are listed in precedence order. The first profile has the highest precedence.

 

Different PowerShell Profiles

Different PowerShell Profiles

HOW TO CREATE A PROFILE :

To create a profile for the current user in the current Windows PowerShell host application, use the following command:

if (!(test-path $profile)) {

new-item -type file -path $profile -force

}

HOW TO EDIT A PROFILE:

You can use either of the following options at PowerShell prompt

  • Option 1
notepad $profile

opens profile file in notepad to edit.

  • Option 2
ISE $profile

opens profile file in PowerShell ISE (Integrated Scripting Environment) to edit.

ADD FOLLOWING CODE TO A PROFILE:

help about_* | 

ForEach-Object{

if (($_.Split())[0] -match 'about_'){

($_.Split())[0]

}

} |

Select-Object @{n='name';e={$_}} |

Get-Random |

Get-Help -ShowWindow

SAVE PROFILE CHANGES AND EXIT

Step 3 – Just Open PowerShell to Get-Help

The above code will execute every time you open PowerShell.

Technically,
- It pulls all the about topics
- Filters empty line and header row 
- From the output and just get the topic name as about_[topic] string
- Get-Random picks a random about_topic and passes to Get-Help
- Get-Help opens a topic in new window.
  • It will present you with one of the about_[topic] randomly. This will take out the burden of choosing topic.
  • Just make sure you read the presented topic at least for 5-10 minutes maximum.
  • I used to do this at least 3 times a day. Morning before starting work, just after lunch (before starting other work) and before I shutdown my computer at the end of the day.
  • So essentially minimum 15 minutes to maximum 30 minutes a day spent reading something about the PowerShell topics. Whenever I had extra time, I used to practice some of the examples from the about_topic.
  • This helped me register most of the topics in my head within 3-4 weeks of time.
  • Few years back, I was talking to someone on learning new skills and the person pointed out Rule Of 21 to me.
  • There are many articles if Google Or Bing it. Some of them even call it myth. I don’t want to talk PRO or CONS here about the same.
  • In my personal case it worked to gain good knowledge. Some of my participants also have responded that it was a helpful tip.

This slideshow requires JavaScript.

Food for thought: If you want to plan focused learning in every week, then you can extend the code. Have a starting and ending date. At the launch check if current dates falls in First, Second or Third week or more. Depending on that Get a random topic from the focused bucket.
i.e. Below command will select one command randomly from module NetAdapter. This was you can focus only on commands from selected module/area of learning.

Get-Command -Module NetAdapter | Get-Random | Select-Object Name

*Replace the Select-Object with Get-Help and you will get the help file.

That’s it for now.

If you like this tip do share your feedback. If you try this and find valuable, then do write me to with your experience.

All the best and enjoy PowerShell Scripting!!!

Download Ubuntu ISO using PowerShell

My previous post Unattended Ubuntu 16.04 Setup Using PowerShell was about building Ubuntu Server Virtual Machine completely in unattended mode on Hyper-V. To do that we need at least a server operating system ISO file of particular Ubuntu version. I wanted to plug this script/functions in earlier script however to keep it simple, I didn’t do that. I will try to merge these script soon to come up one single script later.

In this post, I will share how to get a particular Ubuntu ISO file using PowerShell script. You can download the complete script from my github repository.

This script has 4 functions as listed below,

  • Get-AvailableUbuntuVersions
  • Get-UbuntuDownloadUrl
  • Save-UbuntuISO
  • Test-Checkum

Since I have explained functions in script manual, I will just talk briefly about them here. For more information please refer the script documentation.

Get-AvailableUbuntuVersions – This is main function which gets called after running the script. This function reads http://releases.ubuntu.com/ and extracts the available version details. It then presents the available version to user and let user select from it.Using the choice of user it then calls Get-UbuntuDownloadUrl to build/identify the download URL for the selected version.

Get-UbuntuDownloadUrl – Generate the download URL. Based on the ubuntu version selected by user, this function generates the download URL for the iso and MD5SUMS file. Before exiting it calls the Save-UbuntuISO to start the download.

Save-UbuntuISO – Download the ISO file. This function gets called from inside Get-UbuntuDownloadUrl with 2 parameters download link and target location to save file locally. It uses BITS module to initiate, monitor and close the download of ISO file. Since the BITS job is set to Asynchronous this function will keep track of the progress of the job and ensure BITs transfer is resumed appropriately in case of failures.

Test-Checkum – Check the MD5Sum of the downloaded file. Once the download is complete, It gets the input parameters (sourceMD5SumsFileURL, sourcefile, targetFile) from Save-UbuntuISO for the iso file which is getting downloaded. It checks if the checksum matches and gives appropriate status message back to user.

Screenshots

This slideshow requires JavaScript.

Notes:

  • This script is tested on PowerShell 5, Windows 10 Pro version 1703 (creator update).
  • I expect it to work on PowerShell 3 and above without any issues.
  • Currently this script will not work on PowerShell for Linux as it needs BITS modules. I will be testing wget or other similar options for the Linux capability. Stay tuned if you are interested to see that update. Having said, I will be glad hear your updates/comments if you test and find the alternatives before I could try myself.

Unattended Ubuntu 16.04 Setup using PowerShell

It was back in 2003-04 when I had worked on linux with RedHat 2x & 3x, Fedora 2x and Knoppix 3x. After that for almost a decade I didn’t use linux.
In last few years (since late 2014) I have been playing Ubuntu releases as part of my experiments, learning DevOps tools and data science topics.
I have another laptop at home with Windows and Ubuntu dual boot which I keep switching between. At times when I am travelling, my preferred option is using Hyper-v virtual machines.
In this post, I will share my recipe building Ubuntu 16.04 virtual machine in unattended mode using PowerShell script.

Disclaimer

  • This script is output my experiment with Hyper-V, PowerShell & Bash on Windows 10 PRO. I have not added extensive error/exception handling.
  • To keep post focused on experiment and outcome, I have drafted much about Kickstart and Preseeding in Linux. I have provided reference to get good enough idea about it.
  • I ran this script 7 times after creating and it produced working VM in an average 35 minutes.

Ingredients

Steps

  1. Run the script to build iso for unattended setup.
  2. Connect to VM (last step in script hence actually not extra step)
  3. Connect through SSH and work with VM

Script Walk-through (High Level Steps)

  • Setting up required variables ( line number 2 to 9 )
  • Ask the user questions about user preferences ( line number 12 to 16 )
  • Check if the passwords match and then generate encrypted hash to use in preseed file ( line number 19 to 29 )
  • Check if the root passwords match and generate encrypted hash to use in ks.cfg file ( line number 32 to 42 )
  • Creating/Verifying Required Folder Structure is available to work ( line number 45 to 69 )
  • Mounting C:\LabSources\ISOs\ubuntu-16.04.1-server-amd64.iso file and copy content locally ( line number 72 to 82 )
  • Copy baseline Kickstart Configuration File [ks.cfg] To Working Folder ( line number 85 )
  • Copy baseline Seed File [mshende.seed in my case] (answers for unattended setup) To Working Folder ( line number 88 )
  • Update the ks.cfg file to reflect encrypted root password hash ( line number 91 )
  • Update the mshende.seed file to reflect the user’s choices ( line number 94 to 97 )
  • Update the isolinux.cfg file to reflect boot time choices ( line number 100 to 101 )
  • Building installer menu choice to make it default and use ks.cfg and mshende.seed files ( line number 104 to 111 )
  • Creating new ISO file at C:\LabSources\ISOs\ubuntu-16.04.1-server-amd64-unattended.iso ( line number 116 )
  • Create Hyper-V Virtual Machine ( line number 119 to 128 )
  • Add host entry on my physical machine pointing to VM ip address to connect VM over ssh ( line number 130 )
  • Starting Virtual Machine and Connecting through VM console ( line number 135 to 136 )
  • Connect to server once ready using putty ( line number 139, type manually once VM is ready after setup)

Links

Screenshots

This slideshow requires JavaScript.

References

Lessons Learnt

  • I am not expert in Linux, however being conceptually aware on diverse topics, I was able to use PowerShell to get what I want.
  • There are many options (both cloud and on-premise) available these days to host virtual machines. This is not about what is better v/s bad. This is about using available resources.
  • While writing this I couldn’t find best suitable alternatives for mkpasswd (line number 36) and mkisofs.exe ( line number 116 ) natively in/with PowerShell. However, because PowerShell can work with variety of applications, I was able to proceed without getting stuck.
  • In this case mkpasswd ( line number 36 ), I am calling bash command from Bash on Windows 10 and use in my script.
  • In this case of mkisofs which is available in bash shell, but I was not able to get it working like in the case of line number 36. I found Windows platform download at https://code.google.com/archive/p/mkisofs-md5/downloads.

Reading, The Phoenix Project…

Last two months were exciting & pleasantly busy which took toll on my new year (2017) resolution to blog more regularly & consistently. It also broke the rhythm of reading books more often and as much as I can. To add to this my new kindle (Thanks to my wife Smita for an advance anniversary gift) was staring at me daily for attention :-). So… I decided to start reading something immediately.

I am currently working on revising my training content & demos for PowerShell DSC (Desired State Configuration). Sometime back I had heard recommendation from Jeffrey Snover @jsnover about this book titled The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

(Blog post images copyright & courtesy IT Revolution Press @ http://itrevolution.com)

Having followed Jeffrey over these years it was easy decision to pick a book from my wish list. I was lucky to find kindle edition available at very good price. I grabbed it immediately and started reading it. After going through Wikipedia, the Introduction and Chapter 1, sounds like… its going to an interesting read for sure with some fictionalization of real world problems and solutions.

Hope to finish this soon to get myself a new knowledge/perspective on DevOps, Continuous Delivery and related areas. And of course pass that to the participants during my upcoming batches.

Get-CloudManagementTools

This PowerShell script can be used to download some of the common cloud (Azure and AWS) management tools available on Windows Platform.

Earlier I had seen Download and Install SharePoint 2013 Prerequisites on Windows Server 2012.  However this technet script uses Start-BitsTransfer cmdlet from BitsTransfer module. I was toying with similar idea while working on my last workshop (Cloud Automation Using PowerShell) preparation. Since some of the URLs do redirect the Start-BitsTransfer wasn’t working for me initially.

I started with simple Invoke-WebRequest cmdlet and added second variation to handle redirected URLs and pass it to Start-BitsTransfer. So let’s go through the script and output for the both the approaches.

I am using Downloads directory to store the files. For demo I have created CloudTools folder inside. I already have AWS Tools for Windows PowerShell downloaded in the folder.

15012017-01

I am already into the working directory where the script is saved.

15012017-02

Started the script execution and you can notice it finds AWS Tools For Windows PowerShell and skips the download for it.

15012017-03

On my Windows Server 2012 R2 VM it created a CloudTools folder which was missing before download starts.

15012017-03-01

# The folder location where the downloads will be saved
$DestinationFolder = "$ENV:homedrive$env:homepath\Downloads\CloudTools"
If (!(Test-Path $DestinationFolder)){
 New-Item $DestinationFolder -ItemType Directory -Force
}

Download is in progress…

15012017-04

Before this, I have created hash table with Download URL and File Name After Downloaded

# Specify download url's for various cloud management tools. Do not change unless Microsoft changes the downloads themselves in future
$Downloads = @{
 # Latest Azure PowerShell
 # Github: https://github.com/Azure/azure-powershell/releases/latest
 # WebPI: https://www.microsoft.com/web/handlers/webpi.ashx/getinstaller/WindowsAzurePowershellGet.3f.3f.3fnew.appids
 "https://aka.ms/azure-powershellget2" = "$DestinationFolder\Azure-PowerShell.msi"; 
 # Latest Azure Storage Explorer
 "https://go.microsoft.com/fwlink/?LinkId=708343" = "$DestinationFolder\StorageExplorer.exe";
 # Latest AZCopy
 "http://aka.ms/downloadazcopy" = "$DestinationFolder\MicrosoftAzureStorageTools.msi";
 # Latest Azure CLI
 "http://aka.ms/webpi-azure-cli" = "$DestinationFolder\Azure-cli.msi";
 # Azure Storage Emulator
 "https://go.microsoft.com/fwlink/?LinkId=717179&clcid=0x409" = "$DestinationFolder\MicrosoftAzureStorageEmulator.msi";
 # Latest AWS Tools For Windows PowerShell
 "http://sdk-for-net.amazonwebservices.com/latest/AWSToolsAndSDKForNet.msi" = "$DestinationFolder\AWSToolsAndSDKForNet.msi";
 }

Approach 1 – using Invoke-WebRequest cmdlet ,

Then I have a function which loop through the elements in downloads hash table using Invoke-WebRequest cmdlet,

function DownloadFiles(){ 
 Write-Host ""
 Write-Host "====================================================================="
 Write-Host " Downloading Cloud Management Tools for Azure & AWS" 
 Write-Host "====================================================================="
 
 $ReturnCode = 0

$Downloads.GetEnumerator() | ForEach-Object { 
 $DownloadURL = $_.get_key()
 $Filespec = $_.get_value()
 # Get the file name based on the portion of the file path after the last slash 
 $FilePath = Split-Path $Filespec
 $FileName = Split-Path $Filespec -Leaf
 Write-Host "DOWNLOADING: $FileName"
 Write-Host " FROM: $DownloadURL"
 Write-Host " TO: $FilePath"
 
 Try 
 { 
 # Check if file already exists 
 If (!(Test-Path "$Filespec")) 
 { 
 # Begin download 
 Invoke-WebRequest -Uri $DownloadURL -OutFile "$Filespec" -ErrorVariable err
 If ($err) {Throw ""} 
 Write-Host " STATUS: Downloaded"
 Write-Host
 } 
 Else 
 { 
 Write-Host " STATUS: Already exists. Skipping." 
 Write-Host
 } 
 } 
 Catch 
 { 
 $ReturnCode = -1
 Write-Warning " AN ERROR OCCURRED DOWNLOADING `'$FileName`'" 
 Write-Error $_
 Break 
 }
} 
 return $ReturnCode 
}

and the downloads completed.

15012017-05

Approach 2 – using Start-BitsTransfer cmdlet,

First a function Get-RedirectedUrl to get redirected URL

# This function gets the end URL to avoid redirection in URL breaking the Start-BitsTransfer
# Courtesy: http://www.powershellmagazine.com/2013/01/29/pstip-retrieve-a-redirected-url/
Function Get-RedirectedUrl {
 Param (
 [Parameter(Mandatory=$true)]
 [String]$url
 )
 $request = [System.Net.WebRequest]::Create($url)
 $request.AllowAutoRedirect=$true
 try{
 $response=$request.GetResponse()
 $response.ResponseUri.AbsoluteUri
 $response.Close()
 }
 catch{
 “ERROR: $_”
 }
}

Then I have a function which loop through the elements in downloads hash table using Start-BitsTransfer cmdlet,

# Import Required Modules: BITS is used for file transfer
Import-Module BitsTransfer

function DownloadFiles(){ 
 Write-Host ""
 Write-Host "====================================================================="
 Write-Host " Downloading Cloud Management Tools for Azure & AWS" 
 Write-Host "====================================================================="
 
 $ReturnCode = 0

$Downloads.GetEnumerator() | ForEach-Object { 
 $DownloadURL = Get-RedirectedUrl -URL $_.get_key()
 $Filespec = $_.get_value()
 # Get the file name based on the portion of the file path after the last slash 
 $FilePath = Split-Path $Filespec
 $FileName = Split-Path $Filespec -Leaf
 Write-Host "DOWNLOADING: $FileName"
 Write-Host " FROM: $DownloadURL"
 Write-Host " TO: $FilePath"
 
 Try 
 { 
 # Check if file already exists 
 If (!(Test-Path "$Filespec")) 
 { 
 # Begin download 
 Start-BitsTransfer -Source $DownloadURL -Destination "$Filespec" -DisplayName "Downloading `'$FileName`' to $FilePath" -Priority High -Description "From $DownloadURL..." -ErrorVariable err 
 If ($err) {Throw ""} 
 Write-Host " STATUS: Downloaded"
 Write-Host
 } 
 Else 
 { 
 Write-Host " STATUS: Already exists. Skipping." 
 Write-Host
 } 
 } 
 Catch 
 { 
 $ReturnCode = -1
 Write-Warning " AN ERROR OCCURRED DOWNLOADING `'$FileName`'" 
 Write-Error $_
 Break 
 }

} 
 return $ReturnCode 
}

while downloading the progress bar looks different,

15012017-06

I had deleted 2 files and ran the script again and following is the output,

15012017-07

If you want to know more about BITSTransfer & PowerShell you can refer this How to copy files with BITS using PowerShell

You can try both variation scripts by downloading from my (cloudscripts) github. I will be updating it later for more tools.

That’s it for now.