Power(Shell) to the people

Type less, write cleaner scripts, run consistently across platforms, and other reasons why Linux and OS X users can fall in love with PowerShell.
Register or Login to like
One lightbulb lit out of several

Opensource.com

Earlier this year, PowerShell Core became generally available under an Open Source (MIT) license. PowerShell is hardly a new technology. From its first release for Windows in 2006, PowerShell's creators sought to incorporate the power and flexibility of Unix shells while remedying their perceived deficiencies, particularly the need for text manipulation to derive value from combining commands.

Five major releases later, PowerShell Core allows the same innovative shell and command environment to run natively on all major operating systems, including OS X and Linux. Some (read: almost everyone) may still scoff at the audacity and/or the temerity of this Windows-born interloper to offer itself to platforms that have had strong shell environments since time immemorial (at least as defined by a millennial). In this post, I hope to make the case that PowerShell can provide advantages to even seasoned users.

Consistency across platforms

If you plan to port your scripts from one execution environment to another, you need to make sure you use only the commands and syntaxes that work. For example, on GNU systems, you would obtain yesterday's date as follows:

date --date="1 day ago"

On BSD systems (such as OS X), the above syntax wouldn't work, as the BSD date utility requires the following syntax:

date -v -1d

Because PowerShell is licensed under a permissive license and built for all platforms, you can ship it with your application. Thus, when your scripts run in the target environment, they'll be running on the same shell using the same command implementations as the environment in which you tested your scripts.

Objects and structured data

*nix commands and utilities rely on your ability to consume and manipulate unstructured data. Those who have lived for years with sed grep and awk may be unbothered by this statement, but there is a better way.

Let's redo the yesterday's date example in PowerShell. To get the current date, run the Get-Date cmdlet (pronounced "commandlet"):

> Get-Date                        

Sunday, January 21, 2018 8:12:41 PM

The output you see isn't really a string of text. Rather, it is a string representation of a .Net Core object. Just like any other object in any other OOP environment, it has a type and most often, methods you can call.

Let's prove this:

> $(Get-Date).GetType().FullName
System.DateTime

The $(...) syntax behaves exactly as you'd expect from POSIX shells—the result of the evaluation of the command in parentheses is substituted for the entire expression. In PowerShell, however, the $ is strictly optional in such expressions. And, most importantly, the result is a .Net object, not text. So we can call the GetType() method on that object to get its type object (similar to Class object in Java), and the FullName property to get the full name of the type.

So, how does this object-orientedness make your life easier?

First, you can pipe any object to the Get-Member cmdlet to see all the methods and properties it has to offer.

> (Get-Date) | Get-Member
PS /home/yevster/Documents/ArticlesInProgress> $(Get-Date) | Get-Member        


   TypeName: System.DateTime

Name                 MemberType     Definition                                
----                 ----------     ----------                                
Add                  Method         datetime Add(timespan value)              
AddDays              Method         datetime AddDays(double value)            
AddHours             Method         datetime AddHours(double value)            
AddMilliseconds      Method         datetime AddMilliseconds(double value)    
AddMinutes           Method         datetime AddMinutes(double value)          
AddMonths            Method         datetime AddMonths(int months)            
AddSeconds           Method         datetime AddSeconds(double value)          
AddTicks             Method         datetime AddTicks(long value)              
AddYears             Method         datetime AddYears(int value)              
CompareTo            Method         int CompareTo(System.Object value), int ...

You can quickly see that the DateTime object has an AddDays that you can quickly use to get yesterday's date:

> (Get-Date).AddDays(-1)

Saturday, January 20, 2018 8:24:42 PM

To do something slightly more exciting, let's call Yahoo's weather service (because it doesn't require an API token) and get your local weather.

$city="Boston"
$state="MA"
$url="https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22${city}%2C%20${state}%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys"

Now, we could do things the old-fashioned way and just run curl $url to get a giant blob of JSON, or...

$weather=(Invoke-RestMethod $url) 

If you look at the type of $weather (by running echo $weather.GetType().FullName), you will see that it's a PSCustomObject. It's a dynamic object that reflects the structure of the JSON.

And PowerShell will be thrilled to help you navigate through it with its tab completion. Just type $weather. (making sure to include the ".") and press Tab. You will see all the root-level JSON keys. Type one, followed by a ".", press Tab again, and you'll see its children (if any).

Thus, you can easily navigate to the data you want:

> echo $weather.query.results.channel.atmosphere.pressure                                                              
1019.0

> echo $weather.query.results.channel.wind.chill                                                                      
41

And if you have JSON or CSV lying around (or returned by an outside command) as unstructured data, just pipe it into the ConvertFrom-Json or ConvertFrom-CSV cmdlet, respectively, and you can have your data in nice clean objects.

Computing vs. automation

We use shells for two purposes. One is for computing, to run individual commands and to manually respond to their output. The other is automation, to write scripts that execute multiple commands and respond to their output programmatically.

A problem that most of us have learned to overlook is that these two purposes place different and conflicting requirements on the shell. Computing requires the shell to be laconic. The fewer keystrokes a user can get away with, the better. It's unimportant if what the user has typed is barely legible to another human being. Scripts, on the other hand, are code. Readability and maintainability are key. And here, POSIX utilities often fail us. While some commands do offer both laconic and readable syntaxes (e.g. -f and --force) for some of their parameters, the command names themselves err on the side of brevity, not readability.

PowerShell includes several mechanisms to eliminate that Faustian tradeoff.

First, tab completion eliminates typing of argument names. For instance, type Get-Random -Mi, press Tab and PowerShell will complete the argument for you: Get-Random -Minimum. But if you really want to be laconic, you don't even need to press Tab. For instance, PowerShell will understand

Get-Random -Mi 1 -Ma 10

because Mi and Ma each have unique completions.

You may have noticed that all PowerShell cmdlet names have a verb-noun structure. This can help script readability, but you probably don't want to keep typing Get- over and over in the command line. So don't! If you type a noun without a verb, PowerShell will look for a Get- command with that noun.

Caution: although PowerShell is not case-sensitive, it's a good practice to capitalize the first letter of the noun when you intend to use a PowerShell command. For example, typing date will call your system's date utility. Typing Date will call PowerShell's Get-Date cmdlet.

And if that's not enough, PowerShell has aliases to create simple names. For example, if you type alias -name cd, you will discover the cd command in PowerShell is itself an alias for the Set-Location command.

So to review—you get powerful tab completion, aliases, and noun completions to keep your command names short, automatic and consistent parameter name truncation, while still enjoying a rich, readable syntax for scripting.

So... friends?

There are just some of the advantages of PowerShell. There are more features and cmdlets I haven't discussed (check out Where-Object or its alias ? if you want to make grep cry). And hey, if you really feel homesick, PowerShell will be happy to launch your old native utilities for you. But give yourself enough time to get acclimated in PowerShell's object-oriented cmdlet world, and you may find yourself choosing to forget the way back.

Software Engineer, with an open source governance bend. Occasional developer outreacher. All opinions are my own. Brain droppings here.

23 Comments

Great article! There were so many times in my sysadmin days when scripting would have been a lot easier if I had a richer data method than strings of text (although strings of text have some advantages, too). But since most *nix commands only or mostly operate on strings of text, what's the advantage to using PowerShell over Python, Ruby, etc?

Good question. There is an element of subjectivity in preferring one tool over another. For me, the advantage of PowerShell is its specific, intentional duality as a command shell and a scripting language. For example, deleting a file in Ruby would take a bit more typing than in PowerShell. And the tab completion and uniform style of command names wouldn’t be there.

I find it helpful to think of PowerShell not as a programming language but as a DSL for automation. You wouldn’t use PowerShell to write a web application. You might use PowerShell to automate configuring a web application on your system or deploying a web application to the public cloud (using modules provided by the vendor).

Does this help?

In reply to by Ben Cotton

This is ridiculous. Powershell is inconsistent - some cmdlets try/catch properly and some do not (i.e.: return an exception).
You're better off with something like Python ....

No thank you. I don't know what else to say and still be polite about it.

Tired of Microsoft and Windows trying to "mimic" the Open Source way!...LoL! Seems they've finally hit "the wall" and have come to realize that their way of doing things will only carry them so far, and now they want to be the Open source "brand new innovators"?. Nope they had their chance to play nice with Unix and Linux and blew it. To the point where we have EVERYTHING we need in the open source world, from media players, to web browsers, to office suites, and PDF readers, and BASH scripts, and Command Line Interfaces, etc. There's no real reason to even attempt to try and commingle the two. Let those that wanna try this go right ahead, but I'm perfectly happy with my Linux Command line syntax and features. Sorry. but not everyone wants to "blend" Windows with their preferred OS.

I'm a Microsoft employee (speaking on my own behalf and not representing the company), and I haven't run Windows on a machine I own for over a decade. I'm perfectly happy with my Linux setup, too. You're right that Microsoft realized that it's approach to open source was wrong. But to say that there's no real reason to mix the two is to completely ignore the fact that many people do run mixed environments (whether by choice or by mandate from their employer). Being able to use the same tools across multiple platforms is valuable to system administrators. You're right that not everyone wants to blend Windows with their preferred OS. But obviously some people do.

In reply to by guest (not verified)

Oh I agree wholeheartedly with you. There most likely ARE people that want to run them both, but for me? I made the decided effort to not use Windows period. And understand...I've been working in the I. T. Field since '99, and have been using Windows since 3.1 So I know the ins and outs of it. And while the masses might be lulled into a sense of it being "new" Its not. its the SAME old Windows as Windows 95...which was the precursor to Windows 98....which was the precursor to Windows NT 4.0 which led to Windows 2000.....then 2003......2007.....Vista.....XP......Win7....Win8...and recently Win10. So I know that while things might be called a different name, and things might get new icons and new locations within the system, its still a system that relies heavily on ".exe's" which can be impregnated with all manner of undesirables! And even if PowerShell is free from most of the code that runs Windows, it's still "attached" to it and for myself, that is not something I want to experiment with. Besides, I've been with Linux since 2001/'02 and have already learned how to move around in the terminal using Linux commands, why would I want to have to "learn" MS's syntax? For the SysAdmins who needs this...I say go for it...but I will never have a need for Windows again...and so I choose to remove any possibility of anything from that platform interfering with or impacting my Linux servers and databases. But I do see your point, a Windows Admin/developer might have a need to run this on a Linux box in order to see if/how it runs against their code, or to see the results of running something from Linux and how it ports to Windows. I just don't need it, and for me...the whole "Microsoft Loves Linux" doesn't fly, I know there are still court cases regarding patents that might be ongoing for the next ten years or more...so no. no thank you...no Windows for me.

In reply to by Ben Cotton

Edward,

PowerShell 6 is built on top of .Net Core, a new .Net moduler runtime that's built natively for each of its target platforms. There are no exes. No emulators. No Windows.

In reply to by Edward G OConnor (not verified)

While not a programmer, I have had to learn Shell scripting to level for successfully and competently completing many critical scripting tasks.

There have been many occasions where Powershell simply does not match BASH for functionality. Take simple ls command with at least 8 - 10 parameters for viewing/listing information that Powershell lacks.

And finally, if Powershell is so "great", then why has Microsoft integrated the complete Ubuntu BASH Shell scripting toolset into Windows 10, as well as ISO Python, openSSH and almost every other UNIX/Linux tool?

PowerShell does include the “dir” cmdlet (alias for Get-ChildItem) which serves the purpose of “ls”. The original Windows version of PowerShell actually aliased it as “ls” as well. I’m assuming that alias was removed so as to not conflict with the native “ls” command on Posix platforms.

You can read about how Get-ChildItem (aka “dir”) works with files here: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell…

And with PowerShell you don’t need as many parameters to specify what information to return, because what you get are .Net objects whose properties you can query with Where-Object (aka “where”, aka “?”) cmdlet.

Do not assume a solution does not exist where it is not immediately apparent.

In reply to by W. Anderson (not verified)

> Because PowerShell is licensed under a permissive license and built for all platforms, you can ship it with your application.

You can ship with any other FLOSS licensed language as well. The reason for using real programming languages instead of bash is error handling, not ease of operation. Show me what a well written error handling code looks like in power shell and I can think about it. Otherwise no reason to switch to a yet another hybrid thing that assumes open source means that everybody should switch to it.

There is btw python shell as well. And probably a horde of other shell initiatives that *fix* things. Especially when I read about case insensitivity, trying to match commands to some prefixes, guessing partial command parameters and other *helpful* features... this makes static analysis impossible, decreases code readability and is error prone. Also those partial parameter matching is definitely not future proof as adding new parameters could easily break existing assumptions.

Alexander,

Here's a tutorial that describes all the exception handling features of PowerShell. https://www.vexasoft.com/blogs/powershell/7255220-powershell-tutorial-t…. The article was clearly written on Windows, as it uses the Windows paths, but all the error handling mechanisms work the same on every platform.

I am not suggesting that you use PowerShell as an alternative to Python in writing application code. I am suggesting that you use PowerShell where you might previously have opted to write a bash shell script. Even if you choose to write the implementation logic in an actual programming language, writing a wrapper script in PowerShell, instead of a Posix shell, can provide a smoother user experience via the features I described in the post.

Keep in mind too that PowerShell runs on top of the .Net Core platform and natively interacts with the .Net type system. So you can, if you so choose, implement the actual application logic in C#, F#, or any other language that builds .Net Standard-compliant libraries, and then invoke that logic through PowerShell and receive structured objects as output, for easy subsequent manipulation.

In reply to by Aleksandar Kos…

PowerShell made it easy for me to modify the "Date Modified" for all of the folders in a directory tree (so I could view my folder of projects by date to see how long since I worked on each). I ended up being able to do this with a SINGLE line in PowerShell. This was possible due to recurse and the ability to pipe and filter returned objects. This was a Windows PC so not sure if there are other approaches that would do the same thing on other systems.

The script deep dives each folder and set each folder's modification date the the newest modification date of any file or folder contained in that folder. I hard coded the paths in the single line version. A longer version prompts you for a target folder and provides a progress bar (and reports errors in the PowerShell IDE).

Get-ChildItem "H:\DropBox\FolderDateFix\Test1" -recurse | Where-Object {$_.PsIsContainer} | ForEach-Object {$_.LastWriteTime = ($_ | Get-ChildItem -recurse | Where-Object {!$_.PsIsContainer} | Sort-Object LastWriteTime | Select-Object -last 1).LastWriteTime}

I break it out and walk through each line in a blog post if you want more detail on what each object does.

http://fettricks.blogspot.com/2014/08/useful-folder-modification-dates-…

This is a great example of what powershell can easily do! I think Get-ChildItem, Select-Object, and Where-Object are among the few cmdlet where the alias is more straightforward than the fullname. So I’d replace them with “cd”, “select”, and “?” (or “where”), respectively.

Have you tried using Measure-Object -Maximum to get the newest file modification date instead of sorting the whole directory?

In reply to by Shane Trent (not verified)

I've been a Windows admin for over 10 years. For a long time I've resisted to use Powershell. And instead used Python. For example by calling the Win32 API using Python's ctypes.

Primarily for the reason that I found Powershell intransparently linked to the Windows installation (and certain frameworks that ship with Windows). Powershell used to be an "unremovable" part of Windows. Every Windows shipped with a different Powershell version, making scripts highly incompatible and hard to test for these various targets (If I use the latest Powershell version on Windows 10 to develop a sufficiently complex script, I'm pretty sure my script won't run on a vanilla Windows 7). Updating Powershell meant that you had to install a Windows update (which behaves differently than a regular MSI installer - it's in many ways harder to deploy on a multitude of computers - i.e. it requires a reboot, etc). This update is an optional update - usually excluded from auto-approval on WSUS servers in large organizations. I've seen more than one Enterprise environment where every computer has a different Powershell version installed.

With Python things were so much easier (I could just put a Python installation on a network share or bundle it with my script).

Fast-forward to early 2018:

- Powershell is open source
- Powershell has been separated from the various frameworks that it or its commandlets need to operate
- the most important framework, .NET, is not required as a prerequisite anymore but the required .NET core runtime is bundled with Powershell
- .NET core itself has been "unbundled" from Windows and became cross-platform open source
- this makes a Powershell installation standalone/portable (just extract the zip file anywhere and run pwsh.exe)
- pwsh still needs some external frameworks provided by the operating system. The most important one for Windows admins is WMI / CIM. While I can now bundle Powershell with my script, it remains annoying that "get-wmiobject / "get-cimobject" only works if Windows Management framework 5+ is installed on the local Windows system (the one that also provides Powershell 5.1). However, it has now at least become transparent which frameworks are actually Windows frameworks and not something out of that Powershell+WMI+other stuff bloat.
- some useful stuff has disappeared from Powershell 6 as it uses Windows-only frameworks (i.e. out-gridview). Sacrifice for cross-platform consistency (no commandlets that are Windows-only).
- transparency through open source: Powershell 6 is a well-maintained project on github. With very friendly maintainers who are keen to answer questions, etc. Never seen such a good public/free support from Microsoft before.

Overall, this resolves most of my painpoints with Powershell. The biggest remaining painpoint is that "get-wmiobject" / "get-cimobject", which are super important for most Windows administration tasks and used in most scripts, still depend on having installed the latest Windows Management framework.

Where Powershell is really much better than Python is when it comes to utilizing external C and .NET librarys. Using .NET assemblies is relatively easy and a native thing. (Remember: Powershell is a .NET application, with a .NET JIT compiler. Your Powershell code gets compiled into .NET bytecode on the fly). Calling the "classic" Win32 API (C libraries) is a bit more complicated, but still much easier than creating structs in Python for ctype.

It's also great to see which direction Powershell was heading since version 5 / 5.1 . More than ever, it is less a shell and more something that I would rather call ".NET Script".
- first of all, the Powershell language now is capable of classes / OOP
- then, Powershell is now capable to run and execute "C#" code (without any need to compile it into .NET bytecode files - it has a .NET JIT compiler built in)

After using *nix shells for many years, I'm learning PowerShell now. And while the new open source approach is giving it a boost, I have a problem with the initial premise of the article -- (1) if you use a specific shell (say, BASH, or SH if you are brave ;), you do have consistency across platforms (2) on the other hand, reading one of the references quoted in the article, it goes on to announce that certain features are no longer supported in the 4.0 version. So much for "same anytime/everywhere"?

Bash/she may give consistency across platforms, but the utilities they invoke sometimes do not.

I do not assert that powershell never deprecates or modifies features or that all syntax is identical across versions. However, you can be certain of getting the same features and syntax in your execution by simply bundling the correct version of Powershell (6.0 or later) with your script(s). In other words, your application can own its automation environment, instead of relying on the operating system to provide it. That’s how you can guarantee consistency.

In reply to by Erez L. (not verified)

However microsoft may invest in providing .deb and .rpm from its own repo for powershell installation, But organization and people will seriously use it in more numbers in serious projects only when it is available from the STANDARD/main REPOSITORY OF DEBIAN/REDHAT/CENTOS itself.
for example: (VVVVery important)
see debian maintainers having some concerns over including powershell in debian main repository
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=834756#16

Until the above things get solved and it is included in main repo, it will not ensure/prove to end user that all the requisites of powershell compilation is met in open source compatible way and without any patent, down the last build chain.

IMHO, trusting in Microsoft is a punch directly on the face of history. The company is now "open source friendly" only because the Microsoft era was under danger. It is better to trust in everyone else that had the initiative when free software was uncertain and mostly a moral philosophy, for whom *people* were more important than *money*.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.