PowerShell vNext: AST.

What is that?

There is one addition in PowerShell vNext that I look really forward to. In v2 we have parser available. It makes work with PowerShell code easier. But PowerShell team haven’t stopped there. AST (Abstract Syntax Tree) is new concept (at least – to me) and it makes parsing code more interesting: instead of stream of tokens (which is still there with AST and is also more accurate than v2 precedents) we get tree of syntax: from Script to single Statement. Browsing thru that tree may be very hard at first. I tried to use view that seems natural for that kind of complex structures: Format-Custom. The problem that I went into was the fact, that each AST contains “Extent” and “Parent” property. And those properties keep whole code which makes Format-Custom lengthy and impractical. So my first step was to create .ps1xml files to make format and object themselves bit different. I hidden two properties I think are not very useful. You can still access them using psbase on each object – and that exactly the information you get if you try to access them directly. Next step I took was to create simple function that would wrap .NET call to System.Management.Automation.Language.Parser static methods (ParseInput, ParseFile) and make use of them more natural and PowerShell-ish. Then I packed it all in the module, so that I could paste it somewhere and let others use it and complain about things I did wrong (like using global scope to store tokens/ errors).

How would I use it?

Parser is the way to look closer at your code. If you want to extend editors of your choice (ISE, PowerGUI Script Editor, or any other you like and use) than having ability to read code as PowerShell does makes some operations more error-prone. It was the case with tokens in v2 – but it’s more the case with addition of AST in v3. Now you know exactly where you are, and you know what to expect. What language elements are justified in current AST, and which are wrong. When I was playing with new stuff I noticed, that it’s not replacement (as far as I can tell) for “old” technique. It’s just extension to it. But I’m not sure if that’s actually the case – it’s probable that I’m missing something obvious. For example: new tokens do not contain any info about absolute position in a script. On the other hand – being able to read nested tokens makes some operations easier (e.g. expanding aliases in sub-expression within a string). Example:

$Script = @'
$AST = Get-AST -InputScript $Script -TokensList TK            
Write-Host New tokens - nested tokens            
$TK | where { $_.NestedTokens } |             
    Select -ExpandProperty NestedTokens            
Write-Host Old tokens - no info of nested tokens here            
[Management.Automation.PSParser]::Tokenize($Script, [ref]$null)

Where is this module?

For now I’ve pasted it into TechNet script gallery. I would paste it into poshcode too, but because .psm1 file is just 1/3 of whole story I decided it makes no sense (yet). If you have some other ideas on what would be nice to have there – go ahead and change it, or let me know and I will try to do it. Uśmiech Would love to see others picking up this neat new concept. In my opinion in makes working with PowerShell code more natural – after all it is, in fact, more tree-like construct rather than flat stream of tokens. Having ability to see this tree structure up front may mean a lot to people who, like me, love PowerShell not only for it’s usability but also for the way it is built. Uśmiech


4 thoughts on “PowerShell vNext: AST.

  1. I’m happy to grab it off the Technet Script Center, but if you really wanted to post it to Poshcode, you could simply embed the .ps1xml files as here-strings, and have it create them if they don’t already exist.

    Trevor Sullivan

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s