PowerShell vNext: AST.

What is that?

There is one addition in PowerShell vNext that I look really forward to. In v2 we have parser available. It makes work with PowerShell code easier. But PowerShell team haven’t stopped there. AST (Abstract Syntax Tree) is new concept (at least – to me) and it makes parsing code more interesting: instead of stream of tokens (which is still there with AST and is also more accurate than v2 precedents) we get tree of syntax: from Script to single Statement. Browsing thru that tree may be very hard at first. I tried to use view that seems natural for that kind of complex structures: Format-Custom. The problem that I went into was the fact, that each AST contains “Extent” and “Parent” property. And those properties keep whole code which makes Format-Custom lengthy and impractical. So my first step was to create .ps1xml files to make format and object themselves bit different. I hidden two properties I think are not very useful. You can still access them using psbase on each object – and that exactly the information you get if you try to access them directly. Next step I took was to create simple function that would wrap .NET call to System.Management.Automation.Language.Parser static methods (ParseInput, ParseFile) and make use of them more natural and PowerShell-ish. Then I packed it all in the module, so that I could paste it somewhere and let others use it and complain about things I did wrong (like using global scope to store tokens/ errors).

How would I use it?

Parser is the way to look closer at your code. If you want to extend editors of your choice (ISE, PowerGUI Script Editor, or any other you like and use) than having ability to read code as PowerShell does makes some operations more error-prone. It was the case with tokens in v2 – but it’s more the case with addition of AST in v3. Now you know exactly where you are, and you know what to expect. What language elements are justified in current AST, and which are wrong. When I was playing with new stuff I noticed, that it’s not replacement (as far as I can tell) for “old” technique. It’s just extension to it. But I’m not sure if that’s actually the case – it’s probable that I’m missing something obvious. For example: new tokens do not contain any info about absolute position in a script. On the other hand – being able to read nested tokens makes some operations easier (e.g. expanding aliases in sub-expression within a string). Example:

$Script = @'
"$($Foo.Count)"
'@             
            
$AST = Get-AST -InputScript $Script -TokensList TK            
            
Write-Host New tokens - nested tokens            
$TK | where { $_.NestedTokens } |             
    Select -ExpandProperty NestedTokens            
Write-Host Old tokens - no info of nested tokens here            
[Management.Automation.PSParser]::Tokenize($Script, [ref]$null)

Where is this module?

For now I’ve pasted it into TechNet script gallery. I would paste it into poshcode too, but because .psm1 file is just 1/3 of whole story I decided it makes no sense (yet). If you have some other ideas on what would be nice to have there – go ahead and change it, or let me know and I will try to do it. Would love to see others picking up this neat new concept. In my opinion in makes working with PowerShell code more natural – after all it is, in fact, more tree-like construct rather than flat stream of tokens. Having ability to see this tree structure up front may mean a lot to people who, like me, love PowerShell not only for it’s usability but also for the way it is built.

Advertisement