Architecture of CodeFormatter

Overview

CodeFormatter is a formatter for Wolfram Language code.

Running the formatter will only change whitespace and newlines, and has nothing to do with adding comments, adding parens, changing to FullForm, etc.

Inserting (*Else*) into If statements is not formatting, it is something else.

Converting And[a, b] into a && b (or vice versa) now involves output and precedence and parentheses.

Using CodeFormatter to insert [[ ]] characters or -> character is not formatting, it is something else.

Breaking up CompoundExpression[] at top-level is not formatting, it is something else, because this changes semantics.

CodeFormatter silently fixes FormatIssues.

Issues like this:

a/.3->.4

are automatically fixed.

It is slightly confusing to report issues like this if the formatter can just fix them automatically.

These sorts of issues “belong” to the formatter, not the linter.

Pipeline of passes

CodeFormatter has a pipeline of passes, like a compiler.

RemoveSimpleLineContinuations
↓
RemoveComplexLineContinuations
↓
RemoveRemainingSimpleLineContinuations
↓
StandardizeEmbeddedNewlines
↓
StandardizeEmbeddedTabs
↓
Fragmentize
↓
InsertNewlineAnchorInformationIntoComments
↓
removeWhitespaceAndNewlines
↓
AbstractFormatNodes
↓
IndentCST
↓
linearize
↓
mergeTemporaryLineContinuations
↓
absorbNewlinesIntoComments
↓
absorbNewlinesIntoSemis
↓
insertNecessarySpaces
↓
breakLinesV1

After StandardizeEmbeddedNewlines, "EmbeddedNewlines" data is removed.

removeWhitespaceAndNewlines canonicalizes to Graphical Syntax. It is nice to go to Graphical Syntax before abstracting. Graphical Syntax == Concrete Syntax - (whitespace + newlines) == Aggregate Syntax + Comments.

Airiness

airiness == -1

newlines are removed from comments

airiness < -0.85

all stay on single line: Module If Switch Which For

airiness < -0.75

groups stay on single line

airiness < -0.5

binary / infix / ternary stay on single line

airiness < -0.25

CompoundExpressions stay on single line

airiness = 0

normal

0.25 < airiness

newlines between elements of CompoundExpression

0.5 < airiness

newlines between binary / infix / ternary stay

0.75 < airiness

newlines between groups

0.85 < airiness

newlines between Commas