Documenting and Presenting with Pillar

Damien Cassou, Cyril Ferlicot Delbecque, Yann Dubois and Thibault Arloing

This chapter describes Pillar version 4.0. The original author of Pillar and current maintainer is Damien Cassou. Many people have also contributed: Ben Coman, Stéphane Ducasse, Guillermo Polito, Lukas Renggli (original author of Pier from which Pillar has been extracted), Benjamin van Ryseghem, Cyril Ferlicot-Delbecque, Thibault Arloing and Yann Dubois. Pillar is sponsored by ESUG.

1. Introduction

Pillar (hosted at http://www.smalltalkhub.com/#!/~Pier/Pillar) is a markup syntax and associated tools to write and generate documentation, books (such as this one) and slide-based presentations. The Pillar screenshot in Figure 0.1 shows the HTML version of chapter .

Pillar has many features, helpful tools, and documentation:

• simple markup syntax with references, tables, pictures, captions, syntax-highlighted code blocks;
• export documents to HTML, LaTeX, Markdown, AsciiDoc, ePuB and Pillar itself, and presentations to Beamer and Deck.js;
• customization of the export through a dedicated STON configuration file (see chapter ) and Mustache templates (see chapter ).
• many tests with good coverage (94% with more than a 2100 executed tests), which are regularly run by a continuous integration job
• a command-line interface and dedicated plugins for several text editors: Emacs, Vim, TextMate, and Atom
• a cheat sheet

1.1. Pillar Users

This book was written in Pillar. If you want to see how Pillar is used, have a look at its source code (http://books.pharo.org/enterprise-pharo/), or check the following other real-world projects:

2. 5 Minutes Tutorial

In this section we give the basic steps to get you started with your first Pillar document and exports. You first need to create a base directory inside which we will put all your text, configuration files, and Pillar itself.

2.1. Installing and Exporting your First Document

The use of pillar is possible on all platform supported by Pharo. In fact Pillar is executed on top of Pharo

so you can use Pillar on Linux, MacOSX, Windows.

2.1.1. Pillar installation on Windows

Windows is not able to deals with .sh and makefile script natively, so we have to install Pillar manually or install https://www.cygwin.com/ which provides functionality similar to a Linux distribution.

If you decide to install cygwin you should be able to follow the Mac OS X / Linux installation section otherwise you can follow the next steps.

If you don't have Pharo you can get it here : http://files.pharo.org/platform/Pharo5.0-win.zip

Then download and extract the content of Pillar-deployment.zip in the Pharo directory.

• Let's create an html document example :

Create for instance welcome.pillar file. You can place it in your Pharo directory and fill in this code example

We would like pillar to generate a json file which will be used by a generation template. Maj + right click in the Pharo directory and open a terminal and paste this command to generate the json file.

This command starts Pharo with Pillar image and trigger the pillar handler. Then it specifies the exportation as html and the output file as welcome. The .html.json will be appended automatically to the end of the filename so welcome.pillar.json will be created and placed in book-result folder (you can specify the outputDirectory in the command or in the .pillar file and this is the same for other attributes like outputFile). The last argument is the path of the input file.

Then we have to create a template, for instance let's create welcome.html.template in the Pharo directory and fill in this template example :

This latest command let us make the html document thanks by the json and template files. Each attribute in the template will be replaced by its value in the json file.

You can check the result in the stdout file which is placed in the Pharo directory

• Exportation in latex

The procedure is the same with other kind of document. This is an another example with latex

In this example we show how to add a bibliography and how to export the result as pdf with pdflatex provided by (MiKTeX distribution)

Let's set some attributes (title, content) in testLatex.pillar for instance

We create testLatex.tex.json with the following command

Then, let's create the template in testLatex.latex.template file for instance

We generate the document in the stdout file with the following command

Create a bibliography if it doesn't exists and place it in the Pharo directory. In our example this is rmod.bib file

Then, link the bibliography with the latex document

2.1.2. Pillar installation on Mac OS X / Linux

You first need to get Pillar. For that, we recommend downloading and executing the script available at https://raw.githubusercontent.com/pillar-markup/pillar/master/download.sh in the base directory if you are on an Unix environment.

Then, you can load an archetype (see Section 2.3) with command:

Now you can just compile the welcome file

You can see the result of the compilation as follow:

2.1.3. Creating a first page

You can create a first page naemd first.pillar file with this content:

And finally compiling it from a terminal (see Section 6 for more information about the command-line interface).

This will generate a first.html file you can open in a web browser. The content of this file will be something like:

2.2. Configuring a Document

As you can see, there is no document title in the generated first.html file. This is because we did not specify any. To specify a title, we have to add it with a configuration at the beginning of the first.pillar file:

When you compile using the same command line,

you should now get a web page with a title:

Another way to achieve the same is to use a dedicated configuration file. This configuration is typically named pillar.conf and is written in the STON format (see Section 4 for more information about the configuration file). Create your first pillar.conf file:

Meta-information specified in Pillar files take precedence over configuration in the pillar.conf file. To see the new title, you thus have to remove the one in first.pillar.

2.3. Exporting a Different Content Using a Template

If you want to tweak the content of the exported file, for example to reference your CSS or to add a footer, you need to create your own template (see Section 5 for more information about templating). You must write such template in its own file, e.g., myhtml.template:

Then, to use this template, you need to replace the HTMLTEMPLATE variable in the Makefile. So, edit your Makefile:

Now, write your name in first.pillar :

You can also write the year in the pillar.conf file:

Finally, compile first.pillar one last time

to generate a file containing:

Look at how the HTML template (myhtml.template) references title, author and year. These variables are referenced by enclosing them in 3 curly braces. The templating engine that transforms your templates in documents is Mustache (see chapter . As you can see, I decided to put the author of the document in the first.pillar file whereas the year and title are specified in pillar.conf: this is arbitrary and you can do whatever suits you best: the differences being that the pillar.conf file applies to all Pillar files of the project and that file meta-information takes precedence.

This concludes our 5 minutes tutorial.

2.4. Archetype - A Pillar skeleton maker

An archetype is a project skeleton to well begin a pillar project. In an archetype you have all the files you will need and an example.

Basically you have:

• Makefile
• pillar.conf
• templates
• ... with some differences between archetypes

Here is the list of the current archetypes :

• book
• presentation
• welcome

Pay attention that you may have to restart from a fresh folder when you change project style.

You can help us by creating your own archetype on the Github Project.

To add an archetype, just create the file tree and put an empty file ".keep" in empty folders.

3. Writing Pillar Documents

In this section we show how to write Pillar documents by presenting the Pillar syntax. You might want to have a look at the cheat sheet and even download and print it.

3.1. Meta-Information

Meta-information of a particular file is written at the start of the file between curly braces using the STON syntax (see chapter ). A meta-information starts with a word between quotation marks acting as a key, is followed by a colon :, and finishes with a value. For example, the following Pillar file,

represents a Pillar document with the title and author set. You can use whatever keys you like. Use them by referencing them in templates (see section 5 for more information about templating). Only keys in the metadata field can be referenced in the template.

3.2. Chapters & Sections

A line starting with ! represents a heading. Use multiple ! to create sections and subsections.

To refer to a section or chapter, put an anchor (equivalent to \label{chapterAndSections} in Latex) using the @chapterAndSections syntax on a separate line. Then, when you want to link to it (equivalent to \ref{chapterAndSections} in Latex), use the *@chapterAndSections* syntax. Anchors are invisible and links will be rendered as: 3.2.

3.3. Paragraphs and Framed Paragraphs

An empty line starts a new paragraph.

An annotated paragraph starts with @@ followed by a keyword such as todo and note. For example,

generates

this is a note annotation.

And,

generates a todo annotation

this is a todo annotation

The annotation (e.g., todo and note) can be any word that is meaningful to the author. In HTML, an annotated paragraph triggers the generation of a paragraph with the annotation as the paragraph class. In LaTeX, an environment with the annotation name is generated. In HTML, you can tweak the output to make it look nice, for example with such JavaScript code:

Above code will prepend the titles "Note" and "To do" to the @@note and @@todo paragraphs. You can make that looks nice using a little bit of CSS:

3.4. Lists

3.4.1. Unordered Lists

generates

• A block of lines,
• where each line starts with -
• is transformed to a bulleted list
3.4.2. Ordered Lists

generates

1. A block of lines,
2. where each line starts with #
3. is transformed to an ordered list
3.4.3. Definition Lists

Definition lists (aka. description lists) are lists with labels:

generates

blue
color of the sky
red
color of the fire
3.4.4. List Nesting

generates

• Lists can also be nested.
1. Thus, a line starting with -#
2. is an element of a bulleted list that is part of an ordered list.

3.5. Formatting

There is some syntax for text formatting:

• To make something bold, write ""bold"" (with 2 double quotes)
• To make something italic, write ''italic'' (with 2 single quotes)
• To make something monospaced, write ==monospaced==
• To make something strikethrough, write --strikethrough--
• To make something subscript, write @@subscript@@
• To make something superscript, write ^^superscript^^
• To make something underlined, write __underlined__

3.6. Tables

To create a table, start the lines with | and separate the elements with |. Each new line represents a new row of the table. Add a single ! to let the cell become a table heading.

Language Coolness
Smalltalk Hypra cool

The contents of cells can be aligned left, centered or aligned right by using |{, || or |} respectively.

generates

 centered header centered centered left right center

To put an anchor (equivalent to \label in Latex), use the @anchorName syntax on a separate line. Then, when you want to link to it (equivalent to \ref in Latex), use the *@anchorName* syntax. Anchors are invisible and links will be rendered as: 3.7.1.

To create a link to an other pillar file, use the *Alias>path.pillar@anchorName*. The Alias and the anchor are optional but you will need them in some cases (for example if you have an inter-file link and you export in LaTeX, or if you have an inter-file link and you export all your file in the same html file).

To create links to external resources, use the *Pharo>http://pharo.org/* syntax which is rendered as Pharo. The same syntax can also represent email addresses: write *damien@cassou.me* to get damien@cassou.me.

To create semantic links ressources, use the *Wikipedia Pharo>wikipedia:Pharo* syntax which is rendered as Wikipedia Pharo. To specify a language, you can use the *Wikipedia Pharo>wikipedia:Pharo|lang=en* syntax which is rendered as Wikipedia Pharo

Same for Youtube links : *Youtube Pharo>youtube:KDvNuOjdjY4*, the result is : Youtube Pharo.

3.8. Pictures

To include a picture, use the syntax +caption>file://filename|parameters+:

generates Figure 0.2 (this reference has been generated using *@pharoLogo*).

3.9. Scripts

Use scripts when you want to add code blocks to your document.

[[[
foo bar
]]]

generates

3.9.1. Script with a Label or Caption

If you want either a label (to reference the script later) or a caption (to give a nice title to the script), write the following:

[[[label=script1|caption=My script that works|language=smalltalk
self foo bar
]]]

which produces script 0.1 (this reference is produced with *@script1*).

3.9.2. Syntax Highlighting

To specify the syntax a script is written in, you need to use the language parameter. For example on 0.1 we used the smalltalk value for the language parameter.

The currently supported languages are bash, css, html, http, json, javascript, pillar, sql, ston, shellcommands and smalltalk

If you don't want syntax highlighting for a particular script, specify no language as value to the language parameter.

3.9.3. Script with Line Numbers

If you need to explain a long piece of code, you may want a script to have line numbers:

[[[lineNumber=true
self foo bar.
self bar foo.
]]]

produces

3.9.4. Script from an External File

If you want you can also include a script from a external file. For example if you have a file myProject.html and you want to take the code from line 15 to line 45, instead of copy/pasting the code you can use:

[[[language=html|fromFile=myProject.html|firstLine=15|lastLine=45
]]]

The firstLine and lastLine parameters are optional.

3.9.5. Generate a Part of your Document with a Script

If you want you can also evaluate a script to generate a part of your document. For example if you write a project's documentation and want to give some metrics about its code, you can write something like this:

[[[eval=true
| packages classes |
packages := RPackageOrganizer default packages select: [ :each |
each name includesSubstring: 'Pillar' ].
classes := packages flatCollect: [ :each | each classes ].
stream
nextPutAll: 'The Pillar project contains:';
lf;
nextPutAll: '- ==';
print: packages size;
nextPutAll: ' packages==.';
lf;
nextPutAll: '- ==';
print: classes size;
nextPutAll: ' classes=='.
]]]

will generate:

The Pillar project contains:

• 29 packages.
• 373 classes

For example section 4.2 of this chapter is generated.

3.10. Structures

You can create structures to render all structures with the same name as the same object. Structures use the scripts syntax (see 3.9) with a particular parameter structure

[[[structure=exercise
{
"question":"What is the answer to life, the universe and everything ?",
}
]]]

There is two kind of structures for the moment:

• exercise, rendered as a definition list
• city, rendered as a table
• country, rendered as a table

3.11. Raw

If you want to include raw text into a page you must enclose it between {{{ and }}}, otherwise Pillar ensures that text appears as you type it which might require transformations.

A good practice is to always specify for which kind of export the raw text must be outputted by starting the block with {{{latex: or {{{html:. For example, the following shows a formula, either using LaTeX or plain text depending on the kind of export.

{{{latex:
$$\label{eq:1} \frac{1+\sqrt{2}}{2}$$
}}}
{{{html:
(1+sqrt(2)) / 2
}}}

Take care: avoid terminating the verbatim text with a } as this will confuse the parser. So, don't write {{{\begin{scriptsize}}}} but {{{\begin{scriptsize} }}} instead.

3.12. Annotations

Annotations are the Pillar way to have extensible syntax. An annotation has this syntax:

3.12.1. InputFile Annotation

You can include a file into another pillar file. The inputFile annotation takes as parameter the path of the file relative to baseDirectory (if you don't change the base directory, it is your working directory). In this example, 2 files are included:

3.12.2. Footnotes Annotation

You can add footnotes to explain or annotate words. The footnotes annotation takes as parameter the note which will appear at the end of the document. In this example, one footnote is added.

3.12.3. Citations

Citations are only available for LaTeX. You can add citations to your document to reference an element in a LaTeX bibliography. The cite annotationtakes as parameter the key of the reference in the bibliography.

The exemple above wil render as cite{reference}

If you want to use other type of citations like citep or citet, please overwrite the command in your LaTeX template: renewcommand{cite}{citep}

á vérifier

3.12.4. Slide Annotation

This annotation is used to create slides structure for a beamer or a deck.js export. The parameter title is required. The label parameter can be used to reference this slide in another slide:

3.12.5. Columns

With Pillar you can put text and other contents in columns. To do that, you need to delimit an environment with the columns and endColumns annotations. Then you can create columns with the column annotation. The column annotation takes 1 required parameter: the width of the column. Here is an example:

The column annotations currently works only for the beamer, HTML and Deck.js export.

3.13. Preformatted (less used)

To create a preformatted block, begin each line with =. A preformatted block uses equally spaced text so that spacing is preserved. In general you should prefer scripts over preformatted blocks.

= this is preformatted text
= this line as well

3.14. Commented Lines

Lines that start with a % are considered comments and will not be rendered in the resulting document.

3.15. Escaping Characters

Special characters (e.g., + and *) must be escaped with a backslash: to get a +, you actually have to write \+. The list of characters to escape is:

In this section we show how to configure the export.

4.1. Configuration File

Pillar exporting mechanism can be configured using STON (see chapter @ston), a lightweight, text-based, human-readable data interchange format (similar to the popular JSON). Configuration is done either in pillar.conf or at the beginning of Pillar files.

baseDirectory

Indicate where to look for files.

Default value: The current working directory

configurations

Each configuration can define several sub configurations, each of which inherits the properties of its parent.

Default value: A dictionary of default configurations from the exporters.

defaultExporters

Collection of exporters to use when none is explicitely specified. You can specify the exporter you want through the --to= command-line argument.

Default value: By default only the text exporter is enabled.

defaultScriptLanguage

Indicate the language in scripts when none is specified. This language is used for syntax highlighting. The currently supported languages are bash, css, html, http, json, javascript, pillar, sql, ston, shellcommands and smalltalk.

Default value: An unspecified language

disabledPhases

Collection of phases that Pillar should ignore.

For exemple, a value of ["scriptEvaluator", "section", "transform"] will disable script evaluation (useful when security is important), sectioning (useful when generating HTML 4) and the transform phases (i.e. all the transformers) .

Default value: By default the collection is empty, i.e., all phases are active.

Indicate how to convert from the level of a Pillar heading to the level of heading in your exported document. For example, a headingLevelOffset of 3 converts a 1st level Pillar heading to an <h4> in HTML.

Default value: 0

inputFile

The Pillar file that must be exported. You can also specify an input file at the end of the command-line interface.

Default value: nil

level1

Configure how headers at level 1 will be rendered. Value must be a dictionnary. These keys are recognized:

numbering
a boolean indicating if headers at this level must be numbered
size
a positive number indicating how many parent levels should be visible in the number: e.g., if 2, the parent header's number and the current header's number will be shown (must be lower than or equal to 1)
renderAs
a string indicating how the numbering is done (must be one of "number", "roman", "letter" or "upperLetter")

Default value: All levels are numbered with digits and all parents are visible.

level2

Configure how headers at level 2 will be rendered. Value must be a dictionnary. These keys are recognized:

numbering
a boolean indicating if headers at this level must be numbered
size
a positive number indicating how many parent levels should be visible in the number: e.g., if 2, the parent header's number and the current header's number will be shown (must be lower than or equal to 2)
renderAs
a string indicating how the numbering is done (must be one of "number", "roman", "letter" or "upperLetter")

Default value: All levels are numbered with digits and all parents are visible.

level3

Configure how headers at level 3 will be rendered. Value must be a dictionnary. These keys are recognized:

numbering
a boolean indicating if headers at this level must be numbered
size
a positive number indicating how many parent levels should be visible in the number: e.g., if 2, the parent header's number and the current header's number will be shown (must be lower than or equal to 3)
renderAs
a string indicating how the numbering is done (must be one of "number", "roman", "letter" or "upperLetter")

Default value: All levels are numbered with digits and all parents are visible.

level4

Configure how headers at level 4 will be rendered. Value must be a dictionnary. These keys are recognized:

numbering
a boolean indicating if headers at this level must be numbered
size
a positive number indicating how many parent levels should be visible in the number: e.g., if 2, the parent header's number and the current header's number will be shown (must be lower than or equal to 4)
renderAs
a string indicating how the numbering is done (must be one of "number", "roman", "letter" or "upperLetter")

Default value: All levels are numbered with digits and all parents are visible.

level5

Configure how headers at level 5 will be rendered. Value must be a dictionnary. These keys are recognized:

numbering
a boolean indicating if headers at this level must be numbered
size
a positive number indicating how many parent levels should be visible in the number: e.g., if 2, the parent header's number and the current header's number will be shown (must be lower than or equal to 5)
renderAs
a string indicating how the numbering is done (must be one of "number", "roman", "letter" or "upperLetter")

Default value: All levels are numbered with digits and all parents are visible.

Each document can have metadata like title or authors

Default value: A dictionary of default default metadata

newLine

The string that separates lines in the exported document. This is often either LF or CR+LF but any string is possible.

Default value: Depend on the operating system.

outputDirectory

Indicate where Pillar will create generated files.

Default value: The value of baseDirectory

outputFile

If separateOutputFiles is false, indicate the name of the output file. This can also be a write stream.

Default value: A file named 'output' with an extension depending on outputType.

outputType

Indicate the kind of output desired. Can be any of text, html, asciidoc, pillar, latex, mock, markdown, deckJS, xhtml, latex:sbabook, beamer, githubmarkdown, navmenu and tocmenu.

Default value: nil

renderStructureAsSlide

When true (the default), Pillar will create a dedicated slide for each Pillar header. This parameter is meaningless when generating a written document.

Default value: true

Indicate if email addresses should appear scrambled to defeat the stupidest spammers looking for them (the default). If false, email addresses will appear unscrambled.

Default value: true

separateOutputFiles

If true, each input file is exported to one output file. If false (the default), all input files are exported to outputFile.

Default value: false

slideInTemplateForDeckJS

Indicate the number of slides created by the DeckJS template. This is important to create anchors.

Default value: 1

verbose

Indicate whether Pillar should write a verbose log when exporting.

Default value: false

5. Templating

Pillar generates json as output so you can use a templating engine to tweak Pillar output. Pillar comes with the Mustache templating engine (see chapter ). This means you can specify a preamble and postamble for your document. Here is an example HTML template using Mustache:

In this example, we can see the use of {{{title}}} and {{{content}}} to refer to the title of the document and its actual content (the one exported from Pillar). You have to put such a template in a dedicated file (named chapter.html.template for example) and reference this file from the HTMLTEMPLATE variable in the Makefile. Then when you will compile in HTML, the Makefile will use this template.

You can also use mustache alone with command:

6. Command-Line Interface

In this section we show how to use the Pillar command-line interface.

One of the basic uses of the command line is:

You can select an export type with the parameter --to. The possible exports are: text, html, asciidoc, pillar, latex, mock, markdown, deckJS, xhtml, latex:sbabook, beamer, githubmarkdown, navmenu and tocmenu.

The Makefile will create a symbolic link named root referencing the output directory into each directory containing output files. You can use this symbolic link to reference files in specified in the support collection.

7. Pillar from Pharo

Pillar has a document model (the root of which being PRDocument), a parser (PRPillarParser) and several export types (subclasses of PRDocumentWriter) implemented as visitors over the document model. Pillar also has phases (subclasses of PRPhase) that take a document model as input and produce a modified document model as output.

7.1. How to Create a Pillar Document

It is possible to create a Pillar document by parsing a string or by instantiating the document model.

Creating a document by parsing a String requires using the PRPillarParser:

Or from a file:

You can also instantiate the document model, one node after the other, starting with PRDocument and adding sub-instances of PRDocumentItem:

7.2. How to Export a Document

Once you have your document, you may want to export it. But exporting, there's an optional step: transforming your document. A transformer is an abstraction that visits a document and changes it. For instance, PRScriptEvaluator replaces a script with eval=true by the result of its evaluation. Exporting is done with a subclass of PRDocumentWriter, like this:

To specify export parameters (see above for a comprehensive list), you may want to use a configuration.

8. Conclusion

Pillar is still in active development because authors keep writing new documents. Because Pillar's source code is of great quality (mainly due to Lukas Renggli), features can be added easily by new developers. Pillar is different from competition thanks to its notion of project that allows managing multiple files coherently.