Markup language flavours

Destry · 2016-03-16 02:01:10

One good and big reason to be glad that Textile is not as popular as Markdown, as I extract from this article about Markdown as a poor choice for documentation development, is that not many flavours of Textile exist.

(Par contre, Markdown : Holy crap.)

Hacker News follow-up on the author’s post.

As noted by the author, CommonMark has been slow to be adopted. That’s no surprise to me considering:

Gruber has a loyal following and was against the standardization of Markdown to begin with, and
“standards” can leave a bad taste in a lot of mouths, especially open source crowds.

(I’m fine with standards, personally.)

Bloke · 2016-03-16 10:05:31

Not sure if this is the right place to mention this, but all this talk of markup languages and their (un)suitability sparked some thoughts in my head.

I like Textile because it does lots. But it has a tonne of features that are marginally useful at best, and in a modern web don’t belong there. e.g. the ability to do inline styles and horizontal/vertical alignment: it’s old hat. Classes are better, but there’s a neater way to handle them than the way Textile does.

I like Markdown because some parts of it make sense. Popular or not, it does have some nice ideas. But other parts are just too unwieldy or it’s too limiting for powerhouse editing of articles. Lack of across-the-board classes for starters. MarkdownExtra includes some much needed functionality, but it’s plagued by the core Markdown feature set, which is limited compared with the flexibility of Textile.

Both have inconsistencies that make memorisation of shortcuts harder. So in my head, I have this grand idea of marrying the two. Yes, a new markup language that takes the best bits of both worlds and combines them. A stripped-down Textile with a souped-up Markdown that I’d call, I dunno, TextMark (hehehe, ^TM) or something wacky like that. Totally extensible and written in PHP at first since that’s my language-du-jour, but I’d license it openly.

Even with WYSIWYG editors, I think there’s room in the market for a better markup system for content editing that can grow with a user’s or platform’s needs. Projects like Grutatext, Setext, reStructuredText and so forth are either niche products or horrible for natural language content. And with Txp 4.6’s ability to have third party extensions, who knows, it might just be useful enough to many to install.

I have a complete document outlining my broad plan for such a language. If anyone’s interested I can post it here.

philwareham · 2016-03-16 10:19:57

Bloke wrote #298296:

I have a complete document outlining my broad plan for such a language. If anyone’s interested I can post it here.

Whilst I am interested in your new markup language, I am more interested with barriers to entry for new users. Whether any of us here (who I assume are already Textpattern users) like/dislike Markdown is moot, it is pretty much expected by new users due to its prevalence everywhere else. There are also many many WYSIWYG solutions built on top of Markdown which we are pretty much locked out of due to our steadfastness to Textile.

Markdown should be supported directly and shipped in the core ASAP, and possible by the default option for new installs too (that is well up for debate). Anything less and you can pretty much give up hope of attracting future users.

For my part, I quite like Textile, but I’m not oblivious to its incredibly niche usage compared to Markdown.

Bloke · 2016-03-16 10:30:54

philwareham wrote #298298:

I am more interested with barriers to entry for new users.

Absolutely. Markdown in core too… sure why not, it’s one extra file (just rip wet’s plugin and use Parsedown as it’s way faster).

At one point we were also looking at WYSIWYG editors as default. Again, great idea, as long as we remember that each installed markup language requires its own bar so having two installed does increase that complexity and footprint somewhat.

But in terms of barriers for new users, neither Markdown nor Textile completely fit the bill. They’re both great at some things and suck at others. Mostly this is due to them offering legacy support for features that don’t make sense in the HTML 5 world. A system that can take either syntax, but make it better / more consistent / more useful means both sets of users are happier in the long term. That’s what I’m proposing.

philwareham · 2016-03-16 10:39:12

This was a great quote from Hacker News:

Markdown has a lot of faults, but it solves the one single biggest problem in writing documentation that basically overrides every single other consideration: getting people to actually write it. Basically nothing else matters except this.

Bloke wrote #298299:

At one point we were also looking at WYSIWYG editors as default. Again, great idea, as long as we remember that each installed markup language requires its own bar so having two installed does increase that complexity and footprint somewhat.

It does, but I hope someday one of the existing Markdown formatting bars – or rah_textile_bar – could be used as a basis for a bar that can be used for both syntaxes.

Bloke · 2016-03-16 10:50:33

philwareham wrote #298300:

I hope someday one of the existing Markdown formatting bars – or rah_textile_bar – could be used as a basis for a bar that can be used for both syntaxes.

*dreams* that’d be grand.

Regarding the Hacker News quote. Yes, precisely. But it goes beyond that. Who wants to write content full stop these days? How many times has someone approached you and said, “Can you make me a website?” to which you reply: “Sure, where’s your content?” to which the reply is “We’ll worry about that later, just show me a design. Here’s some Lorem Ipsum for the time being.” Noooooooo!

Markdown syntax is nice from a newbie perspective, which is why I think a markup system that supports it is important. But it doesn’t go far enough to allow you to grow when you want to do something a little bit out of the ordinary (e.g. adding classes or ids, which aren’t universally supported).

Textile goes too far so dialling it back and supporting both Markdown and Textile syntaxes (in a reduced facility) makes sense to me. Thus you have the immediacy of Markdown with the flexibility of Textile, and some common sense conventions that make both easier to remember and apply.

philwareham · 2016-03-16 11:05:21

Bloke wrote #298301:

Markdown syntax is nice from a newbie perspective, which is why I think a markup system that supports it is important. But it doesn’t go far enough to allow you to grow when you want to do something a little bit out of the ordinary (e.g. adding classes or ids, which aren’t universally supported).

Textile goes too far so dialling it back and supporting both Markdown and Textile syntaxes (in a reduced facility) makes sense to me. Thus you have the immediacy of Markdown with the flexibility of Textile, and some common sense conventions that make both easier to remember and apply.

That would be nice – a syntax that combines both of those syntaxes. But unfortunately certain parts of Markdown and Textile directly clash (things like list # in Textile makes a heading in Markdown) so I don’t have a clear idea of how that could work.

CommonMark seems to be the gold standard for Markdown when it gets finalised soon (I’m not worried about perceived slow adoption of it right now, since it hasn’t actually been officially released as v1.0 yet).

Bloke · 2016-03-16 11:11:42

philwareham wrote #298302:

Things like list # in Textile makes a heading in Markdown so I don’t have a clear idea of how that could work.

~~Easy to distinguish. If a blank line above, it’s the start of a list, otherwise it’s a heading.~~

EDIT: Oh, I get what you mean. That stupid syntax of specifying multiple # symbols before a heading. Yeah, that is tricky. That syntax always seems stupid to me. The underline one is much better, imo.

Last edited by Bloke (2016-03-16 11:21:05)

Bloke · 2016-03-16 12:14:51

Textup

A mashup of a simplified Textile and souped-up Markdown: best of both worlds.

Core principles:

Blazingly fast parser

:-) Goes without saying. The regex-based approach of Textile makes for a nice jump-start, removing the cruft, adding callbacks and folding in the alternative Markdown syntaxes.

Every shortcut tag is registered and is overridable / extensible

Callbacks enable people to tweak tags to their liking or add new ones.

Use parentheses on any shortcut tag

Reject Textile’s parenthetical nonsense in favour of:

Classes (.className1 .className2)
IDs (#id1)
Language (@fr)
Custom data attributes ($key=value)
Arbitrary attributes (key) or (key=value)

Note that arbitrary attributes could be prefixed with data-, which is a shorthand for the $ notation that also allows for empty data-* attributes to be defined.

Doing this has a few benefits:

Classes use a familiar jQuery-esque syntax, preceded by dot.
Simpler. One set of parentheses enables better (and faster) regex parsing, which in turn…
… means the trailing Textile dot is only necessary if you choose not to use parentheses.

Example 1

p. I'm a regular paragraph.

Renders:

<p>I'm a regular paragraph.</p>

Example 2

p(.pullout) ...

Renders:

<p class="pullout">...</p>

Example 3

p(.pullout .highlight) ...

Renders:

<p class="pullout highlight">...</p>

Example 4

p(.pullout #myId) ...

Renders:

<p class="pullout" id="myId">...</p>

Example 5

p(@de .pullout)Sprechen Sie Deutsch!

Renders:

<p lang="de" class="pullout">Sprechen Sie Deutsch!</p>

Example 6

p(.rhyme title="Short and stout" hidden) I'm a ltttle teapot.

Renders:

<p class="rhyme" title="Short and stout" hidden>I'm a ltttle teapot.</p>

Example 7

p(.pullout $ref=special) Custom data attributes, FTW.

Renders:

<p class="pullout" data-ref="special">Custom data attributes, FTW.</p>

Omissions compared with Textile

No horizontal or vertical alignment (use classes instead).
No curly-brace inline styles (use classes, or parentheses, like p(style="border: none;"), or raw HTML instead).
No square-bracket language designators.
No automatic abbreviation (because it sucks).

Omissions compared with Markdown

No hash-marks to designate heading level, as it conflicts with numbered lists. Use equals signs instead.

Default tags

Paragraphs are automatically assumed if separated by blank line, or using p to escape previous blocks.

Single line breaks render <br> tags.

Block-level markup

Asterisks (or bl) for bullet lists. Multiple recurrence for nested lists. Hanging indent optionally supported.

Hash symbol (or nl) for numbered lists, with continuation (e.g. #6 Starts at number 6, #_ continues previous numbering after other block). Multiple recurrence for nested lists. Hanging indent optionally supported.

Semicolon (or dl) for definition lists. Subsequent <dt> and <dd> elements use colon at start of line, as per Textile.

Square brackets for footnote and auto-numbered links. Same implementation as per Textile using fn, with optional exclusions and backlinks (Wikipedia-style).

Three or more hyphens with blank lines above and below renders an <hr> separator. Alternatives as per Markdown permitted.

Double-backtick (or bc) for multi-line pre + code block that may NOT contain blank lines.

Triple backtick (or bc+) for multi-line pre + code block that may contain blank lines.

Greater-than symbol (or bq) for blockquote. If a double-hyphen and some text appears on its own line, treat as citation and end the blockquote.

Double-greater-than (or bq+) for multi-line blockquotes. If a double-hyphen and some text appears on its own line, treat as citation and end the blockquote automatically. Blockquotes can contain other markup (e.g. lists with # or *). And vice versa: other block markup such as lists can contain blockquotes. Similar to Markdown.

Equals sign (or h[num]) for heading level [num]. Alternatives:

Underline by >1 equals sign for h1.
Underline by >1 hyphen for h2.
Underline by >1 tilde for h3.
Underline with >0 equals signs + number to denote heading level.
Put [n] equals signs on same line as heading to denote that heading level.

(Can use parentheses on any of these: see above).

Tables: tbd. Markdown’s syntax is nice, but + symbols for corners also has merit.

<section>, <div>, <aside>: tbd.

Comments: tbd.

Inline markup

Asterisk for <strong>.

Underscore for <em>.

Asterisk and underscore (any order) for strong emphasis.

Plus for insertion (<ins>).

Hyphen for deletion (<del>).

Backtick for inline code.

Caret for superscript (<sup>). May require square bracket hack to permit space to be gobbled up (e.g. degrees-celsius: 23[^o^]).

Tilde for subscript (<sub>). May require square bracket hack to permit space to be gobbled up (e.g. chemical formulae H[~2~]O).

Less-than symbols are also encoded as < unless they form part of other tags.

Links come in a few flavours:

First, regular Textile quoted-colon inline links. Self-links using "$":http://textup.io or <http://textup.io> are supported.

Second, references for footnote-style links that get rendered inline.

Example:

my "link here":[ref]

and then, somewhere else in the doc on its own line:

[ref] http://link.com. "Optional title."

Cross-ref this with named auto-numbered footnotes.

Abbreviations are not automatically encoded like in Textile. You need to explicitly define them, inline- or reference-style. Case sensitive.

Inline-style example.

Refer to the [W3C]:(World Wide Web Consortium) for more on HTML.

Reference-style example that would add an <abbr> to the above sentence around HTML:

[HTML]: Hyper Text Markup Language

Empty definitions are permissible, which are rendered as <abbr> without title attribute.

Image: tbd. Both systems currently use an exclamation mark. Neither are perfect.

Span: tbd. Percent (like Textile) is okay.

Backslash escapes any specific markup tags

Wherever this occurs, tags are rendered verbatim. Three backslashes escape entire blocks.

Other general examples

My heading
==========

Renders: <h1>My heading</h1>

My heading
----------

Renders: <h2>My heading</h2>

My heading
~~~~~~~~~~

Renders: <h3>My heading</h3>

h4(.fourth) My heading

OR:

My heading
=4(.fourth)

OR:

====(.fourth) My heading

Renders: <h4 class="fourth">My heading</h4>

Lots to work out yet, including a shortcut way to specify syntax highlighting language. This is just the preliminary work on combining and standardising. Comments welcome, including comments such as “go away, you stupid man, there’s no need for this abomination in the world”.

philwareham · 2016-03-16 12:31:49

That actually looks pretty great. I love the way you can append any attribute you like to a tag.

Bloke · 2016-03-16 12:40:49

philwareham wrote #298305:

That actually looks pretty great. I love the way you can append any attribute you like to a tag.

:-) Future proof, baby yeah!

Destry · 2016-03-16 13:10:05

I love the idea of a hybrid language. A best of both and not the cruft.
I agree with Phil about the barriers with other popular editing tools, however, and that…
Markdown should be shipped in Txp ASAP.
“TextMark” is a pretty good label, but I dare say Textup — [T↑] — would be one better, and a direct response to Markdown, [M↓]. I’d rather not see another intercapped tech name, which just screams “developer schtick”. ;)
h1(.first) My heading, etc, is definitely the easiest; not only for veteran Textile users, but logically against HTML, thus new users. That style of mark is one of the things I really like about Textile over Markdown; the direct implication with the associated HTML elements.
Please create hr(.class)
Please create dl(.class)
Please create a table of contents mark; e.g., toc(.n), where “n” indicates the max level of headings (6 being max) to generate from a given article’s existing headings.
Don’t lose footnotes. ;)

Textpattern CMS support forum

#1 2016-03-16 02:01:10