Add a function to check keywords.txt formatting

[per1234]

Correct format:

KEYWORD\tKEYWORD_TOKENTYPE\tREFERENCE_LINK\tRSYNTAXTEXTAREA_TOKENTYPE

Leading and trailing space on the keyword is ignored. Leading tabs on the keyword are ignored.

Correct separator

Currently the Arduino IDE requires each field to be separated by a single true tab. Spaces may be allowed as separators in a future IDE release (https://github.com/arduino/Arduino/pull/7245) but this will still apply to previous IDE versions.

It is common to see multiple tabs used in an attempt to "prettify" keywords.txt by aligning fields into columns and, at a glance, this appears to work but in fact it will cause keywords to be colored according to editor.function.style regardless of the identifier used because each field is separated by a tab. Two tabs just results in a null value for the field.

Correct identifier

There are actually two keyword identifier fields, each with their own valid identifiers. When an unrecognized identifier is encountered the Arduino IDE 1.6.5 and newer colors the keyword according to editor.function.style (as used by KEYWORD2, KEYWORD3, LITERAL2); Arduino IDE 1.6.4 and older does not color it.

• KEYWORD_TOKENTYPE
  • KEYWORD1
  • KEYWORD2
  • KEYWORD3
  • LITERAL1
  • LITERAL2
• RSYNTAXTEXTAREA_TOKENTYPE
  • RESERVED_WORD
  • RESERVED_WORD_2
  • DATA_TYPE
  • PREPROCESSOR
  • LITERAL_BOOLEAN

With Arduino IDE 1.6.5 and newer, the identifier must not have leading space or it is not recognized (and thus the default editor.function.style is used). With Arduino IDE 1.6.4 and older leading space on the identifier is ignored. Trailing space on the identifier is ignored.

Reference link

The REFERENCE_LINK field provides the filename of the Language reference page, which is located in {Arduino IDE installation folder}/reference/www.arduino.cc/en/Reference/. This page may be opened from the Arduino IDE via right click > Find in Reference or Help > Find in Reference. It's fairly common to see library authors add a REFERENCE_LINK value in their keywords.txt when no reference page of that name exists. Most likely this is due to them using one of the Arduino library keywords.txt as a reference without fully understanding the purpose of the field (unsurprising since it's undocumented)(has since been documented).
Leading/trailing spaces are removed from REFERENCE_LINK.

Is REFERENCE_LINK case-sensitive?

Filename

Spelling the keywords.txt filename with other case will cause it to not be recognized on filename case sensitive OSs such as Linux.

Notes

• Check on esp8266/Arduino repository to see if they implement something before I get to it (reference: Use correct separator in keywords.txt esp8266/Arduino#4461 (comment)). Or if I get to it first submit a PR to them.

[per1234]

Resolved by e7e58c6