Space-less custom language in syntax-editor

[Peter]

Dear Team

Please help me in this topic. I love this syntax editor solution, it would save a lot of time once i figure out how to solve this:

I use a language called "ZPL". Its a barcode label descriptor language, similar to CNC, PCL or other coordinate-based languages. The problem is that i pciked up the main commands:

  Private Shared ReadOnly Keywords As String() = New String() {"^", "XA", "XD", "FO", "FX", "GB", "XZ", "BY", "FD", "FS"}

But since this language has no space between commands and their first parameter, the syntax editor doesnt make them blue (see the atatched picture)

Is there a way to set such a language syntax?

 

for example in line #5 the

^FO should be blue,
the 50 is red,
the 173 also red (this is ok on the screenshot)
^FD has to be also blue - this means 'FIELD DATA', means: text starts here
^FS this is blue (already ok) - this means theis is the end of the text

Pleas help

Thanks

Peter

[Dess | Tech Support Engineer, Principal]

Hello, Peter,

According to the provided information, it seems that you are using a custom language implementation in RadSyntaxEditor. I believe that you are following a similar approach as the one illustrated in the online documentation:
https://docs.telerik.com/devtools/winforms/controls/syntax-editor/features/taggers/custom-language 

However, since I am not familiar with the exact custom implementation, it wouldn't be easy to determine what causes the undesired behavior on your end. Could you please elaborate? It would be greatly appreciated if you can provide a sample project demonstrating the undesired behavior that you are facing. Thus, we would be able to make an adequate analysis of the precise case and provide further assistance. Thank you in advance for your cooperation.

RadSyntaxEditor is designed to display text for different languages styled in the appropriate way. By using a WordTaggerBase you can classify different words as keyword, identified, literal, etc. You can also have a look at the internal implementation of the built-in CSharpTagger by downloading the source code: https://docs.telerik.com/devtools/winforms/installation-and-upgrades/download-product-files 

In the CSharpTagger there is a StringPattern const defined. It represents the StringMatchingRegex pattern used to match strings in the documents which this tagger recognizes in the GetTags method. The SplitIntoWords method inheriting from the WordTaggerBase class is responsible for splitting the document into words. So if you want to achieve any parsing logic that is not currently available in RadSyntaxEditor, you have the possibility to do it.

The following tutorial demonstrates how to attach the Telerik source code to your project in order to investigate how the built-in CSharpTagger works and splits the words so you can use a similar approach:
https://docs.telerik.com/devtools/winforms/knowledge-base/attach-telerik-source-code-to-your-project 

I am looking forward to your reply.

Regards,
Dess | Tech Support Engineer, Principal
Progress Telerik

[Peter]

Dear Dess,

Yes i can elaborate.  i would like to write a code editor for a language which is not pre-defined in syntax editor.

This language is ZPL, and this is how it looks like:

https://www.zebra.com/content/dam/zebra/manuals/printers/common/programming/zpl-zbi2-pm-en.pdf

It is a barcoedlabel descriptor language, and to now i used an online editor, which shows the result as well on an image:

http://labelary.com/viewer.html

This is the best editor for now, but i would like to create a better one with your syntax editor. If i simply pick up the commands to WordTaggerBase  it results the above picture i attached in my first post. This language doenst have SPACEs between the command and  the command parameter.

For example a line of code looks like this:

^FO190,900^GB170,833,1^FS

This is how syntax editor colors it, if i add the keywords {"^FO","^GB","^FS"}

^FO190,900^GB170,833,1^FS

but this is how i would like to see it (i colored this manually:)

^FO190,900^GB170,833,1^FS

as yuo can see, since there is no space, the syntax editor thinks that ^FO190 is a command, not a command with parameter, so it doenst color it.

This is what i did so far, based on the articles you linked too:

Public Class ZPLTAGGER
    Inherits WordTaggerBase
    Private Shared ReadOnly Keywords As String() = New String() {"^", "XA", "XD", "FO", "FX", "GB", "XZ", "BY", "FD", "FS"}
    Private Shared ReadOnly Comments As String() = New String() {"#"}
    Private Shared ReadOnly Operators As String() = New String() {"+", "~", "*", "/"}
    Public Shared ReadOnly FruitsClassificationType As ClassificationType = New ClassificationType("Fruits")
    Private Shared ReadOnly Fruits As String() = New String() {"^FO", "^FD", "cherry"}
    Private Shared ReadOnly WordsToClassificationType As Dictionary(Of String, ClassificationType) = New Dictionary(Of String, ClassificationType)()
    Shared Sub New()
        WordsToClassificationType = New Dictionary(Of String, ClassificationType)()
        For Each keyword In Keywords
            WordsToClassificationType.Add(keyword, ClassificationTypes.Keyword)

        Next
        For Each preprocessor In Operators
            WordsToClassificationType.Add(preprocessor, ClassificationTypes.[Operator])
        Next
        For Each comment In Comments
            WordsToClassificationType.Add(comment, ClassificationTypes.Comment)
        Next
        For Each comment In Fruits
            WordsToClassificationType.Add(comment, FruitsClassificationType)
        Next
    End Sub
    Public Sub New(ByVal editor As RadSyntaxEditorElement)
        MyBase.New(editor)
    End Sub
    Protected Overrides Function GetWordsToClassificationTypes() As Dictionary(Of String, ClassificationType)
        Return ZPLTAGGER.WordsToClassificationType
    End Function
    Protected Overrides Function TryGetClassificationType(word As String, ByRef classificationType As ClassificationType) As Boolean
        Dim number As Integer
        If Integer.TryParse(word, number) Then
            classificationType = ClassificationTypes.NumberLiteral
            Return True
        End If
        Return MyBase.TryGetClassificationType(word, classificationType)
    End Function
End Class

 

And this is how i call it:

 Dim ZPLTAGGER As ZPLTAGGER = New ZPLTAGGER(Me.RADSYNTAX.SyntaxEditorElement)
        If Not Me.RADSYNTAX.TaggersRegistry.IsTaggerRegistered(ZPLTAGGER) Then
            Me.RADSYNTAX.TaggersRegistry.RegisterTagger(ZPLTAGGER)
        End If
        Me.RADSYNTAX.TextFormatDefinitions.AddLast(ClassificationTypes.NumberLiteral, New TextFormatDefinition(New SolidBrush(Color.Red)))
        Me.RADSYNTAX.TextFormatDefinitions.AddLast(ClassificationTypes.[Operator], New TextFormatDefinition(New SolidBrush(Color.YellowGreen)))
        Me.RADSYNTAX.TextFormatDefinitions.AddLast(ZPLTAGGER.FruitsClassificationType, New TextFormatDefinition(New SolidBrush(Color.LightCoral)))

How should i approach if i would like to achieve the given example? I didnt understand the thing you wrote about SplitIntoWords   , if you could give a little code snippet that would be fantastic.

BR

Peter

[Dess | Tech Support Engineer, Principal]

Hello, Peter,

The provided detailed information is greatly appreciated. I am pasting here the default C# implementation of the SplitIntoWords method of WordTaggerBase to get better understanding of how the document is interpreted:  

        public override IEnumerable<TagSpan<ClassificationTag>> GetTags(NormalizedSnapshotSpanCollection spans)
        {
            TextSnapshot snapshot = this.Document.CurrentSnapshot;

            foreach (TextSnapshotSpan snapshotSpan in spans)
            {
                string inputString = snapshotSpan.GetText();
                var words = SplitIntoWords(inputString);

                int startIndex = snapshotSpan.Start;

                foreach (string word in words)
                {
                    TextSnapshotSpan tempSnapshotSpan;
                    ClassificationType classificationType;
                    if (this.TryGetClassificationType(word, out classificationType))
                    {
                        if (classificationType == ClassificationTypes.Comment)
                        {
                            tempSnapshotSpan = new TextSnapshotSpan(snapshot, Span.FromBounds(startIndex, snapshotSpan.End));
                            yield return new TagSpan<ClassificationTag>(tempSnapshotSpan, new ClassificationTag(ClassificationTypes.Comment));
                            break;
                        }
                        else
                        {
                            tempSnapshotSpan = new TextSnapshotSpan(snapshot, new Span(startIndex, word.Length));
                            yield return new TagSpan<ClassificationTag>(tempSnapshotSpan, new ClassificationTag(classificationType));
                        }
                    }

                    startIndex += word.Length;
                }

                startIndex = snapshotSpan.Start;
                if (!string.IsNullOrEmpty(this.StringMatchingRegex))
                {
                    foreach (Match match in Regex.Matches(inputString, this.StringMatchingRegex))
                    {
                        var tempSnapshotSpan = new TextSnapshotSpan(snapshot, new Span(startIndex + match.Index, match.Length));
                        yield return new TagSpan<ClassificationTag>(tempSnapshotSpan, new ClassificationTag(ClassificationTypes.StringLiteral));
                    }
                }
            }

            if (!this.areMultilineTagsValid && this.EnableMultilineTags)
            {
                this.RebuildMultilineTags();
                this.areMultilineTagsValid = true;
            }

            foreach (var tagSpan in this.MultilineTags)
            {
                if (spans.IntersectsWith(new NormalizedSnapshotSpanCollection(tagSpan.SnapshotSpan)))
                {
                    yield return tagSpan;
                }
            }

            yield break;
        }

        internal static int GetCharType(char c)
        {
            if (c == '#')
            {
                return 0;
            }

            if (char.IsWhiteSpace(c))
            {
                return 1;
            }

            if (char.IsPunctuation(c) || char.IsSymbol(c))
            {
                return 2;
            }

            return 0;
        }

Note that the content is split into words considering whether the character is a letter (part of the word) or punctuation. "^" is considered as punctuation by default and it is split as a separate word:

However, the numbers are considered as a part of the words. It is possible to override the character type for the numbers and thus separate the letters from the numbers:

        Protected Overrides Function SplitIntoWords(value As String) As IList(Of String)
            Dim words As List(Of String) = New List(Of String)()
            Dim word As String
            Dim lastCharType As Integer = -1
            Dim startIndex As Integer = 0

            For i As Integer = 0 To value.Length - 1
                Dim charType As Integer = GetCharType(value(i))

                If charType <> lastCharType Then
                    word = value.Substring(startIndex, i - startIndex)
                    words.Add(word)
                    startIndex = i
                    lastCharType = charType
                End If
            Next

            word = value.Substring(startIndex, value.Length - startIndex)
            words.Add(word)
            Return words
        End Function

        Friend Shared Function GetCharType(ByVal c As Char) As Integer
            If c = "#"c Then
                Return 0
            End If

            If Char.IsWhiteSpace(c) Then
                Return 1
            End If

            If Char.IsPunctuation(c) OrElse Char.IsSymbol(c) Then
                Return 2
            End If
            If Char.IsNumber(c) Then
                Return 3
            End If

            Return 0
        End Function

This is how the split words would like with this implementation:

This is the observed result with the sample input:

 

I believe that you will find the provided information useful. Should you have further questions please let me know.

[Peter]

Dear Dess,

Its almost perfect! If this last issue stays i can live with it, but maybe you can suggest a solution to this too.

^FD is a command, but as you also wrote ^ has to be picked up separately i recorded them separately. But now the FD is not found by the coloring function, because its along with other characters. How can i make blue these too?  (see below image)
Thanks 
P

[Dess | Tech Support Engineer, Principal]

Hello, Peter, 

Indeed, if the "FD" is followed by letters, the "FD" text is not classified as keyword because it is not found as a separate word after splitting the text into words:

The possible suggestion that I can give in this case is to check the split words and see whether each word starts with or ends with "FD". If yes, split this word into two subwords. Thus, you will separate "FD" as a single word and it is expected to be colored as a keyword. Please give this approach a try and see how it would work for your custom scenario.