"Code Insert" function adds characters and causes compiler malfunction when copy

[trastikata]

Hello,

HAL asked me about a problem with my recent code post. "Code Insert" function adds invisible to Proton IDE characters and causes compiler malfunction therefore we decided to post and make people aware of the issue.

Thus the "Code Insert" function adds characters and causes compiler malfunction if you copy and paste the code snippet - I noticed the issue some time ago but I was lucky to use VS Code where you can immediately see the additional characters. 

Few screenshots from VS Code and Proton IDE showing the error

[John Lawton]

But what are these characters, a hex viewer/editor should identify them.

John

[trastikata]

But what are these characters, a hex viewer/editor should identify them.

I am not sure John. If you copy the text from the forum in my latest post for example after "EndProc" and post it in an ASCII converter it will give ASCII 32, which is Space:

69 110 100 80 114 111 99 32

However after pasting the same text in Proton IDE and VS Code, that character is converted to 194 160:

69 110 100 80 114 111 99 194 160

I'd guess it is more a OS issue than a forum issue?

Edit: Looking through Google results, it seems a common occurrence, it's an interesting problem that I wasn't aware of.

[John Lawton]

I've had this in the distant past when code suddenly doesn't work anymore and there appears to be nothing wrong with it. Unfortunately IDEs don't display all ASCII characters so unwanted but hidden ones can creep into the code and cause problems.

[trastikata]

I've had this in the distant past when code suddenly doesn't work anymore and there appears to be nothing wrong with it. Unfortunately IDEs don't display all ASCII characters so unwanted but hidden ones can creep into the code and cause problems.

John,

I've made it a habit first checking in VS Code whenever a problem occurs. It does display the hidden characters, look at the first two screenshots. I've indicated where the problematic characters are. 

That's why HAL suggested opening this topic to make people aware and hopefully find a solution.

[Dompie]

Hmmmm but &nbsp (194 160) is an HTML encoding so it could still be a forum issue, although UniCode has also defined this No-Break Space.
I found the following Wikipedia description clear "is a space character that prevents an automatic line break at its position."

I assume that there is a space at the end of the relevant lines, which is then converted to a &nbsp by the "Code Insert" function. This seems to me to be the most plausible situation because typing a &nbsp yourself requires some skill (Alt + 0160) and is not entered accidentally.
So I think this phenomenon is a bug of the "Code Insert" function.
Now that I know, I will be very critical when copying code from the forum.

Thanks for the information.
Johan

[Dompie]

This is very intriguing. I selected and copied the code ( this one ) and then entered it into UltraEdit and turned on the HEX view. I don't find any &nbsp characters. And indeed there are four Positron statements in the source that are closed with <space> <CR> <LF> ($20 $0D $0A) and those are exactly the wrong lines. All other Positron lines are terminated with <CR> <LF>
But if I copy the relevant source from the forum and paste it into Proton IDE, Positron compiles this source without any problems??? ?? Would there be Windows 10 settings that prevent this problem for me?

Johan

[HAL]

Thanks trastikata

Your post was a superb description and explanation of the problem.

I have not noticed it on all of the code that is posted.  But if I find it again, I will remove the non-printing characters and post the code again (using W7) to see if the problem is related to the W10 operating system. 

Best regards and thanks

Hal

[HAL]

Hello everyone

This is a line of code with an extra (unprintable character) at its end.

Dim wTimer1 As TMR1L.Word

Note that if you highlight it by selecting the line, the highlighted portion will extend one space beyond the last letter (the d in "Word" in this example) as shown by trastikata.  The character does not display in the IDE nor in notepad.  The free program Notepad++ will allow this extraneous character to be displayed by switching to ANSI encoding.

Using Notepad++ and changing to ANSI encoding, the same line of code with an extra (unprintable character) will display as:

Dim wTimer1 As TMR1L.WordÂ

The easiest "work around" seems to be going to the lines in error move out a few spaces beyond the end of the code and then backspace to the end of the last visible character or word.  If you see an error flagged on a blank line, then the the error is at the beginning of the line.  Just delete the line and the IDE/compiler should be happy.

Hal

[HAL]

In the previous post the line with the extra character was posted as text.
I tried to copy and paste that line from the post and the character seemed to have become a space.

Code Select Expand

Dim wTimer1 As TMR1L.Word 
Dim wTMR1Overflow As Word
Dim dPulseCount As Dword
Let's post the same line as code and see what we get back.

[HAL]

Hi Gents

What seems to happen is that if I upload code with the extra character via W7 (whether as text or as a code file) and then download the very same statement, the previously offending character seems to become a space.  Perhaps someone can/would download with W10 and see if the results are different.  Can we conclude that W7 will download the error but will not upload it?  Decidedly strange stuff.

Best regards
Hal

[Dompie]

Your code from post #9 with Win10 in UltraEdit in HEX view is in the attachment.
Only the first sentence has a <Space> at the end.

Johan

[John Lawton]

Windows 7. I get the same result as Johan by pasting the sample in #9 into Notepad++ then saving to a file and viewing the file using Hexedit.

[HAL]

Hi

You would have to download trastikata's original code file to see the error because my windows 7 will only download the errant character. 

When I upload the code with the extra character, W7 uploads it as a space.  Good old W7!

That was my (perhaps not so clear) explanation in post #10.

Best
Hal 

[HAL]

Hello

Back again.  I used Ultraedit to look at the three lines and print the hex representations of the characters.  The errant added character is hex A0 and occurs in three places.  When or if I upload with W7 to the forum it changes the hex A0 to hex 20 a space.  So I expect the upload problem may be with W10.  W7 will copy the error but will not paste it.

00000000h: 0D 0A 44 69 6D 20 77 54 69 6D 65 72 31 20 41 73 ; ..Dim wTimer1 As
00000010h: 20 54 4D 52 31 4C 2E 57 6F 72 64 A0 0D 0A 0D 0A ; TMR1L.Word ....
00000020h: 54 31 47 47 4F 20 3D 20 31 A0 0D 0A 0D 0A 52 50 ; T1GGO = 1 ....RP
00000030h: 49 4E 52 31 32 20 3D 20 25 30 30 30 30 31 31 30 ; INR12 = %0000110
00000040h: 30 A0 0D 0A                                                             ; 0 ..


Sorry about post 9.  Only line 1 had the error.  Whoops/Duh.

The three lines in error are:

Dim wTimer1 As TMR1L.Word

T1GGO = 1

RPINR12 = %00001100 ..

The first post by trastikata shows the entire program and exactly where the extra hex A0 code appears.

Good day  Hal