Little endian to Big endian efficiency?

[trastikata]

Hello all,

I am writing some code and I need efficient Little endian to Big endian 32b conversion. Presently what I do is byte swap:

Dim dTemp As Dword = 0x03020100

Swap dTemp.Byte3,dTemp.Byte0
Swap dTemp.Byte2,dTemp.Byte1

Since this takes only 8 instructions I am happy with that, but I was wondering if there's a more efficient way to that conversion?

[top204]

At first, I thought you meant Little-Endian to Big-Endian bit-wise, not byte-wise, which would take a loop using the Carry flag to holds the bit rotated out, then rotated the opposite way into the variable via a temporary variable. :-)

The code I created to swap bytes took a couple of years to optimise whenever I added something to the compiler, in-order to create the present efficiency of the Swap code. It uses the WREG for temporary storage and some querks with mnemonics to manipulate the bytes through the WREG to each other, and was originally written, about 21 years ago, for the standard 14-bit core devices that do not bring out the WREG as an SFR.

Code Select Expand

F1_000049 equ $ ; in [TEST_18F25K22.BAS] Swap dTemp.Byte3,dTemp.Byte0
    movf dTempHHH,W,1
    subwf dTemp,W,1
    addwf dTempHHH,F,1
    subwf dTemp,F,1
F1_000050 equ $ ; in [TEST_18F25K22.BAS] Swap dTemp.Byte2,dTemp.Byte1
    movf dTempHH,W,1
    subwf dTempH,W,1
    addwf dTempHH,F,1
    subwf dTempH,F,1

I still remember trying many different code pieces to get the most efficient method I could, and the same with a '1 << variable' expression, and others. :-)

The only time it will become larger is if the bytes are in different RAM banks. So on a multi-byte variable that may be stradling a RAM bank, it is better to add:

Declare Auto_Variable_Bank_Cross = On

So multi-byte variables are always within the same RAM bank. See page 421 of the Positron8 compiler's manual.

On an 18F device, you could use the code below, but it still takes 8 instructions, because the Movff mnemonic takes two words, and does not disturb the contents of WREG, and if a variable is sitting above RAM address 4095, the compiler will need to use the Movffl mnemonic, which takes three words. But the Swap commands look neater in the code listing.

Code Select Expand

    WREG = dTemp.Byte0            
    dTemp.Byte0 = dTemp.Byte3
    dTemp.Byte3 = WREG
   
    WREG = dTemp.Byte1
    dTemp.Byte1 = dTemp.Byte2
    dTemp.Byte2 = WREG

If someone can make the code used to swap two bytes even more efficient, please do so, and I will add it to the compiler's assembler creater for the Swap command.

[trastikata]

At first, I thought you meant Little-Endian to Big-Endian bit-wise, not byte-wise, which would take a loop using the Carry flag to holds the bit rotated out, then rotated the opposite way into the variable via a temporary variable. :-)

Thank you Les, I didn't think it could get faster than 8 instructions, but still could have missed some "trick".

I've started writing a simplified 32 bit NTFS support for SD cards, which is much simpler than FAT32 at the basic file-folder support level, but all offsets in the MFT are stored as byte-wise Little-Endian's.

[top204]

Can you not actually read the SD card in reverse at low level when required, so the result is little-endian to start with?

I was going to look at the "different" format micro$oft now use for FAT32, which is "FAT32 Extra", but never found the time to do it. Why on earth would they change such a stable format, and in their Format window, so it will not actually format as standard FAT32?

[trastikata]

Can you not actually read the SD card in reverse at low level when required, so the result is little-endian to start with?

Unfortunately for SD cards you set the sector number to be read or written and you clock 512 bytes out or in which is always in one direction. 

[top204]

Yes. You are correct. I had totally forgotten that the whole sector is normally read and then parsed. However, many years ago, I created a FAT16 read mechanism that did not use a RAM buffer, and each section of a sector was read and parsed as it was read in, and it counted how many bytes were read from a sector and any that were left over, it clocked in and discarded them. So the SD card still saw 512 bytes being read. It made the reading of an SD card extremely efficient and fast, but with limits on complexity of other commands on a FAT system.

I had to do that because the original standard 14-bit core devices did not have 512 bytes of RAM in them, and even the first versions of the 18F devices did not have a lot of RAM in them to waste on a RAM buffer. Wow... We're going back some years now... About 20 years ago! :-)

[top204]

Or you can create an alias of the variable/s being used, but reversed:

Code Select Expand

    Dim dTemp          As Dword             ' Create the variable  
    Dim bTemp_Dummy    As dTemp.Byte3       ' \
    Dim bTemp_DummyH   As dTemp.Byte2       ' | Create aliases to reverse the bytes within dTemp
    Dim bTemp_DummyHH  As dTemp.Byte1       ' |
    Dim bTemp_DummyHHH As dTemp.Byte0       ' /
    Dim dTemp_Rev      As bTemp_Dummy.Dword ' Create an alias of dTemp, but with the bytes reversed
   
    dTemp_Rev = $12345678

Then there is no reversing code required, and if dTemp_Rev is used to read the SD card's value, or in an expression, the result will be the bytes reversed in dTemp.

[trastikata]

Or you can create an alias of the variable/s being used, but reversed ...

Thank you Les, this is really what I was looking for.

Takes a bit more careful thinking when dimension the variables, but simplifies and speeds up the code afterwards.

[top204]

You can automate it, somewhat, by using the preprocessor. As the code listing below shows:

Code Select Expand

'
'   /\\\\\\\\\
'  /\\\///////\\\
'  \/\\\     \/\\\                                                 /\\\          /\\\
'   \/\\\\\\\\\\\/        /\\\\\     /\\\\\\\\\\     /\\\\\\\\   /\\\\\\\\\\\  /\\\\\\\\\\\  /\\\\\\\\\
'    \/\\\//////\\\      /\\\///\\\  \/\\\//////    /\\\/////\\\ \////\\\////  \////\\\////  \////////\\\
'     \/\\\    \//\\\    /\\\  \//\\\ \/\\\\\\\\\\  /\\\\\\\\\\\     \/\\\         \/\\\        /\\\\\\\\\\
'      \/\\\     \//\\\  \//\\\  /\\\  \////////\\\ \//\\///////      \/\\\ /\\     \/\\\ /\\   /\\\/////\\\
'       \/\\\      \//\\\  \///\\\\\/    /\\\\\\\\\\  \//\\\\\\\\\\    \//\\\\\      \//\\\\\   \//\\\\\\\\/\\
'        \///        \///     \/////     \//////////    \//////////      \/////        \/////     \////////\//
'                                  Let's find out together what makes a PIC Tick!
'
' A set of preprocessor meta-macros that will create an alias of a, 'byte-wise', reversed multi-byte variable.
' Written for the Positron8 BASIC compiler by Les Johnson.
'
    Device = 18F25K22                                       ' Tell the compiler what device to compile for
    Declare Xtal = 16                                       ' Tell the compiler what frequency the device is operating at (in MHz)
'
' Setup USART1
'
    Declare Hserial1_Baud = 9600
    Declare HRSOut_Pin = PORTC.6
   
-------------------------------------------------------------------------- 
' Create a byte reversed alias to the 32-bit Dword variable passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'pVarname'Rev
' Notes     : The ## will joint two texts via the preprocessor
'
$define Reverse_Dword(pVarname)                 '
    Dim pVarname##Dmy    As pVarname.Byte3      '
    Dim pVarname##DmyH   As pVarname.Byte2      '
    Dim pVarname##DmyHH  As pVarname.Byte1      '
    Dim pVarname##DmyHHH As pVarname.Byte0      '
    Dim pVarname##Rev    As pVarname##Dmy.Dword
   
'-------------------------------------------------------------------------- 
' Create a byte reversed alias to the 24-bit Long variable passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'pVarname'Rev
' Notes     : The ## will joint two texts via the preprocessor
'
$define Reverse_Long(pVarname)                  '
    Dim pVarname##Dmy   As pVarname.Byte2       '
    Dim pVarname##DmyH  As pVarname.Byte1       '
    Dim pVarname##DmyHH As pVarname.Byte0       '
    Dim pVarname##Rev   As pVarname##Dmy.Long
    
'--------------------------------------------------------------------------
' Create a byte reversed alias to the 16-bit Word variable passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'pVarname'Rev
' Notes     : The ## will joint two texts via the preprocessor
' 
$define Reverse_Word(pVarname)                  '
    Dim pVarname##Dmy  As pVarname.Byte1        '
    Dim pVarname##DmyH As pVarname.Byte0        '
    Dim pVarname##Rev  As pVarname##Dmy.Word
 
'--------------------------------------------------------------------------
' Create some variables
'      
    Dim MyWord  As Word                     ' Create a Word variable
    Reverse_Word(MyWord)                    ' Create an alias of MyWord, but reversed and named: MyWordRev
   
    Dim MyLong  As Long                     ' Create a Long variable
    Reverse_Long(MyLong)                    ' Create an alias of MyLong, but reversed and named: MyLongRev
   
    Dim MyDword As Dword                    ' Create a Dword variable  
    Reverse_Dword(MyDword)                  ' Create an alias of MyDword, but reversed and named: MyDwordRev
     
'--------------------------------------------------------------------------
' The main program starts here
'
Main:
    MyDword = $12345678
    HRSOutLn "MyDword   : ", IHex8 MyDword
    HRSOutLn "MyDwordRev: ", IHex8 MyDwordRev
   
    MyLong = $123456
    HRSOutLn "MyLong    : ", IHex6 MyLong
    HRSOutLn "MyLongRev : ", IHex6 MyLongRev
   
    MyWord = $1234
    HRSOutLn "MyWord    : ", IHex4 MyWord
    HRSOutLn "MyWordRev : ", IHex4 MyWordRev

The Reverse_Dword, Reverse_Long, and Reverse_Word meta-macros, create an alias to the variable passed, but reversed.

From the code listing above, the serial terminal will display the texts:

MyDword  : $12345678
MyDwordRev: $78563412
MyLong    : $123456
MyLongRev : $563412
MyWord    : $1234
MyWordRev : $3412

[top204]

Or you can alter the meta-macros and make them create both the variable and the reversed alias all in one, making the code listing easier to manage:

Code Select Expand

'-------------------------------------------------------------------------- 
' Create a byte reversed 32-bit variable and alias of the name passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'Variable'Rev
' Notes     : The ## will join two texts via the preprocessor
'
$define Create_Reverse_Dword(pVarname)          '
    Dim pVarname         As Dword               '
    Dim pVarname##Dmy    As pVarname.Byte3      '
    Dim pVarname##DmyH   As pVarname.Byte2      '
    Dim pVarname##DmyHH  As pVarname.Byte1      '
    Dim pVarname##DmyHHH As pVarname.Byte0      '
    Dim pVarname##Rev    As pVarname##Dmy.Dword
   
'-------------------------------------------------------------------------- 
' Create a byte reversed 24-bit variable and alias of the name passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'Variable'Rev
' Notes     : The ## will join two texts via the preprocessor
'
$define Create_Reverse_Long(pVarname)           '
    Dim pVarname        As Long                 '
    Dim pVarname##Dmy   As pVarname.Byte2       '
    Dim pVarname##DmyH  As pVarname.Byte1       '
    Dim pVarname##DmyHH As pVarname.Byte0       '
    Dim pVarname##Rev   As pVarname##Dmy.Long
    
'--------------------------------------------------------------------------
' Create a byte reversed 16-bit variable and alias of the name passed to the meta-macro
' Input     : pVarname holds the name of the variable to alias and reverse
' Output    : Will be the alias of the variable pVarname, but reversed and named: 'Variable'Rev
' Notes     : The ## will join two texts via the preprocessor
' 
$define Create_Reverse_Word(pVarname)           '
    Dim pVarname       As Word                  '
    Dim pVarname##Dmy  As pVarname.Byte1        '
    Dim pVarname##DmyH As pVarname.Byte0        '
    Dim pVarname##Rev  As pVarname##Dmy.Word
 
'--------------------------------------------------------------------------
' Create some variables
'      
    Create_Reverse_Word(MyWord)                    ' Create a 16-bit Word variable and reversed alias of MyWord, named: MyWordRev   
    Create_Reverse_Long(MyLong)                    ' Create a 24-bit Long variable and reversed alias of MyLong, named: MyLongRev
    Create_Reverse_Dword(MyDword)                  ' Create a 32-bit DWord variable and reversed alias of MyDword, named: MyDwordRev


[trastikata]

Thank you @Les, much appreciate it.

P.s. Les, I think there might be a bug, this code:

Code Select Expand

Dim bSectorBuffer[512] As Byte

Dim dOffset_LE          As Dword At bSectorBuffer#3
Dim bOffset_Dummy       As dOffset_LE.Byte3
Dim bOffset_DummyH      As dOffset_LE.Byte2
Dim bOffset_DummyHH     As dOffset_LE.Byte1
Dim bOffset_DummyHHH    As dOffset_LE.Byte0
Dim dOffset_BE          As bOffset_Dummy.Dword

will result in this Assembler code, note the upper bytes.

Code Select Expand

; ADDRESSED VARIABLES
dOffset_LE equ 0x03
dOffset_LEH equ 0x04
dOffset_LEHH equ 0x05
dOffset_LEHHH equ 0x06
; ALIAS VARIABLES
#define bOffset_Dummy dOffset_LEHHH
#define bOffset_DummyH dOffset_LEHH
#define bOffset_DummyHH dOffset_LEH
#define bOffset_DummyHHH dOffset_LE
#define dOffset_BE bOffset_Dummy
#define dOffset_BEH dOffset_LEHH
#define dOffset_BEHH dOffset_LEHHH
#define dOffset_BEHHH DOFFSET_LEHHHH


[top204]

I had hoped to correct it before you noticed it! :-)

I am now into the compiler's source to see why it is adding too many "H" characters to the byte2 and byte3 #defines. I noticed it as soon as I looked at the assembler code this afternoon to see if the variables were being loaded correctly.

Thankfully, the compiler does not use these defines, and they are there for users to use if writing assembler code etc...

Also... Make your bSectorBuffer array a Heap type, so it  does not fill the standard user RAM area, and the assembler code requiring more RAM bank mnemonics. The Heap directive will make it be created above the standard variables, and out of the way of things.

Even if you have the Declare that makes that automatic, it is better, IMO, to add the Heap directive to a Dim, just in case the Declare is later removed for some reason.

Best Regards
Les

[trastikata]

Thank you for the effort and information Les, it certainly will come handy.

But I've just understood that with so much information about NTFS I've read yesterday, my brains got overwhelmed and refused to assimilate the information anymore.

I got so confused and single headed, that only now, when I saw the assembler code with those variables and some basic code I've written today, I realized that it was the Little-Endian format which aligns with normal variables programming. Damn I feel ashamed