Looking for some help with the TMCC Serial interface

Hi Captain -

It looks like you're using a derivative of code I wrote for the Arduino some time ago, but it's impossible to say what the problem is without seeing your code.  Can you post it?

The answer is 28.

Command Input:

{ deviceType: 0, deviceId: 3, commandField: 0, dataField: 11100 }

11100 is the binary value taken from the protocol specification, but the context above is a decimal value.  Try  using 28 in place of 11100.

Look at the equation: const command3 = (deviceId << 7) | (commandField << 5) | dataField

If deviceId = 3 (0b0000 0011), then (deviceId << 7) = 128, when constrained to an 8-bit byte.  I'm guessing this is constrained to a byte value by having deviceId defined as a byte.  This function takes the least significant bit of the device address and puts it in the most significant bit of the byte.  Another way of seeing this is: this function determines the bit value shown in red: 0 0 A A A A A A A C C D D D D D.  If the deviceId is even, (deviceId << 7) is 0.  If the deviceId is odd, (deviceId << 7) is 128.  

The commandField = 0.  Bit-shift the value 0 five places to the left equals 0.

If the dataField is 0b11100 or 28 decimal, then command3 = 128 | 0 | 28 = 156, which you said is the value that make the engine respond with the horn blowing.

Wish I understood any of that.🥴

Ray

@Rayin"S" posted:

Wish I understood any of that.🥴

Ray,

You can.  It's not that difficult.

Many things in this hobby are intimidating until you get to know them.

Electronic communications and networking technology was quite a bit simpler back when Lionel was founded.  You had telephone and telegraph.  No radio, or teletype, or modems using telephone lines, or internet, at that point.

TMCC is just a more modern version of the telegraph, using radio in addition to wires.

Dots and dashes are now 1's and 0's, we send words as well as letters, and machines can now converse with us, or with each other using this.

The only part that's even remotely hard is that these words aren't usually written in English anymore because it's not terribly efficient.  They're now in what we call a 'protocol'.  Just like talking in English, a networking protocol sets rules for constructing letters, words, and sentences, and for what order they are allowed to be sent, so that a conversation doesn't turn into gibberish.  On the other hand if we do hear gibberish, we can  usually just ask for a repeat (What did you say?  Could you repeat it?).

English is a protocol for communications between people, as is German, or Japanese, or Spanish.  Our protocol on the other hand is TMCC, or more recently TMCC with Legacy extensions.  While it might seem very modern, TMCC is already almost 30 years old.

@SirCaptain is having trouble understanding one of the rules in the TMCC protocol, in the same way that you or I might have trouble understanding a new word in English, or someone speaking a common sentence that we would normally understand, but having a foreign accent, or trying to decipher a perfectly normal phrase being heard on the radio while mixed in with bad static.

He's been eavesdropping on a TMCC transmission to help him understand the TMCC protocol, just like someone in Mexico trying to learn English by watching TV transmissions coming across the U.S. border, and he's been stumped by a small detail.

The good news is that we have this forum, to discuss anything model railroading or toy train related, including the challenge that's presently stumping him.

Yes, there are only a very few of us on this forum who understand what he's asked.  For that reason we'd like to invite more of you to learn it.

Our hobby has taught each of us many things over the last 120 years including mechanical, electrical, electronic and artistic skills.  This is just one more for us to learn. 

In the same way that it's taught us all these other things over that time, kids as well as adults, and sent many of us on to rewarding careers using what we've learned, 'communications and networking' are here for us to master and pass on as well.

It's just one more thing, like anything else.

Mike

@Chadford posted:

The answer is 28.

Command Input:

{ deviceType: 0, deviceId: 3, commandField: 0, dataField: 11100 }

11100 is the binary value taken from the protocol specification, but the context above is a decimal value.  Try  using 28 in place of 11100.

Look at the equation: const command3 = (deviceId << 7) | (commandField << 5) | dataField

If deviceId = 3 (0b0000 0011), then (deviceId << 7) = 128, when constrained to an 8-bit byte.  I'm guessing this is constrained to a byte value by having deviceId defined as a byte.  This function takes the least significant bit of the device address and puts it in the most significant bit of the byte.  Another way of seeing this is: this function determines the bit value shown in red: 0 0 A A A A A A A C C D D D D D.  If the deviceId is even, (deviceId << 7) is 0.  If the deviceId is odd, (deviceId << 7) is 128.  

The commandField = 0.  Bit-shift the value 0 five places to the left equals 0.

If the dataField is 0b11100 or 28 decimal, then command3 = 128 | 0 | 28 = 156, which you said is the value that make the engine respond with the horn blowing.

YOU ARE A LIFE SAVER! Turns out all I had to do was use 0b<binary> instead of just <binary> by itself.

In addition - I was pulling my hair out over the the deviceId bitshift... and it turns out it's because Javascript bitshift by default doesn't work quite the same as another language.

deviceId >> 7 was not getting me an 8-bit long 7-shifted decimal, so I eventually found a site(https://onlinetoolz.net/bitshi...e=log&allsteps=1), inspected how it does it logical shift pattern, and built my own hacked together function on top to convert the deviceId into a padded binary, then stepped through the binary until all 7 steps were shifted, and finally converted it back to a base10.

End result: Everything is now working as expected! I'll be writing up a new thread soon on the program, but for now I can finally start implementing the serial commands.

Thanks so much!

I may have spoke too soon (all commands that don't rely on passing in a custom datafield D are working though!). Working on the abs speed step now, passing in the following:

• 0b11 for commandField
• binary representation of abs speed step (0 - 11111)

However I'm guessing it's another input issue, or data type issue, as any abs speed command I pass results in the engine going max speed step.

here's a sample of what I'm passing "abs speed 0"

// function

sendCommand(0o0, 3, 0b11, '00000', 1)

// serial handler

{
commandField: 3,
inputDatafield: 0,
command0: 254,
command1: 1,
command2: 224
}

Binary, Octal, of Hexadecimal representation ... I've always preferred Hex. as I can look at a 3-byte TMCC command packed as the 24 low order bits in an unsigned int.   The entirety of the TMCC command set can be represented as a table/list/dictionary.

When the TMCC protocol was published in the '90s, I wrote some code to decode and display the commands received from a serial port.  An initial interpretation of the protocol resulted in messy if-else code.  The Legacy protocol made things worse.  I've transition to a table-driven approach.  Each command is an entry in the table where the entry ID (key, index, whatever) is the TMCC command with the variable bits (address and data) masked out.  The exception is set absolute speed, and a few other commands, where a data value is needed.  An entry for each possible data value (shown in the Set Relative Speed commands) enlarges the table.  Doing the same for Set Absolute Speed Step would enlarge the table considerably as TMCC1 has 32 speed steps and TMCC2 has 200 steps (with today's plentiful memory, that may no longer be a barrier).

Decoding a three byte TMCC command is simple.  Pack the three bytes as a 24-bit value in an unsigned int (32 bits or larger).  Use the uint as a key to lookup the TMCC table entry by first masking out the address bits.  If there are no variable data bits, the table entry is found on the first lookup.  If command has data bits, then the table entry can't be found so my code has retry logic (using a variable data mask) to lookup variable data commands.  Once the table entry is found, then the command can be decoded into human readable text.

The construction of a TMCC command is simple once the table entry is selected.  The TMCC command is a unsigned int (32 bits or larger).

          //
          // build the 24-bit command, given the protocol, address, optional data value, and a reference to the TMCC command table
          //

          uint dataMask = CreateBitMask(tableEntry.maxDataValue);  // bitmask for the largest variable data value
          uint dataBits = ((uint)dataValue & dataMask);

          uint protocolTypeBits     = (uint)protocol << 16; //16252928=F8  16449536=FB  16646144=FE
          uint addressBits              = ((uint)tmccAddr << ((protocol == TMCCProtocolType.TMCC1) ? 7 : 9));
          uint tmccCommandBits = ((uint)tableEntry.ID & 0xFFFF);

          uint tmccCommand = protocolTypeBits | addressBits | tmccCommandBits | dataBits;

The TMCC command table:

          // TMCC1 commands with 0xFE prefixed:
          //   ID                TYPE                    CATEGORY          MaxDataValue   Text
          { 0xFE0000, TMCCType.Engine, TMCCCategory.Direction, 0,   "Forward Direction"},
          { 0xFE0001, TMCCType.Engine, TMCCCategory.Direction, 0,   "Toggle Direction"},
          { 0xFE0003, TMCCType.Engine, TMCCCategory.Direction, 0,   "Reverse Direction"},
          .........
          { 0xFE0040, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed -5"},
          { 0xFE0041, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed -4"},
          { 0xFE0042, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed -3"},
          { 0xFE0043, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed -2"},
          { 0xFE0044, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed -1"},
          { 0xFE0045, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed  0 (no change)"},
          { 0xFE0046, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed +1"},
          { 0xFE0047, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed +2"},
          { 0xFE0048, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed +3"},
          { 0xFE0049, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed +4"},
          { 0xFE004A, TMCCType.Engine, TMCCCategory.Speed, 0,   "Set Relative Speed +5"},
          { 0xFE0060, TMCCType.Engine, TMCCCategory.Speed, 31,   "Set Absolute Speed Step"},          //Max data value = 31
          { 0xFE4000, TMCCType.Switch, TMCCCategory.Direction, 0,   "Throw THROUGH"},
          { 0xFE401F, TMCCType.Switch, TMCCCategory.Direction, 0,   "Throw OUT"},
          { 0xFE402B, TMCCType.Switch, TMCCCategory.Settings, 0,   "Set Address"},
          { 0xFE4040, TMCCType.Switch, TMCCCategory.Settings, 9,   "Assign to Route THROUGH"},          //Max data value = 9
          { 0xFE4060, TMCCType.Switch, TMCCCategory.Settings, 9,   "Assign to Route OUT"},          //Max data value = 9
          ..................
          { 0xFEFFFF, TMCCType.System, TMCCCategory.Other, 0,   "System Halt"}

The attached C# code snippets have the full TMCC1 table.  It may port to other coding environments without too much effort.

@Tracker John - Going to have to take a deep look into it, JS is probably one of the worst languages to have to deal with bits, especially bit operators. Hopefully I can figure out the right way to build a JS-native bit operation for the data field.

I read where "JavaScript stores numbers as 64 bit floating point numbers.   Before the bitwise operation, JavaScript converts numbers to 32 bits signed integers.  After the bitwise operation the result is converted back to 64 bit JavaScript numbers."

There's a discussion in stackoverflow on how to use the unsigned right shift (>>>) (zero-fill right shift) operator to overcome the side effects.

Since the TMCC commands are 24 bits (high order bits are zero), the JS signed integer used for bitwise operations should remain positive.  YMMV.  Test thoroughly.

@Tracker John posted:

I read where "JavaScript stores numbers as 64 bit floating point numbers.   Before the bitwise operation, JavaScript converts numbers to 32 bits signed integers.  After the bitwise operation the result is converted back to 64 bit JavaScript numbers."

There's a discussion in stackoverflow on how to use the unsigned right shift (>>>) (zero-fill right shift) operator to overcome the side effects.

Since the TMCC commands are 24 bits (high order bits are zero), the JS signed integer used for bitwise operations should remain positive.  YMMV.  Test thoroughly.

... this is exactly what I needed! I just didn't know the write terms to google-fu this to completion. so >>> makes it unsigned. Just tested and it worked exactly as expected! Now just need to see if I can get the data fields to work (I identified a problem with my octal for the command field)

Update: With the correct commandfield octal, the engine doesn't blast off anymore. Working on dialing in the datafield as it seems to jump speed steps pretty heavily, need to inspect the speed step binary

Success! Javascript types byte me in the butt (heh pun intended). Turns out it was interpreting the binary value as an integer, so I converted it to hexadecimal before operating it into the command2 field. Engine movement using abs speed is now working!

@Tracker John I see why you prefer hex over binary now. That took me far longer than I want to admit to figure out!

Two final questions before I wrap up this thread in preparation to open the new control app thread.

- I see an interesting thing that the Legacy abs speed API specifies up to 200 speed steps, does anyone know what happens if you set an abs speed higher than 127? Does the base/loco cap itself to 127, or the loco doesn't "know" what 128-199 is so it ignores them?

- Does ERR 100 speed steps fall under TMCC1 or TMCC2 commands? If TMCC 1 - I'm guessing 0-99 speed steps passed in for abs speed?