Python code to issue TMCC, Legacy, and PDI Commands

Sounds like an exciting project Dave!

One thing that I've asked Lionel for and suggested previously on this Forum, is some provision for RANDOMNESS.  As far as I know, the built-in macro recording capability of the Legacy base, and even Lionel's LCS lack any such provision, and that's a shame!

To my way of thinking: if you build a layout with multiple routes, passing sidings or hidden staging that has the ability to store and stage trains, adding random selection of trains and routes turns the whole thing into a kind of "mechanical fish tank" that visitors could watch for several minutes without seeing the same train on the same track twice.  Code for semi-autonomous operation might make train layouts a fixture in doctors' offices, hotel lobbies, even store windows!  When you take into account the ongoing costs of maintenance, a train display would probably cost less than an ACTUAL fish tank!

One thing I am picky about is realistic starting and stopping.  Smooth starts and stops are also much easier on rubber tires and gears.  So the code should contain provisions for gradual stops and starts, perhaps by sending multiple speed commands.  I'm not sure how you're sensing train position, but to make it reliable and repeatable, care has to be taken to make sure the train stops within a staging block, not fouling other routes / trains.

Can't wait to see what comes of your efforts!  Please keep us posted (and a video demo would be great!)  Thanks for sharing!

I’ve made some progress over the weekend and now have Python code working to control switches, operate accessories, fire routes and run engines and trains. The code supports both TMCC1 and TMCC2. (Legacy) syntax, and provides a means to specify the command format you want from the command line.

The code can be found on GitHub here:https://github.com/cdswindell/PyLegacy.

TMCC2 provides far more options for engine/train control than does the older TMCC1 syntax. I have about 50% of the TMCC1 engine commands implemented, along with their corresponding TMCC2 counterparts. Of course, this means I’ve only implemented about 10-15% of the TMCC2 engine commands. However, as in TMCC2 land, engine and train commands are the same, except for the first byte of the command, this means I also have a bunch of train commands working!

The CLI layer calls into a set of command generation classes that construct the byte sequences to send to the SER2. This layer is relatively clean and uses a set of Python enums that use the bit patterns associated with each Lionel command to uniquely define them. The address of the entity being controlled can then be logically ORed to this bit string to build the command bytes. Frankly, the hard part is working out a sensible CLI command syntax for the TMCC2 engine/train commands. The Python library ArgParser is definitely my friend!

After I complete the implementation of the TMCC1 and 3-byte TMCC2 engine commands, I will turn to writing unit tests and packaging the code. Then, I’ll move on to connecting physical button presses on my Raspberry Pi to my command architecture.

  — Dave

You're really cruising along Dave, sounds like a great project!

Dave....Thanks for sharing. Following

Richard

It was gratifying to see the code up and working on my little Pi this morning. There were some surprises:

• The 'Ring Bell' commands (both TMCC1 and 2) actually turn the bell ringing on and off (like sliding and holding the bell slider on the Cab 2 remote)
• Unclear why the "Bell Slider Command [2 - 5]" didn't seem to do anything
• For the "Bell One-shot Ding [0 - 3]" command, the numeric argument controls the bells volume
• The TMCC1 engine commands work on Legacy (tmcc2) engines and carry out the corresponding functionality.
• Not sure what "Blow Horn 2" does; none of my engines responded to it. Is this reserved for the Vision Line series?
• Seems to be a slight difference between the TMCC1 and TMCC2 Halt commands. Both stop all engines dead on their tracks (pun intended), but the TMCC1 command shuts down enabled power districts on my LCS BP2 whereas the TMCC2 command left the power districts on.

I still have several commands left to implement, but as I said in my last message, I want to get some unit tests in place before I go too much further. Everything is checked in, and the README.md file contains instructions on how to grab my code and get it to compile. Although that said, if you are not familiar with Python, it will be a challenge...

  -- Dave

FYI, I've implemented most all of the TMCC1 and TMCC2 commands on pages 4, 5, 6, 8, and 9 of the LCS Partner Documentation Legacy Command Protocol, rev 1.21. The commands I've omitted are those that add/remove engines and rail sounds cars from consists/trains.

The Python code is checked in at github.com/cdswindell/PyLegacy. There are a series of command line tools (CLI) you can use to control an engine from a shell window of a Linux/Mac/Windows PC, as well as a series of classes to issue commands programatically. With respect to the cli commands, the engine command was the most challenging, because there are so many options!! Typing the command:

(legacyEnv) davids@Sprucewood-iMac PyLegacy (master) % src/cli/engine.py -h         

usage: Control specified engine/train (1 - 99) [-h] [-a | -s | -fwd | -rev | -t | -fc | -rc | -r | -b |
                                               -boost | -bl [0 - 7] | -brake | -kl [0 - 7] |
                                               -tb [0 - 7] | -sui | -sud | -sdi | -sdd] [-tr]
                                      [-tmcc | -legacy]
                                      [-baud BAUDRATE] [-p PORT]
                                      Engine/Train {speed,sp,bell,be,horn,ho,momentum,mo,sound,so}

positional arguments:
  Engine/Train          Engine/Train to control
  {speed,sp,bell,be,horn,ho,momentum,mo,sound,so}

  Engine/train sub-commands
    speed (sp)                    Speed of engine/train
    bell (be)                       Bell operations
    horn (ho)                     Horn operations
    momentum (mo)       Momentum operations
    sound (so)                   Sound operations

options:

-h, --help            show this help message and exit

-a, --set_address     Set engine address

-s, --stop_immediate Stop immediate

-fwd, --forward_direction

                        Set forward direction

-rev, --reverse_direction

                        Set reverse direction

-t, --toggle_direction

                        Toggle direction

-fc, --front_coupler Open front coupler

-rc, --rear_coupler Open rear coupler

-r, --ring_bell       Ring bell

-b, --blow_horn       Blow horn

-boost, --boost_speed

                        Brake speed

-bl [0 - 7], --boost_level [0 - 7]

                        Boost level

-brake, --brake_speed

                        Boost speed

-kl [0 - 7], --brake_level [0 - 7]

                        Brake level

-tb [0 - 7], --train_brake [0 - 7]

                        Train brake

-sui, --start_up_immediate

                        Start up immediate

-sud, --start_up_delayed

                        Start up delayed (prime mover)

-sdi, --shutdown_immediate

                        Shutdown Immediate

-sdd, --shutdown_delayed

                        Shutdown delayed with announcement

-tr, --train          Direct command to addressed train rather than engine (for TMCC2 commands)

-tmcc, --tmcc1        Use TMCC1 command syntax.

-legacy, --tmcc2      Use TMCC2/Legacy command syntax.

-baud BAUDRATE, --baudrate BAUDRATE

                        Baud Rate (9600)

-p PORT, --port PORT Serial Port (/dev/ttyUSB0)

provides a list of command options that can be applied to the specified option. For example the command:

src/cli/engine.py 71 speed 65

sets the absolute speed of the Legacy engine with the Engine ID of 71 to 65.

The command:

src/cli/engine.py 71 -r

turns the bell on (or off) on Engine #71.

I've added tests for the TMCC1 commands. I need to think a bit about how to test the TMCC2 commands, as it is tedious to code the bit patterns from the Lionel doc into source code, but that is most likely what I need to do...

Next up is to implement a serial communications module that allows a single instance to take commands from multiple threads and meter them out to the serial input of the LCS SER2 (so its buffer doesn't overflow).

To follow are implementing the multi-word Legacy commands, the Raspberry Pi code to respond to button presses, complete unit tests, better documentation, and packaging the project into a downloadable project on PyPi.

    -- Dave

That's really cool. Could be the beginning to a lot of possibilities

This is certainly an ambitious project Dave, following along closely.

Just checked in the code for Effects and Lighting controls. No CLI for these yet. Any of you guys have the Lionel stock cars that have effects? Who/what is Fred?

Today's checkins include support for a comms buffer that sends command bytes to the LCS SER2 with buffering and throttling. The buffer can hold up to 2,048 commands (3 or 9 byte packets) and then sends them to the SER2 no faster than one every 50ms. This behavior is transparent to the client command code, meaning programmers just have to think about what they want their trains to do.

I also added support for the Lionel's "official railroad speeds". For example:

davids@Sprucewood-iMac PyLegacy (master *) % src/cli/engine.py 10 sp limited

sets the speed of the Legacy/TMCC2 engine with TMCC ID#10 to "Limited", which corresponds to[ponds to the absolute speed step of 118. Appropriate speed tables have been implemented for TMCC1 as well (where only speed steps of 0 - 31 are supported)

I've also added a repetition option, which sends the same command multiple times. For example:

davids@Sprucewood-iMac PyLegacy (master *) % src/cli/engine.py 12 -re 10 -boost

sends the "boost" command 12 times to Engine # 10. Why you may ask? Because this is essentially what happens when you press/roll up repeatedly on the multi-controller wheel on the Cab 2. Sending the command multiple times corresponds to holding the "boost" function wheel for a period of time. You could also use the -(re)peat option to ring the bell, say, 5 times or blow the whistle 3 times, etc.

I have been a bit surprised to find that the "Train Brake" functionality actually sends along absolute speed commands as you move the slider control on a Cab 2. In add-on to sending the bytes for the train brake command, the Cab 2 sends multiple decreasing absolute speed commands along as well. It appears that it is these speed commands that actually slow the train down. When you push the slider back up (no brake), a series of increasing speed commands are sent as well. Huh.

Tests are coming along as well. Tests have been added for all of the TMCC1 and TMCC2 3 byte commands. I've done this by entering them in my constants file as hex digits. The test classes use binary representation ("1"'s and "0"'s) taken directly from the documentation and compare values. Not perfect, but it means I had to make the same transcription mistake twice in two different formats.

Here is a summary of the CLI commands that exist today:

• src/cli/acc.py: issue accessory commands
• src/cli/engine.py: issue engine/train commands
• src/cli/halt.py: issue an emergency stop/halt command
• src/cli/route.py: fire routes
• src/cli/switch.py: throw switches

The engine, halt, and route commands can issue either TMCC1 or TMCC2 command sequences. Python classes have been developed (and are in /src/protocol) that are used by the CLI and can be incorporated into other python programs to build up more elaborate command flows (start up a train, ramp up speed over time, ring bells and blow horns, fire routes, etc.)

I will next add the remaining sound effects controls that are defined for the TMCC2 multi-word commands and then on to physical buttons.

  -- Dave

I had a really productive day today with the project. Using the gpiozero library by Ben Nuttall and Dave Jones, I connected the command generation and sending code I've developed to physical button presses on a Raspberry Pi (4)! For example, by writing this one line of code:

GpioHandler.when_button_pressed(21, TMCC2RouteOption.ROUTE, 10)

I can fire Route 10 (using  TMCC2/Legacy command syntax) when I press a button connected to my Pi's GPIO pin 21!

The gpiozero library really made the integration easy. For example, with these two lines:

from gpiozero import Button

button = Button(21)

you create a "Button" object tied to GPIO pin 21.

Then, with this line:

button.when_pressed = CommandReq.as_action(TMCC2RouteOption.ROUTE, 10)

you assign a callback function to call whenever the button is pressed.

Here's a picture of the setup:

The red wire is attached to one of the Pi's Gnd (ground) pins, and the brown wire is attached to GPIO pin 21. You push the big red button and the byte string 0xFA14FD is sent to my LCS SER2 and route #10 is fired!!

  -- Dave

It is exciting to see your progress.  Thanks for making the project available on github.

For me, I'd love to implement this on a microcontroller running circuit python.  Think of the possibilities if it could also receive and interpret commands on the serial bus.  Very nice to have this as an alternative to the whole LCS partner shebang.

I don't know how many Python fans there are in our community, but it is definitely a +1 from me.

I've been busy the last few days on doing a fairly major restructuring of the code. The end result is that all of the logic to apply address and data values to the TMCC1, TMCC2, and a good portion of the TMCC2 Parameter commands. I've also unit tested all of this code (which isn't as fun as building new features, but, alas, it is critical to producing production-quality software).

The CLI and the classes that back end it (and implement the command syntax to make things happen on a layout) all now use this new code and benefit from the unit testing of the underlying subsystem. These classes are CommandReq and ParameterCommandReq. They can be used directly, are used by the CLI, and support several class methods to issue directives to a layout (via the LCS SER2) without creating intermediate CommandReq objects.

I've also implemented a command buffer which receives byte strings from the CommandReq fire or send methods and queues them for dispatch to the LCS SER2. Per other posts I've read on this forum, output to the SER2 is throttled so no more than one byte sequence is sent every 40 milliseconds (configurable, of course).

For others to be able to use this code, I really need to write more documentation and to use a python package manager to bundle everything up. Doing the packaging is a little new to me, but the gpiozero package I'm using to connect my work to physical button presses gives me some ideas on how to do that. The packaging will probably be my next step.

I've also signed up for the Lionel Partner Program and am looking forward to participating there as well.

  -- Dave

p.s. I do, on occasion run trains too. Below is a picture of the layout I'm building.

Just checked in code to issue Legacy/TMCC2 Lighting Effects commands, as documented on page 18 of the Lionel Partner Program documentation. These are part of the 3-word commands, and use 9 bytes to express a command. The Lighting control effects include support for:

• Dog house lights (on/off)
• Hazard lights (on/off/auto)
• Ditch lights (on/on pulse off with horn/off pulse on with horn/off)
• Loco/engine marker lights (on/off)
• Tender marker lights (on/off)
• Ground lights (on/off/auto)
• Strobe lights (on/double flash/off
• Mars lights (on/off)
• Cab lights (on/off/auto)
• Rule 17 (on/off/auto)
• Car cabin lights (on/off/auto)

Support includes the cli script lighting.py, and a command class.

I also built a cli that reads requests from the command line and, using the parsers developed for each of the cli commands, parses the input and sends the appropriate command to the LCS SER2. The program continues to process your input until you enter "q" or hit <ctrl-c>.

Here is an example:

(legacyEnv) davids@Sprucewood-iMac PyLegacy (master) % src/cli/control.py 
PyLegacy train controller, Ver 0.1
>> eng 88 speed 50            # sets the absolute speed of engine 88 to 50
>> li 88 ditch -on            # turns ditch lights on
>> eng 88 -b -re 4 -de 0.5    # blow horn 4 times w/delay of 0.5 seconds between

You can also get help on command options like so:

>> eng 1 sp -h
usage: control.py Engine/Train speed [-h] [-a | -r]
      Engine/Train speed: 0 - 199 (Legacy) or 0 - 31 (TMCC) or roll, restricted,
      slow, medium, limited, normal, or highball

positional arguments:
      Engine/Train speed: 0 - 199 (Legacy) or 0 - 31 (TMCC) or roll, restricted,
      slow, medium, limited, normal, or highball

options:
    -h, --help          show this help message and exit
    -a, --absolute      Set absolute speed
    -r, --relative      Set relative speed speed (-5 to 5)
>>

The Effects controls are next up (smoke on/off, stock car options, pantograph control, etc).

Enjoy!

  -- Dave

I'm liking this project, you're making real progress on filling out the command capability!

So what do you do when your control panel needs more than the 26 GPIO pins a standard Pi makes available? You use the train control program control.py in client-server mode, of course! I just checked in changes that allow multiple Pis to slave off of a server Pi that has the connection to the LCS SER2.

Examples are coming, but this development will allow stand-alone control panels to be built, powered by an inexpensive RaspberryPi with WiF. All wiring, except for power to the Pi, can be managed inside the panel enclosure.

  -- Dave

Here is an example screen capture of a client control program sending commands to a server. Again, the server has the serial connection to an LCS SER2. The server is in the top window and the client is in the bottom window:

In the top window, the control program is launched in the normal way:

(legacyEnv) davids@Sprucewood-iMac PyLegacy (master) % src/cli/control.py

In the bottom window, the control program is launched with the special argument -server, which tells it to use client/server mode and send all commands to the program on the server 'localhost':

% src/cli/control.py -server localhost

In practice, you would specify the IP address of your server on your local network, like so:

% src/cli/control.py -server 192.168.3.35

Here, the IP Address 192.168.3.35 is the address of my development machine, an older iMac (from 2021).

This is all checked in, although I still haven't done any packaging nor distributed the code on PyPi. The GitHub sources are here: https://github.com/cdswindell/PyLegacy.

  -- Dave

Dave, this is awesome.  Really important stuff.

Have you built a CLI mode that listens for commands from BASE2 / BASE3 and displays them in the window -- maybe interprets them too (both raw code and human-readable info)?  This would effectively allow anyone to reverse-engineer the different commands sent by the Legacy Bases when operated by any remote, or sent by your software.

Another potential use I see -- mapping special keypresses or sequences on older less-functional remotes to enable users to access Legacy functionality typically requiring CAB2.  Essentially, someone with a CAB1L could fire off AUX3 on a Legacy-equipped steamer by tapping AUX1 twice within three seconds, for example.  You could even use it to make your own hardware remote, or a software emulator of a CAB2.  Etc.

Simply great work.  Keep it up.

@BlueFeather posted:

Dave, this is awesome.  Really important stuff.

Have you built a CLI mode that listens for commands from BASE2 / BASE3 and displays them in the window -- maybe interprets them too (both raw code and human-readable info)?  This would effectively allow anyone to reverse-engineer the different commands sent by the Legacy Bases when operated by any remote, or sent by your software.

Another potential use I see -- mapping special keypresses or sequences on older less-functional remotes to enable users to access Legacy functionality typically requiring CAB2.  Essentially, someone with a CAB1L could fire off AUX3 on a Legacy-equipped steamer by tapping AUX1 twice within three seconds, for example.  You could even use it to make your own hardware remote, or a software emulator of a CAB2.  Etc.

Simply great work.  Keep it up.

Thanks!! I haven’t taken on reading serial input yet. I just got the documentation from the Lionel partners program yesterday. I definitely want to be able to read some serial commands, as I wanna be able to set the state of things when my programs power on, and the only way to do that is to read state  via the Ser2.

I hadn’t thought about key emulation, but you could very easily do that by wiring your own buttons to a raspberry pi!!

  — Dave

@cdswindell posted:

So what do you do when your control panel needs more than the 26 GPIO pins a standard Pi makes available? You use the train control program control.py in client-server mode, of course! I just checked in changes that allow multiple Pis to slave off of a server Pi that has the connection to the LCS SER2.

Examples are coming, but this development will allow stand-alone control panels to be built, powered by an inexpensive RaspberryPi with WiF. All wiring, except for power to the Pi, can be managed inside the panel enclosure.

  -- Dave

Sounds like a good place for a canned product.

With tonight's checkin, you can use a potentiometer on a Pi to control the speed of an engine/train. As the Raspberry Pi is a digital device, and by their nature, potentiometers (rheostats) are analog devices, you need to purchase a some additional hardware to make this all work. Specifically, you need a MCP3008 analog to digital chip (available at Amazon at 2 for $14) to convert the analog output of the pot to a digital value the Pi can work with. The chip allows up to 8 pots to be connected to it at a time, BTW.

Below is a picture of my Pi and a development breadboard, which makes prototyping much simpler. The interesting bits are the little black chip (circled in red below), and the pot (the thing to the lower left of the black chip). The chip is the MCP3008. It takes the analog value from the pot (a voltage from 0 to 3.3), and maps it into a float value between 0.0 and 1.0. This value is then scaled from 0 to 199 (for TMCC2/Legacy engines) or 0 to 31 (for TMCC1 engines).

Here's the Python code to make this work (controls the speed of a TMCC1 engine with the ID of #1:

GpioHandler.when_pot(TMCC1EngineCommandDef.ABSOLUTE_SPEED, 1)

Of course, you can also use this command in client-server mode:

GpioHandler.when_pot(TMCC1EngineCommandDef.ABSOLUTE_SPEED, 1, server='192.168.3.35')

This code sends the speed changing commands to a TrainController server running on 192.168.3.35.

  -- Dave

One more checkin. It is becoming clear to me that the "train control" program I developed (src/cli/control.py) is a useful platform to integrate the code to connect Raspberry Pi GPIO pins to physical buttons and devices. Thus, I added an option to the script to execute commands from another Python script at startup. By default, the program looks for a file named buttons.py in the local directory. You can specify a different script via command line argument.

For example, here is the contents of my local buttons.py:

from src.gpio.gpio_handler import GpioHandler
from src.protocol.command_req import CommandReq
from src.protocol.tmcc1.tmcc1_constants import TMCC1AuxCommandDef
from src.protocol.tmcc2.tmcc2_constants import TMCC2RouteCommandDef
from src.protocol.tmcc2.tmcc2_constants import TMCC2EngineCommandDef

"""
    Simple examples of how to associate Lionel commands to Raspberry Pi buttons
"""
GpioHandler.when_button_held(26, TMCC2EngineCommandDef.BLOW_HORN_ONE)

GpioHandler.when_button_pressed(21, TMCC2RouteCommandDef.ROUTE, 10)

off = CommandReq(TMCC1AuxCommandDef.AUX2_OPTION_ONE, 9)
on = CommandReq(TMCC1AuxCommandDef.AUX1_OPTION_ONE, 9)
GpioHandler.when_toggle_switch(13, 19, off, on, led_pin=20)
# GpioHandler.when_toggle_button_pressed(19,  on, led_pin=20)

GpioHandler.when_pot(TMCC2EngineCommandDef.ABSOLUTE_SPEED, 88)

print("Buttons registered...")

This file does the following:

• broadcasts the BLOW_HORN_ONE command to all engines when the button connected to pin 26 is held (sends the command to the default address of 99).
• sends the Fire Route 10 command whenever the button connected to pin 10 is pressed.
• defines a toggle switch that  simulates the Cab 2's Aux1 and Aux2 buttons on TMCC accessory #9; on my layout, this is an LCS BP2, so track power is turned on when the switch is turned on, and power is turned off when the switch is pressed off. A LED connected to pin 20 is lit when track power is on.
• controls the absolute speed of Engine #88 via a pot.
  
  

Here's what it looks like when I run control.py on my pi:

(legacyEnv) davids@raspberrypi:~/dev/legacyEnv/PyLegacy $ src/cli/control.py
Sending commands directly to LCS Ser2...
Listening for client connections on port 5110...
Loading startup script: buttons.py...
Buttons registered...
PyLegacy train controller, Ver 0.1
>>

On a separate note, what should I call this program? Train Control? I think that is already taken, and it's pretty boring. PyLegacy? Lionel may not be too happy with that. PiTrains? Hmm, potential...

  -- Dave

PyCC, or PyCommand?

Anyway, stay on it!  I can see a path forward for hardware hackers to build their own hardware controls of all sorts, and even for DIY "kits" for a future remote.

@BlueFeather posted:

PyCC, or PyCommand?

Anyway, stay on it!  I can see a path forward for hardware hackers to build their own hardware controls of all sorts, and even for DIY "kits" for a future remote.

I'm beginning to feel PyTrain!

I've added extensive online help to PyTrain. Here are some examples:

(legacyEnv) davids@Sprucewood-iMac PyLegacy (master) % src/cli/pytrain.py
Sending commands directly to Lionel LCS Ser2...
Listening for client connections on port 5110...
PyTrain, Ver 0.1
>> ?
usage: [h] [accessory | effects | engine | halt | lighting | route | switch | quit]

Valid commands:

options:
  h, help    show this help message and exit
  accessory  Issue accessory commands
  effects    Issue engine or train effects commands
  engine    Issue engine or train commands
  halt      Emergency stop
  lighting  Issue engine or train lighting effects commands
  route      Fire defined routes
  switch    Throw switches
  quit      Quit PyTrain

Commands can be abbreviated, so long as they are unique; e.g., 'en', or 'eng' are the same as typing 'engine'.
Help on a specific command is also available by typing the command name (or abbreviation), followed by '-h',
e.g., 'sw -h'
>> św -h
usage: Fire specified switch (1 - 99) [-h] [-t|-o|-a] Switch Number

positional arguments:
  Switch Number      switch to throw

options:
  -h, --help         shows this help message
  -t, --through      Throw Through
  -o, --out          Throw Out
  -a, --set_address  Set switch address
>>

  -- Dave

Checked in support for RailSounds Effects commands, excluding subway stops (Parameter index 0x4. These are part of the Lionel Multi-word commands.

  -- Dave

With the addition of the physical buttons, how much of a leap would it be to make a dedicated controller with physical buttons that communicates with the Pi wirelessly?

What I'm thinking about is using an Elgato Stream Deck (essentially a USB device with programmable buttons that integrates with the rPi) to set up "scenes" around the layout, with the buttons mapped to specific functions.

@J.Dooley posted:

What I'm thinking about is using an Elgato Stream Deck (essentially a USB device with programmable buttons that integrates with the rPi) to set up "scenes" around the layout, with the buttons mapped to specific functions.

Does it let you bind Python code to buttons? If so, my stuff should work with it. I will have to check it out.

  -- Dave

@cdswindell have you considered interfacing with the LCS system through wifi?  Most raspberry pis have wifi these days and it eliminates the need for the SER2 and a second pi.  Perhaps a second interface option on the back end would be a nice extension.  From the LCS partner documentation, it seems like wifi is the preferred method anyway.

I'm very encouraged by your rapid progress here.  A DIY hardware remote that is powered by a raspberry pi (or other wifi enabled python interpreter) now seems well within reach.  It could be something as simple as a Cab1L with an aux3 and a whistle slider, or even more complicated like a Cab2 with the context sensitive touch screen.  A raspberry pi could easily power a touch screen gui.

@Patrick Keistler posted:

@cdswindell have you considered interfacing with the LCS system through wifi?  Most raspberry pis have wifi these days and it eliminates the need for the SER2 and a second pi.  Perhaps a second interface option on the back end would be a nice extension.  From the LCS partner documentation, it seems like wifi is the preferred method anyway.

I'm very encouraged by your rapid progress here.  A DIY hardware remote that is powered by a raspberry pi (or other wifi enabled python interpreter) now seems well within reach.  It could be something as simple as a Cab1L with an aux3 and a whistle slider, or even more complicated like a Cab2 with the context sensitive touch screen.  A raspberry pi could easily power a touch screen gui.

@crazytrain asked the same question. I just gained access to the partner program documentation on Monday. It contains info on the LCS WiFi module. Given that the older LCS app communicates just fine with a Base 3, even though it was designed with the WiFi module in mind, I will have a look at the documentation and see what is involved with making a direct WiFi connection.

  — Dave

@Crazy Train posted:

With the addition of the physical buttons, how much of a leap would it be to make a dedicated controller with physical buttons that communicates with the Pi wirelessly?

With a Pi wirelessly or with the Base 3 wirelessly? The system I built already supports allowing a client Pi to talk to the server Pi (the one connected to the Ser2) wirelessly.

i have to see if I can figure out how to send packets directly to a Base 3 over WiFi…

  — Dave

Just checked in about 25% of the Rail Sounds Dialog controls. This one was fun, as I can now trigger engineer to tower and tower to engineer dialogs. Combined with the speed commands, this means one can now trigger "ok to change speed" "yes, change speed", change the speed, and then ack back to the tower the speed change!

  -- Dave

Just checked in support for "Sequenced Requests" These allow you to send one or more commands with delays in-between. For example, the subclass SpeedReq takes an engine address and a desired speed then sends, in order:

1. The Tower dialog to request the speed change
2. Changes the speed
3. The Engineer acknowledgement that the speed was changed as requested.

This sequence works particularly well when selecting one of the "official rail road speeds" (slow, restricted, limited, etc.). Sequenced commands can be bound to buttons, just like regular commands.

There are other sequenced requests I could think up too, such as an engineer requesting clearance to move out, the tower denying it, wait 30 seconds, have the engineer request clearance again, have the tower grant clearance, and roll out!

I will get back to coding up the remaining RailSounds dialogs...

  -- Dave

I've started reading data from the Lionel Ser2. I can now interpret all of the TMCC1 commands and have unit tests in place verifying that I can construct a command, send it, and recover the exact same command (and address/data) from the transmitted bits. Decoding the 3 byte TMCC2 commands should be straightforward.

Interestingly, the SER2 strips away any of the PDI prefix and suffix bytes. I don't know if this means I can only read TMCC/legacy commands from the SER2 (and not read commands coming from the other LCS devices). I have to do some more experiments.

I just ordered a Raspberry Pi Zero with WiFi to test my code on. These little boards are only $15 and support WiFi. If they have the memory and horsepower to run my stuff, they would be the perfect chip to put into a control panel.

  -- Dave

Just checked in support for interpreting the 3 byte TMCC2/Legacy commands. I have to give some thought to what I want to do here. Writing code to echo the command stream the Ser2 is easy enough. But I am wondering what else we could do with this data?

One thought is to maintain the sensed state of engines, switches, accessories, etc. This would allow control panels to reflect current state (is a switch thrown or through?). But another would be to assist in sending sequenced commands. For example, if the tower sends the "go to limited speed" command (triggers a dialog), the engine should decelerate if it's speed is above this speed step, and accelerate if below it. Interestingly, this is what the Cab2 throttle does when you press one of the "official railroad speed" buttons. The controller doesn't just set an absolute speed, it sends multiple commands to ramp up to/down to this speed. By maintaining the sensed engine speed state, I could mimic this behavior.

It appears Momentum is also handled by the Cab2. For higher momentum settings, the Cab2 accelerates an engine slower by sending more absolute speed commands with smaller deltas.

I'm sure there are other applications as well.

I'm still not sure if I can query other LCS devices via the Ser2. This would be more reliable for things like switch, accessory, and power district state, and would be the preferred way of obtaining state, of course.

  -- Dave

@Dennis-LaRock posted:

Dave,

Great Work!  You are nudging me perilously close to picking up a pi.  There are a couple of M2s around here somewhere... so, may as well Chapeau it.

Thanks!

I just ordered a $20 Pi zero with wifi to see if it is up to the task. It probably will be, especially if used in client (slave) mode.

@BlueFeather posted:

Dave, this is awesome.  Really important stuff.

Have you built a CLI mode that listens for commands from BASE2 / BASE3 and displays them in the window -- maybe interprets them too (both raw code and human-readable info)?  This would effectively allow anyone to reverse-engineer the different commands sent by the Legacy Bases when operated by any remote, or sent by your software.

Another potential use I see -- mapping special keypresses or sequences on older less-functional remotes to enable users to access Legacy functionality typically requiring CAB2.  Essentially, someone with a CAB1L could fire off AUX3 on a Legacy-equipped steamer by tapping AUX1 twice within three seconds, for example.  You could even use it to make your own hardware remote, or a software emulator of a CAB2.  Etc.

Simply great work.  Keep it up.

I’ve started work on reading the byte stream from the Ser2. Probably won’t spin this out into its own CLI until next week. I need to learn a bit more on how to code efficient parsers that are just prefix-defined (prefix and length), which the TMCC1 & 2 commands are.

cds

I have command echoing working via the LCS Ser2! Here's the output from a sequence command where the tower requests a full stop, the engine speed is set to zero, and the engineer acknowledges:

[TOWER_SPEED_RESTRICTED ID# 65 Data: 0]
[ABSOLUTE_SPEED ID# 65 Data: 24]
[ENGINEER_SPEED_RESTRICTED ID# 65 Data: 0]

I will add the display of the byte sequence to this output as well.

@Patrick Keistler, @BlueFeather fyi

  -- Dave

Fyi, use src/cli/echo.py to echo commands. Sadly, commands generated by other LCS modules do not appear to be echoed from the Ser2; only commands that you can generate from the Cab2 or Cab3 app (or my software) are echoed.

  -- Dave

Just checked code to support subscribing to system events, and have refactored the command echoing program echo.py to use it. The mechanism uses a simple subscribe/publish mechanism to allow "listeners" to subscribe to "events" they are interested in. Events can be broad, like any changes to any engine, or specific; changes to a specific engine (or switch or accessory or train). There is also a "broadcast" channel you can subscribe to which receives all TMCC events the system reads from the LCS Ser2. This is how echo.py now receives its updates.

I threw together a quick and dirty command syntax to query device state and added it into pytrain.py. I've only wired in switches and accessories, but will work on engine and train state next. I have to figure out how to serve this information up to client pytrains. Once I do that, I'll be able to change speeds more gradually by querying an engine's current speed and ramping up/down to the new requested speed. I also want to wire listeners in to the GPIO support so indicated light states can be kept in sync.

I've asked the Lionel folks how to communicate directly with the Base 3 over wifi. This will be a far better mechanism to get the current state of switches, accessories, and power districts, but as far as I know, engine and train states are most likely perceived by interpreting the TMCC command stream (over wifi or serial).

  -- Dave

My $15 Raspberry Pi Zero W (2017 vintage) arrived yesterday. The short answer is it works with my software and can be used to control trains! The longer answer is it is v e r y  v e r y s l o.  w.

It takes very little power, with is a plus, but it is pretty slow to boot up. I'll post some pictures of it running my Lionel Strasburg switcher once my LCS Ser2 arrives (I've switched locales to Boston for a few months; I miss my layout in NH already.

  -- Dave

Slow in what way Dave?  If it's just booting that's slow, that's not a big deal.  However, if it's slow handling the commands, that's a different matter.

Dave, that's a $15, seven year old board.  :-)  You can spend a little more and speed things up fairly easily if that's the bottleneck.

@BlueFeather posted:

Dave, that's a $15, seven year old board.  :-)  You can spend a little more and speed things up fairly easily if that's the bottleneck.

LOL, no,  just wanted to see if it works. IMHO, it boots too slow to be useable in a control panel. You want the panels to be up and operational as soon as the Base 3 is booted.

Because of its small size, the Pi Zero would have been a good choice to build right into a panel. I have a Pi Zero 2 on order at the whopping price of $24, so there may still be hope!

but frankly, that's something a little bigger than the size of a stick of chewing gum can be a fully functioning computer with a GUI, TCP/IP, etc., is still pretty amazing!

  -- Dave

@gunrunnerjohn posted:

Slow in what way Dave?  If it's just booting that's slow, that's not a big deal.  However, if it's slow handling the commands, that's a different matter.

It was surprisingly responsive in handling commands and passing through the output of the LCS Ser2. It's about 0.5 - 1.0 seconds slower to handle button presses than a Pi 4B (also old).

  -- Dave

Well, it looks like you're on the trail of something faster.

@gunrunnerjohn posted:

Well, it looks like you're on the trail of something faster.

As someone that's worked with software all my life, I am still fascinated by the array of hardware we now have available and its capability. Here's some timing, using the elapsed time it takes to run my test suite:

Mac M2 Ultra (2023):          1.36 sec
Raspberry Pi 5 (2024):        1.44 sec
Raspberry Pi 4B (2018):       3.11 sec
Raspberry Pi Zero W 2 (2023): 4.31 sec
Raspberry Pi Zero W (2017):  24.27 sec

It's making me wonder if my big, honking M2 Ultra was worth the price

  -- Dave

My grandson just went to college, and I hooked up a Raspberry Pi 5 to a SSD for him for remote storage.  I was amazed at the size of the package and the capability it offers.

Fairly major update. The code that parses the byte stream from the LCS Ser2 is paying off in spades. I've could echo received commands for a couple of weeks now. This means that no matter how the TMCC command was generated, via a Cab2, the Cab 3, another third-party vendor, or my Python code, I can capture it via the output stream from the LCS Ser2 and echo it to a console.

One of the issues with building control panels is keeping system state in sync. For example, if I have a light on a control panel that reflects if a power district is on or off, I want that light to reflect the correct state no matter how that state is changed. So, if a power district is on, and the corresponding led is lit, it should turn off if:

• I manually turn off the district via a cab2
• I manually turn off the district via a cab3
• It is turned off via a 3rd party app
• A TMCC Halt command is issued
  
  

For this to work correctly, you need:

• A way to maintain state on the PyTrain server
• A way to keep that state in sync on all PyTrain clients (the ones powering the panels)
• An understanding of the dependencies that exist between Lionel commands

This last item is particularly interesting. It's obvious that a power district led should turn off if it sees a "turn power district off" command (Accessory Aux2, in my case), but what about that Halt command? And when you do an engine reset, (the (R) button on the Cab2/Cab3) it sets the engine speed to zero, resets direction to forward, turns the bell off, sets RPM to zero, sets engine labor to 12, and performs an immediate startup, and I may still be missing some; phew!).

With this morning's checkin, the following is available:

• A state management system has been implemented that observes the command stream from the LCS SER2 and determines device states from it,
• Commands received on the server (the one with the LCS Ser2 connection) are now propagated to all clients; they in turn, using the same code, keep their own local state caches in sync,
• I've implemented a command listener system that listens for specific commands (scoped to device type, specific device, device command, and device command data) that calls callback methods in response, and
• I've started developing a dependency mechanism that understands the relationship between commands (e.g., Halt command should disable power districts)

Devices are engines, trains, switches, and accessories. There is still a lot more work to be done, but with these components, the system is now aware of events happening elsewhere on the layout and from other control mechanisms, and can maintain device state accordingly.

My next step is to wire these dependency-aware event listeners into the code to control LEDS so their state always reflects reality.

I am hoping that one day soon I will learn how to query LCS devices via wifi, but for now, you can do a lot with an LCS Ser2.

  -- Dave

@cdswindell posted:

One of the issues with building control panels is keeping system state in sync. For example, if I have a light on a control panel that reflects if a power district is on or off, I want that light to reflect the correct state no matter how that state is changed. So, if a power district is on, and the corresponding led is lit, it should turn off if:

• I manually turn off the district via a cab2
• I manually turn off the district via a cab3
• It is turned off via a 3rd party app
• A TMCC Halt command is issued
  
  

No offense Dave, but the simple way to do this is to monitor the power at the actual power district and control your panel light with that.  No mistakes, and you can't get a false indication unless a bulb burns out.

However, a TMCC halt command, IMO, probably shouldn't turn the light off.  It doesn't remove power from the track, it just stops all current TMCC/Legacy activity.  However, you can immediately spin the throttle and be on the move again, so in that scenario, I feel the power district light should remain on.

@gunrunnerjohn posted:

No offense Dave, but the simple way to do this is to monitor the power at the actual power district and control your panel light with that.  No mistakes, and you can't get a false indication unless a bulb burns out.

However, a TMCC halt command, IMO, probably shouldn't turn the light off.  It doesn't remove power from the track, it just stops all current TMCC/Legacy activity.  However, you can immediately spin the throttle and be on the move again, so in that scenario, I feel the power district light should remain on.

None taken, but with my BP2s configured to accept accessory commands, when the TMCC1 Halt command is given, those districts all turn off. Power isn't restored until you reenable that power district (at least on my layout).

I used power district as an example. You encounter the same situation when you reset an individual engine (Numeric 0). The direction is set back to "FWD", speed is set to 0, the bell is turned off, etc.

  -- Dave

State management is now in place and working well, both on the PyTrain server as well as clients. Engine speed, direction, and startup/shutdown status is tracked, and maintained correctly when changed via a Cab 3 as well as impacting commands (engine reset and halt commands).

My next plan is to use the engine speed state to smoothly ramp up or down speed changes.

Stay tuned.

  — Dave

I'm getting pretty close to putting together my first control panel that uses the PyTrain code. I've added several methods to the GpioHandler class that model Lionel LCS modules and accessories, which makes it easier to build out physical devices and associate pushbuttons and switches to Raspberry Pi pins.

For example, here is the code to support a Command/Control Smoke Fluid Loader:

GpioHandler.smoke_fluid_loader(12,
                               boom_pin_1=12,
                               boom_pin_2=16,
                               dispense_pin=21,
                               lights_on_pin=26,
                               lights_off_pin=19)

This is all the code a user would need to write to operate the Smoke Fluid loader with Accessory ID #12. Boom rotation is done via a rotary encoder that is connected to Pi pins 12 and 16. Smoke fluid droplets are dispensed via a pushbutton connected to pin 21. And you can turn the unit's lights on and off via a toggle switch connected to pins 26 and 19.

Of course, you could use which ever pins you want...

I've defined the following helper methods:

GpioHandler.route

GpioHandler.switch

GpioHandler.power_district      # LCS BP2 in accessory mode

GpioHandler.accessory           # LCS ASC2

GpioHandler.culvert_loader      # Command & Control

GpioHandler.smoke_fluid_loader  # Command & Control

GpioHandler.gantry_crane        # Command & Control

These are all items I have on my layout and that I want to build panels for. Other devices (including the pre-command/control variants of those listed above) are easy enough to piece together.

All of these devices are state-aware, that is, if an operation is started via a control panel, it can be completed via a Cab 2, Cab 3, or other 3rd-party software. Indicator lights on the panels will reflect actual state, regardless of how it got there.

My first panel will control 1 switch, 2 power districts, and a gantry crane. I will send pictures as I make progress. What I am most happy about is that each panel will only need one wire; the power wire for the Raspberry Pi itself. All control operations and state management will be done via the PyTrain software, WiFi, and a single Pi connected to an LCS SER2.

  -- Dave

p.s. The $25 Raspberry Pi Zero W 2 looks like a real winner for this application. It is about the size of a piece of chewing gum and can be powered by a small USB power adapter, like they used to send out with your iPhones. I will build the Pi directly into the panel itself, so there will be short wire runs from the physical switches and status LEDs to the Pi's GPIO pins.

I am learning how to interact with the Base 3 over WiFi. I still don't know enough to say if I can dispense with the LCS Ser2 altogether. One interesting observation is that commands sent and received appear to be the ASCII representations of the byte sequences and not the byte sequences themselves. For example, the "keep alive" sequence one needs to send to the Base to keep the TCP/IP connection open is D1 29 27 DF (thanks to a new friend who sent me that info in a DM). I assumed I would be sending 0xd12927df, but instead, I have to send "D12927DF"!! This was a bit of a surprise and makes me a little worried the mechanism I'm using to connect is a debugging backdoor that may be slammed shut one day...

I've also modified my client code so that it requests a complete update from the server when started. This will allow all clients (deployed in separate control panels) to be up to date when booted up, should the server receive state before clients boot.

Back to it...

  -- Dave

@cdswindell posted:

I am learning how to interact with the Base 3 over WiFi. I still don't know enough to say if I can dispense with the LCS Ser2 altogether. One interesting observation is that commands sent and received appear to be the ASCII representations of the byte sequences and not the byte sequences themselves. For example, the "keep alive" sequence one needs to send to the Base to keep the TCP/IP connection open is D1 29 27 DF (thanks to a new friend who sent me that info in a DM). I assumed I would be sending 0xd12927df, but instead, I have to send "D12927DF"!! This was a bit of a surprise and makes me a little worried the mechanism I'm using to connect is a debugging backdoor that may be slammed shut one day...

I've also modified my client code so that it requests a complete update from the server when started. This will allow all clients (deployed in separate control panels) to be up to date when booted up, should the server receive state before clients boot.

Back to it...

  -- Dave

Originally when Lionel released the Legacy code, they made a point of saying you had to use the SER2 to access the full Legacy command set.  I always figured they had done something different in the internal communications that would make it more difficult to use the "raw" data.  Of course, the app has to communicate with the BASE3 over the network, but there might be some sort of "key" that gets changed periodically with releases that will break the communications.

@cdswindell posted:

State management is now in place and working well, both on the PyTrain server as well as clients. Engine speed, direction, and startup/shutdown status is tracked, and maintained correctly when changed via a Cab 3 as well as impacting commands (engine reset and halt commands).

My next plan is to use the engine speed state to smoothly ramp up or down speed changes.

Stay tuned.

  — Dave

That's pretty cool!  Are you keeping some sort of state table for everything?  If so, I'm guessing it is dynamic in nature (record creation and destruction) so as to minimize memory usage.

George

@G3750 posted:

That's pretty cool!  Are you keeping some sort of state table for everything?  If so, I'm guessing it is dynamic in nature (record creation and destruction) so as to minimize memory usage.

George

Yes, I'm keeping a state table for the devices discovered by sniffing the TMCC command flow fro the LCS Ser2. The info is fairly minimal, to reduce the memory footprint. For example, for switches, just the thru/diverged info. For Accessories, a triplet for Aux on/off, Aux 1 on/off, and Aux2 on/off. For engines and trains, for now, I'm only listening for speed, direction, and if the engine has been "started" or "shutdown". I could maintain more info, such as momentum, brake level, smoke, etc, but I'm not sure I'll need that for what I'm doing (but it could be added).

I never really delete any state records, as on a typical layout, you are not removing switches or accessories. I could time out engine/train state records, but they really don't take too much memory.  On my Pi Zero W 2, which I think only has about 512MB of ram, the program only uses about 20mb.

  -- Dave

@cdswindell posted:

Yes, I'm keeping a state table for the devices discovered by sniffing the TMCC command flow fro the LCS Ser2. The info is fairly minimal, to reduce the memory footprint. For example, for switches, just the thru/diverged info. For Accessories, a triplet for Aux on/off, Aux 1 on/off, and Aux2 on/off. For engines and trains, for now, I'm only listening for speed, direction, and if the engine has been "started" or "shutdown". I could maintain more info, such as momentum, brake level, smoke, etc, but I'm not sure I'll need that for what I'm doing (but it could be added).

I never really delete any state records, as on a typical layout, you are not removing switches or accessories. I could time out engine/train state records, but they really don't take too much memory.  On my Pi Zero W 2, which I think only has about 512MB of ram, the program only uses about 20mb.

  -- Dave

Makes sense.  If you limit the amount of information kept per engine and only keep state for detected, i.e. started locomotives, you'll be fine.  Statically accommodating comprehensive records for hundreds of locomotives might eventually be a problem, but you are going about this logically and efficiently.  And you're doing this with a Raspberry Pi, not an Arduino.

George

I'm now able to send TMCC commands directly to the Base3 via wifi (not via the LCS Ser2). Although this is good (), it is tempered by not all TMCC commands are echoed out from the base. It may be possible to query the base for detailed engine state info (is the bell ringing?; is the engine started up?), it may be a lot easier to read the state info as the stream of TMCC commands available via the Ser2.

I may focus for the time being on reading from the Base 3 only the state information I'm most interested in for managing control panel state. This includes switch positions, accessory state, and power district state. But I now have in place the some of the architecture I'll need to perform bidirectional communication with a Base 3.

  -- Dave

I ran into an interesting situation today while coding up my PDI packet parser for the ASC2.  In TMCC land (via the LCS Ser2), accessory commands focus on the Aux 1 and Aux 2 buttons on the Cab2 controller. This makes it very easy to understand which button was pressed. That said, it is Not at all clear, whether buttons are being used to control a momentary action device, such as the little man that goes round and round inside a control tower, or an on/off toggle device like the interior light in that control tower.

With the ASC2, you are presented with very clear on and off states. The Aux1 button sends a series of “on” packets for as long as you keep the button pressed, and sends a single “off” packet once you take your finger off the button. The Aux2 button functions completely as a toggle. The first time you press it, and “on” packet is sent. With the second press, an “off” packet is sent. This is great, however, you have no way to tell which button was pressed!

I want to use the PDI data available on the Base 3 to initialize my system. This mismatch between TMCC and PDI lands makes this challenging.

Any thoughts?

Thanks,

  — Dave

I’m now reading switch state from the PDI events generated by the STM2 and the ASC2, as well as power district events from the BPC2. I still need to add code to query initial states when the system powers up, but I have most of the code in place to do this.

I am taken by how very different the PDI command structure is from TMCC. There is really no overlap. PDI devices all share/support a set of “common” commands (get, set, receive), and actions (config, status, info, reset, identify, clear errors, and firmware) as well as a few device-specific commands and actions. Most of the common commands focus on what I would call "system administration"; reporting comm errors on the PDI bus, getting firmware revisions, etc. These have really nothing to do with running trains or operating accessories,  And, as I mentioned previously, PDI land doesn’t receive any notifications at all about many of the events that occur in TMCC land, meaning I will need my Ser2 when I roll out my first control panel.

  -- Dave

I’ve added support for Python 3.10, 3.11, and 3.12, as well as have started using Tox to run my test suites. This week, I’ll add support to send commands directly to the Base 3, removing the requirement for an LCS Ser2. However, you will still need the Ser2 if you want to issue the extended TMCC commands (like to trigger dialog), and if you want to monitor the TMCC command stream that is made bailable via the Ser2 (not all TMCC commands are broadcast from the Base3).

I am also reading up on how to package the code and put it up on PyPi. At that point, it will be much easier to install and use the main executable, PyTrain!

  -- Dave

Code is checked in to send commands directly to a Base 3 (or Base 2 via a LCS WiFi module). This means that those of you without an LCS SER2 (or any LCS devices) can also use PyTrain to control your layouts.

  -- Dave  

I can now read Acs2 and Bpc2 states at system startup. I will add support for other Lcs modules ads/if folks (and myself) need it.

  -- Dave

Is there anything people are working on that would take the place of a Base unit?

Thanks,

John

I can now read the Engine and Accessory rosters from the Base 3. It will be easy enough to do the same for switches and routes as well. Between this and the code I added earlier to query LCS devices t system startup, I have all the pieces I need to keep system state in sync.

Reading data from the Base 3 also lets me get the road names nd numbers of everything, but I don’t think this will be of too much use in fabricating control panels, at least the ones I plan to build.

The Raspberry Pi’s continue to be able to handle all of the work, and the memory footprint is still pretty minimal.

The next big piece left is packaging the code so I can upload it to PyPi. This will make it much easier for end users to grab the code without having to set up a development environment.

  — Dave

I am now updating the Base 3 database when changes are made to an engine/train's speed, momentum, diesel run level (rpm), engine labor (efx), smoke level, and train brake via TMCC commands. All changes are reflected back from the Base 3. Only speed and momentum changes are correctly reflected on the Cab 3. I've made @Dave Olson aware.

I'm not sure if the Base 3 maintains the fwd/rev direction state of a train or engine (relative to motion, not the direction an engine is facing in a consist). Would be nice to sync this state as well, if possible.

So, to recap progress (and I may repost this separately), here is what's working:

• All TMCC-style commands can be sent via Ser2 and/or Base 3 except for consist creation and modification (See V1.21 of the Legacy Command Protocol document)
• All 3 byte Legacy/TMCC2 commands can be sent via Ser2 and/or Base 3 except for consist creation and modification (Also in V1.21 of the Legacy Command Protocol document)
• Almost all multibyte Legacy/TMCC2 commands have been implemented. They can be sent  via Ser2 and/or Base 3 (multibyte commands include smoke levels, dialog controls, advanced lighting controls, etc, see V1.21 of the Legacy Command Protocol document)
• Ability to define a sequenced command (maybe I should call this a component command) that allows you to issue multiple commands with specified time delays and repeats, as needed. For example, I've developed ones to perform the grade crossing whistle blows that happen when you use an IR track, and that do a sequenced speed increase that includes engine and tower dialog. These mimic the "official" railroad speed buttons on the Cab 2 and Cab3, but include the tower and engineer dialogs issuing the speed-up (or slow down) dialog, the change speed command, then the engineer acknowledge dialog.
• All TMCC and Legacy commands broadcast from the Ser2 and echo'd to the console.
• All PDI commands broadcast from the Base 3 can be echoed to the console
• All broadcast TMCC, Legacy, and PDI commands can be "subscribed to" and callback functions can be invoked when specific actions occur (this feature is used to support state management as well as console echoing)
• Switch, Accessory, and some Engine/Train state are maintained.
• State is initialized at startup by querying LCS modules and the Base 3 itself
• Client/Server mode is supported where the code on one Raspberry Pi can service multiple connected clients. Clients can send and receive commands and state updates as if they were the server module. Only the server needs connection to an LCS Ser2 and/or a Base 3.
• TMCC state changes are pushed to the Base 3 (and Cab2/3 controllers) automatically
• A command line interface is provided that allows all TMCC and Legacy trains, engines, switches, accessories, and routes to be operated and monitored. This same program is used to perform TMCC/Legacy/PDI command echoing.
• Most implemented commands can be issued via the command line interface.
• Support has been added for the LCS BPC2, STM2, Ser2, and ASC2 modules.
• Software support exists to issue all of the common LCS  PDI commands (Firmware, Status, Config, Info, Clear Errors, Reset, and Identify). It is a TODO to make these commands available via the command line interface.
• Helper methods are provided to simplify the association of buttons, LEDs, toggle switches,  joysticks, rotary encoders, and pots to TMCC and Legacy commands and dynamic state. This means you can associate the commands to throw a track switch to buttons on a Raspberry Pi, and have an indicator light that correctly reflects the switch state, even if thrown another way (e.g., the black Lionel controller, your Cab2, derail prevention feature, LCS STM2, etc). This was the raison d'être for the project
• Higher-level methods exist to control Lionel Culvert Loader/Unloaders, Gantry Crain (some work left to finish here), Smoke Fluid Loader, Generic Route, Generic Switch, Generic Accessory, Power District, and Engine.
• Ability to define buttons, lights, components, etc. in a startup file that is loaded when the command line program runs; this is where you will "define" the physical components on your control panel and associate them with the TMCC/Legacy operations to perform when the controls are manipulated.
• All code is in Python3 and runs and has been tested on Python 3.10, 3.11, and 3.12 (the most recent version supported by the Pis is 3.11) and is up on Github today.
  
  

Remaining work:

• Package the code to enable downloading from PyPi. This will allow folks that don't want to download source code to install the executable that makes all this work.
• Documentation (sigh)
• More tests
• Implement logging (remove print statements)
• Modify sequenced speed commands to look at engine state, so if you give a speed change command, the engine gracefully ramps up to or down to the new speed)
• Add generic support for rotary encoder
• Complete Dialog Control commands
• Build a control panel

  -- Dave

I should add, if anyone is interested in giving the code a try in its current state, it would be much appreciated! Email or DM me for more info.

  -- Dave

Email sent. Thanks!

Just checked in the rest of the engineer/tower dialog effects on page 13 of the Lionel protocol spec. These are some fun things, like the engineer speaking the current speed, fuel, and water level, and the tower to engineer and engineer to tower arriving, arrived, departing, and departed dialogs. Using randomization and setting up event watchers on the IR sensor tracks would allow users to trigger random dialogs as your train motors along the rails.

  -- Dave

Email also sent, thanks!

The things I'll do to avoid taking on something I have trepidations about (packaging).

I've implemented support for the seven common PDI commands for the LCS devices (FIRMWARE, STATUS, INFO, CONFIG, RESET, IDENTIFY, and CLEAR_ERRORS). These can be issued from the server pytrain instance (the one with the Base3 connection). The commands let you display information about your LCS devices (modes, voltage, firmware, errors), turn on and off the little red Ident LED, clear out error counts, and reset the module (I have no idea what this does; sounds dangerous!).

I've also returned to the house where my layout is at and tweaked support for the STM2, BPC2, an Sensor Track modules, and I must say, things are working pretty nicely! At power-on, you get a complete list of all engines, trains, switches, and accessories, along with their names, road numbers, and current state.

One thing I need to do is to send a discovery command when a power district is turned on. Switches that are contained in a power district don't report their actual state until power to that track section is turned on. This is a simple addition for me to make.

  -- Dave

This is very exciting work! I could only dream of having the time and knowledge to tackle something like this. I'm having a little trouble understanding some of the posts but it gives me a good reason to try out Python. I'm trying to learn C# at the moment for work so I figure learning any code will benefit me.

@BillYo414 posted:

This is very exciting work! I could only dream of having the time and knowledge to tackle something like this. I'm having a little trouble understanding some of the posts but it gives me a good reason to try out Python. I'm trying to learn C# at the moment for work so I figure learning any code will benefit me.

Happy to answer any questions you have either here or in email!!

  -- Dave

I have support for Sensor tracks in. This could definitely be used to trigger events when engines pass over them. I could also have the engine actually "respond" to the "go to normal speed"/"go to slow speed". I will probably enable this as an option...

  -- Dave

Would these events include items outside of the Lionel ecosystem? Like say environmental sounds, scenery animations, or digital information? And as far as digital information, that would be things like moving digital freight around the layout.

This is starting to smell a lot like @Ryaninspiron's gamification project.

https://ogrforum.com/topic/int...rack-based-game-idea

It seems like that may have been stalled in the PDI communication implementation, but I don't see why it couldn't use the wifi connection.  I believe his goal was to have a pure LCS implementation (no serial), and it sounds like sensor track may have been the last piece of that puzzle.  I assume sensor track signals can be received via LCS wifi here right?

@Patrick Keistler posted:

This is starting to smell a lot like @Ryaninspiron's gamification project.

https://ogrforum.com/topic/int...rack-based-game-idea

It seems like that may have been stalled in the PDI communication implementation, but I don't see why it couldn't use the wifi connection.  I believe his goal was to have a pure LCS implementation (no serial), and it sounds like sensor track may have been the last piece of that puzzle.  I assume sensor track signals can be received via LCS wifi here right?

Absolutely. In fact, the Irda/Sensor Track packets are only sent out via PDI/WiFi.

@BillYo414 posted:

Would these events include items outside of the Lionel ecosystem? Like say environmental sounds, scenery animations, or digital information? And as far as digital information, that would be things like moving digital freight around the layout.

Yes, absolutely. For example, if you had a gate man connected to an LCS ASC2, you could trigger him coming out of his little house when the train went over the sensor track. True, you could accomplish the same thing with physical wiring, but there is an elegance to doing so by issuing TMCC commands and even using random numbers to control how long the response action lasted.

@Patrick Keistler posted:

This is starting to smell a lot like @Ryaninspiron's gamification project.

Yup, @Ryaninspiron has done a bunch of programming work... the posts in his profile are worth a peek for sure.

@cdswindell my layout is based on a steel mill. It's still in the very early stages of development but I often plan later stages for future reference since I find slowly working on an idea helps me get it right the first or second time when I build it.

I've toyed with the idea of a fast clock but many people aren't experts on steel making and running against a rigid clock might end up causing chaos in an ops session. I was thinking an event driven sequence would be better. So a car being spotted under the blast furnace would trigger a timer and the furnace tapping animation sequence (lights, smoke, sound) would occur after a set amount of time.

This is probably worth emailing you about for the finer details at this point but having the ability to use the locomotive/car itself to activate this would save me some wiring and hiding of sensors.

I should probably start learning how to use LCS and Python!

Hey all, thank you for the @mentions 😁, I appreciate it. Yes stalled but not completely dead in the water at the PDI phase. I wanted to skip over Wi-Fi because that introduces a whole new level of hassle to the user not to mention possibilities configuration issues getting in the way. For my game idea I wanted to kind of develop the game as a PDI module to make it as easy to implement as possible for the widest number of people. I did take on a new job in May which have been consuming so much more time besides what my business had originally. But I am absolutely down to contribute anything I can to any project hopefully continue with everything else I was doing too. My availability has been starting to get a little better lately (knock on wood).

Most recently I've also been working with my train club TMB to . For them a large scale map of the layout on the wall with live tracking block occupancy across the layout. Which as of last year is now located in a 6,000 ft basement below a church.

I'm going to have to do some reading to catch up on these last two pages in this thread. But I love any and all projects that give LCS a purpose for existing, besides setup and simple location pinging in the old lcs ipad app. Plus I love to see interaction happening with the layout I think that's where the next era of fun to be had can be found.

Plus I recently learned some fun tricks using python code that could absolutely help me break down the PDI data, struct.pack and struct.unpack. Part of what made things tricky before was working in Arduino code exclusively and facing off with decoding raw binary data via bitmasking was not nearly as fun.

And here are a few of the modules I was tinkering with to develop these interfaces. On the back left corner there is a radio that is supposedly capable of impersonating a DCS remote which I was planning to interface with TMCC.

@Ryaninspiron posted:

Plus I recently learned some fun tricks using python code that could absolutely help me break down the PDI data, struct.pack and struct.unpack. Part of what made things tricky before was working in Arduino code exclusively and facing off with decoding raw binary data via bitmasking was not nearly as fun.

@Ryaninspiron, we may make a good team I like decoding raw binary data via bitmasking !

But seriously, I have code in place that decodes 80 to 90% of the PDI commands in the confidential Lionel PDI/LCS docs. The code is all checked in on GitHub. Just search for the PyLegacy project!

  — Dave

I decided to start maintaining state for the LCS Sensor Track. It plugged into the stack with about 90 minutes of coding, which felt pretty good. I could also manage state for (the) other LCS modules, but I’m not sure I’ll need to. I was already processing state changes signaled by the Base, BPC2, ACS2, and STM2 by translating their updates into Switch, Accessory, and Engine/Train space (e.g., recording an STM2 update as a Switch Thrown update).

It will now be easy to have an engine/train go to slow speed or go to normal speed when it crosses over a Sensor Track programmed to generate that sequence.

  — Dave

@Ryaninspiron posted:

Plus I recently learned some fun tricks using python code that could absolutely help me break down the PDI data, struct.pack and struct.unpack. Part of what made things tricky before was working in Arduino code exclusively and facing off with decoding raw binary data via bitmasking was not nearly as fun.

@Ryaninspiron, huh, I was unaware of the struct package! That would be very useful for my code that writes PDI/LCS packets that I send to the base, thanks!

  — Dave

@BillYo414 posted:

@cdswindell my layout is based on a steel mill. It's still in the very early stages of development but I often plan later stages for future reference since I find slowly working on an idea helps me get it right the first or second time when I build it.

I've toyed with the idea of a fast clock but many people aren't experts on steel making and running against a rigid clock might end up causing chaos in an ops session. I was thinking an event driven sequence would be better. So a car being spotted under the blast furnace would trigger a timer and the furnace tapping animation sequence (lights, smoke, sound) would occur after a set amount of time.

This is probably worth emailing you about for the finer details at this point but having the ability to use the locomotive/car itself to activate this would save me some wiring and hiding of sensors.

I should probably start learning how to use LCS and Python!

@BillYo414, happy to have an email conversation. The code in its current form can definitely generate events in response to an engine (or car with an IR transmitter) traveling over a sensor track. The event packet contains the ID of the activated sensor track, the direction of travel, the engine/train ID passing over, and a bunch of what I call simulated state (fuel/water level, odometer, total runtime, etc.)

You could write code that “subscribes” to sensor track events. That code would then receive these updates and could go from there. You can also access current engine speed and direction from other components of my system and/or by subscribing to engine/train events.

I don’t currently have any code to sense engine position (other than the sensor track), so I’m not sure how you would tell when a car was next to/underneath your steel mill stuff. My code gives you fine control over train speed (absolute speed step control), as well as commands to immediately stop engine motion immediately. In fact, I’ve implemented all of the TMCC commands in the public Lionel spec (except those to build/tear down train consists)!

  — Dave  

@cdswindell posted:

@Ryaninspiron, we may make a good team I like decoding raw binary data via bitmasking !

But seriously, I have code in place that decodes 80 to 90% of the PDI commands in the confidential Lionel PDI/LCS docs. The code is all checked in on GitHub. Just search for the PyLegacy project!

  — Dave

Absolutely would be great to join forces, and since you love bitmasking so much you might just love to see the work I've done on reverse engineering the sensor box cars and therefore the ability to create postage stamp size modules to add sensor track capabilities at any number of cars or engines. I've already demonstrated one of these modules in operation on one of my YouTube videos on the subject. But until now I haven't had anybody else to share the load with. I had actually ordered about 20 old school irda transceivers, and use that to capture the data into a bit array that I then saved into Arduino code, and then put matching code to retransmit that bit stream back out on the two in IR LEDs. I also did testing and figured out exactly which wavelength of ir LED you can use off the shelf from Amazon as a sensor card transmitter. I have one other person who tested and deployed my code successfully too. But they did not end up having time to help me make progress with completely cracking the code such as transmitting a-la-carte engine IDs and names. But I do have a demo on youtube from several years ago where I trigger sounds on a line Chief engine using my home made sensor car. https://youtu.be/qgGiVOycX8o?si=cXwrYXJELr2Zrtvv

So yes I agree we could make an amazing team, I'm also an LCS partner too. Honestly that made me nervous about what I was allowed to disclose and it made me hesitant to simply publish my code before.

Of course everything I learned on the sensor car box car was through traditional reverse engineering since none of that documentation was included in the partner program. I also had set up a discord server to host discussions about this sensor car reverse engineering project including links to Amazon for buying all the hardware to test it on your own. https://discord.gg/ptmd79Zd

@cdswindell posted:

@BillYo414, happy to have an email conversation. The code in its current form can definitely generate events in response to an engine (or car with an IR transmitter) traveling over a sensor track. The event packet contains the ID of the activated sensor track, the direction of travel, the engine/train ID passing over, and a bunch of what I call simulated state (fuel/water level, odometer, total runtime, etc.)

You could write code that “subscribes” to sensor track events. That code would then receive these updates and could go from there. You can also access current engine speed and direction from other components of my system and/or by subscribing to engine/train events.

I don’t currently have any code to sense engine position (other than the sensor track), so I’m not sure how you would tell when a car was next to/underneath your steel mill stuff. My code gives you fine control over train speed (absolute speed step control), as well as commands to immediately stop engine motion immediately. In fact, I’ve implemented all of the TMCC commands in the public Lionel spec (except those to build/tear down train consists)!

  — Dave  

Wow I was just in the middle of implementing a project that uses industrial communication modules(modbus rtu) RS485 to capture the occupancy of all the blocks on a large scale 6,000 sqft layout and was just envisioning tracking engine IDs along the way by tacking them onto the continuous chain of block occupancy that passes over a sensor track.

This feels a bit like a milestone to me. I now have trains automatically slowing down or speeding up as they pass over a sensor track! They only do this if the sensor track sequence is set to one of the two values where a speed dialogue is given, based on the direction of travel. I have some work to do to make the speed up or slow down, look more realistic by taking the train’s Momentum, setting and current speed into account, but I have all the pieces sitting around to do this (in code, of course).

I’ll post a video to YouTube and include its link in a day or two.

I still haven’t started packaging.

  — Dave

Dave,

Looks like I'm in... with a  little overkill.  I just pulled the trigger on the 8G Pi 5 (bare board).  I have a Samsung M.2 970 Pro that I just took out of the workstation and it's gathering dust so I ordered the Pineboards POE+ (2280 M.2).  Not the cheapest configuration... but hey, it's for the trains.

@Dennis-LaRock posted:

Dave,

Looks like I'm in... with a  little overkill.  I just pulled the trigger on the 8G Pi 5 (bare board).  I have a Samsung M.2 970 Pro that I just took out of the workstation and it's gathering dust so I ordered the Pineboards POE+ (2280 M.2).  Not the cheapest configuration... but hey, it's for the trains.

That will more than do!!! FYI, I just installed the M2 hat that Raspberry Pi itself distributes and it worked perfectly. One thing to watch out for. I had better luck using the “Copy SD card” in the Pi gui to clone my system image onto the M2 drive (requires a VNC connection; everything else I do via ssh).

  — Dave

Pi5 in the house... didn't use the micro sd drive, booted straight off the USB with a Samsung T5 ssd.

Just checked in support to ramp speeds up and down, more like the Cab2 and Cab3 throttles do. You can still jump immediately to a desired speed, but the default is to now use a ramp that also takes the engine’s momentum into account.

ill post a video tomorrow. Looks nice when the speed is changed by going over a sensor track!!

  — Dave

Checked in support for a Rotary Encoder. Think of these like digital potentiometers, except they rotate round-and-round, just like the big red throttle knob on the Cab 2.

I've incorporated this support into GpioHandler.gantry_crane and GpioHandler.track_crane to rotate the cab. Of course, with that said, when I unboxed my Lionel Legacy CSX Track Crane car, I found it was missing a gear necessary to rotate the cab! But my code does send the right commands and I can see the gears they did install turn in the correct direction. Sigh

Here's what the code looks like a user would have to write to build a dedicated panel to control a track crane car:

GpioHandler.crane_car(
    address=91,
    cab_pin1=20,
    cab_pin2=21,
    adc_channel=0, (1)
    boom_pin=16,
    bh_pin=25,
    sh_pin=26,
    boom_led_pin=23, (2)
    bh_led_pin=24, (2)
    sh_led_pin=22 (2)
)

I better get going on this before you do everything cool!

I really can’t wait to build my own legacy controller - I just keep getting sidetracked with other projects!!!

@cdswindell posted:

…use a ramp that also takes the engine’s momentum into account

  — Dave

I was playing with “momentum” in analog controllers over the summer. I found that for it to really look and feel right, you need to be playing calculus games - actively changing the rate of change over time (second derivative? I forget, it’s been an awfully long time…) to ease in and out of acceleration and deceleration. Otherwise it’s just like a sort of delayed speed change effect.  With complete control over the software stack like you are building, I bet you can really do momentum right!

@christie posted:

I was playing with “momentum” in analog controllers over the summer. I found that for it to really look and feel right, you need to be playing calculus games - actively changing the rate of change over time (second derivative? I forget, it’s been an awfully long time…) to ease in and out of acceleration and deceleration. Otherwise it’s just like a sort of delayed speed change effect.  With complete control over the software stack like you are building, I bet you can really do momentum right!

Calculus was an awfully long time ago for me, but you absolutely could build any kind of speed table that you want. I took a look at what the Lionel Cab3 controller does and tried to emulate that. My implementation at this point is all based on linear map, but I guess I was just lazy.

  -- Dave

@christie posted:

I better get going on this before you do everything cool!

I really can’t wait to build my own legacy controller - I just keep getting sidetracked with other projects!!!

The software I've developed implements ALL of the TMCC commands and most of the PDI/LCS commands. you absolutely could!

  -- Dave

Added a "decode" command to PyTrain to convert hex codes seen in the LCS Monitor app (or Wireshark Sniffer) to the corresponding TMCC or LCS Command.

For example:

>> decode D1 28 F8 87 CC 8D DF
0xd128f887cc8ddf --> [PDI Tmcc_Rx [ENGINE 67 ENGINE_LABOR 12 (0xf887cc)] (0x28f887cc)]
>> decode 0xF88703
0xf88703 --> [ENGINE 67 REVERSE_DIRECTION (0xf88703)]

-- Dave

Finally added logging to the code. Log messages can go to both a file and the console at the same time. Logging can also capture command echoing, so you can have a complex transcript of all the TMCC commands happening on your layout, be they from a Cab 2, a Cab 3, or from the PyTrain command line. This really lets you see the commands the Cab 2 and Cab 3 generate when you operate a control; you’d be surprised!

  — Dave

I received my Gantry Crane today and ****ed if the components I’ve wired up to control it did just that! I turned the rotary encoder and the crane cab turned. I pushed up and down on the joystick and the magnet raised and lowered. And I pushed left and right on the joystick and the crane moved left and right!!

I have some tweaking to do, as the code that sends the numeric prefix (<Aux 1> [1, 2, 3]) needs to be synchronized so that state isn’t lost when you operate multiple controls at the same time, but it works!

  — Dave

I've switched gears to working on support for making the system easier to use and more bulletproof. This is involved two changes. The first is I've developed some scripts that allow you to run the software when the raspberry pie boots up. This will be important because when powers turned onto the layout, I want the pies that are powering the control panels to just boot up and be working, like an embedded device.

The second change is to use a discovery service to allow the PyTrain server(s) to register their presence on the local network. This will allow the client applications to not need to fool with IP addresses. They will auto-discover The IP address of the server, and just simply connect to it. Since the Base 3 Doesn't echo out very many of the commands it receives in, I still think it will be advantageous to have an LCS Ser2 connected to the server. That said, I'm investigating if it makes sense to have each client send train commands directly to the Base 3 over WiFi. This should help distribute the load.

I'm working now on actually building the first control panel to operate my gantry crane. I'll send pictures along once there's something to show. By the way, all of the code is checked in.

  -- Dave

Added support to allow engines to be looked up by road number. This makes it easier to address engines, as the need to remember the associated TMCC IDs is eliminated. This is a precursor to adding support for 4 digit TMCC IDs…

I also figured out how to update the Base 3 and Cab 2 in a more efficient way. This support isn’t checked in yet, but will come online this week.

i also am about to fabricate my first controller panel! It will just be a prototype. I’ve switched over to using an I2C matrix keypad, which should give me 19 GPIO pins to wire up to fire commands!

  — Dave

Here's a picture of the completed prototype:

The throttle knob is a $12 rotary encoder intended for CNC machines. It has a nice feel with distinct "clicks" as you rotate. My attempt at soldering leads onto a "soft pot" was a total failure. I hadn't noticed you could buy them with connectors already attached. That Amazon order arrives next week

On the software side, I've added a number of commands intended to make operating a cluster of PyTrain servers (actually, 1 server and many clients). You can now issue a single command that will update the PyTrain software across the cluster, as well as upgrade the Linux OS on all systems with one command from one system. I've also enabled the ability to run a client on the same system as a server. Why, you may ask? My intension is to start my software automatically when each box boots. This way, when you turn the power on to your layout (and control panels), all of the integrated Pi systems will just boot up and run the code. But with this method, where all the systems are "headless", there is no operator console available to issue commands! This is where the need to run a client on the server comes in. When it comes time to upgrade the OS or my code, you cans onto one of the systems (preferably the primary server), run the software in "client" mode, then issue the appropriate command (update to update my software, upgrade to upgrade the raspberry pi OS as well as my software), or reboot, because sometimes...

I also added an ability to automatically find the Base 3 on a local network. I used Wireshark to try to figure out how the Cab 3 finds the Base, and, as best as I can tell, it simply tries all IP addresses on the local network (255 - 3, or 252), stopping once it succeeds. I am using Python's "multiprocessor support" to do this, so on my 24 core development Mac, it finds the base in just a few seconds! On a Pi 5, with just 4 cores, it can take 10 - 20 seconds to find the Base, but as it only needs to do this once on start-up, and only on the "server" instance (clients can get the base address from the server), this may be sufficient. I will probably hard-wire the base address into my server start-up script, but I wanted the option...

  -- Dave

For 30 years we've been stuck with the control surfaces that Lionel gave us or those developed/licensed by a small number of competitors and developers. Now, we can create basically any interface our imagination can think up, or anything our layouts demand. This is amazing work, Dave.

This is awesome work.  Well done, Dave.

Now that they're officially out of the handheld controller game, I hope Lionel takes notice and really open things up with Legacy to allow 3rd parties to bring their own control hardware, and even actively support integration with Base3 and a lot of the other stuff they make.  This is a great way to get technically minded folks into the hobby, too.  I don't really see how it could hurt them unless they ever change their mind about making hardware remotes, but even then, most people would probably just buy theirs again anyway.

Just added an "when_pressed_or_held" handler that can issue one of 2 commands, based on how long a button is held. This is useful for handling things like startup and shutdown, where there are really two flavors of the command; one that does the action without dialog, and another that does it with dialog. The threshold is settable, but .3 seconds seems to work nicely...

  -- Dave

Just added more realistic syncing of Diesel RPM levels as you increase/decrease throttle via the PyTrain methods. I have to do the same for Engine Labor as well (TBD).

Does anyone know offhand if Engine Labor (effects) applies to Steam and/or Electric, as it does to Diesel? I'm only applying RPM effects to Diesel engines (as does the Cab 3).

  -- Dave

I do believe they apply to steam, you use the same controls on the CAB1 to increase the labor for steam.  I can't say for sure how that translates into the Legacy commands, but given you use the same techniques for steam or diesel to increase labor, I would suspect it would be the same commands.

@gunrunnerjohn posted:

I do believe they apply to steam, you use the same controls on the CAB1 to increase the labor for steam.  I can't say for sure how that translates into the Legacy commands, but given you use the same techniques for steam or diesel to increase labor, I would suspect it would be the same commands.

Thanks John.

Yes, the commands are the same, I just didn't know if the concept applied to steam. I would have thought the concept of RPM applied to steam too, but if you pull up a steam engine on the Cab 3, the RPM bars to the left of the throttle disappear...

It's not RPM on steam, it's labor.  Obviously, unless the steam engine changes speed suddenly or has wheel-slip, the RPM won't change.  However, if you pull the train brake down and then run the speed up past where the train brake is limiting the speed, the labor increases.  By the same token, if you have momentum set at high and spin the throttle up quickly, you'll also get increased labor until the actual locomotive speed catches up with the speed setting.

The concept of "working harder" applies, but with a diesel, it's the prime mover ramping up in speed to generate more power, with steam it's the amount of steam fed to the cylinders to produce more power to the wheels.  Same idea, just a totally different way of accomplishing it.

@cdswindell posted:

Thanks John.

Yes, the commands are the same, I just didn't know if the concept applied to steam. I would have thought the concept of RPM applied to steam too, but if you pull up a steam engine on the Cab 3, the RPM bars to the left of the throttle disappear...

I think you’re taking about two different things. RMP definitely is only in diesels, but going into the speed menu on the Cab-2 and now you have labor adjustments labeled EFX. This shows up on diesel, steam, and I’m pretty sure it’s also on electrics.

I haven’t looked for that menu on the Cab3 App. I’ll get back to the base 3 tonight and see if I can find it.

Finally completed support for using I2C digital I/O extender boards to add more buttons. The code creates a new type of gpiozero “Button” that has all of the same behavior. This means that you can use the “when_pressed” and “when_held” call backs, which was central to my design.

The code was developed for the CQ Robots Ocean 16 port board, but should work with any board with a MCP23017 chip. The nice thing about this chip is it provides interrupt support so you do not have to poll the board looking for button presses.

I don’t think I’m going to take the time now to add support to drive LEDs, although I think doing so would be pretty simple.

  — Dave

@cdswindell posted:

Finally completed support for using I2C digital I/O extender boards to add more buttons. The code creates a new type of gpiozero “Button” that has all of the same behavior. This means that you can use the “when_pressed” and “when_held” call backs, which was central to my design.

The code was developed for the CQ Robots Ocean 16 port board, but should work with any board with a MCP23017 chip. The nice thing about this chip is it provides interrupt support so you do not have to poll the board looking for button presses.

I don’t think I’m going to take the time now to add support to drive LEDs, although I think doing so would be pretty simple.

  — Dave

It was pretty simple; support for I2C LEDs is now in place

Code has been modified to use gpio extender ports or regular pi ports in all GpioHandler methods (route, switch, controller, etc).

  — Dave

I finally tackled packaging and now have the code up on Pypi’s test site. And, as I had hoped, you will soon be able to get up and running by typing 3 commands! I’ve also developed a tool to configure a Raspberry Pi to run the code and remove extraneous services and packages not needed to run PyTrain. This helps reduce the system load on the small Pi devices, although it runs fine even without the optimizations.

This wraps up the development efforts for PyTrain. The major outstanding task is completing four digit addressing support. I’m hoping to have a four digit addressable engine next month, at which point I will complete that support.

Next up, I've decided to implement a Restful API. This will allow developers working in other languages to control trains. And it is a necessary component for my next major feature; Alexa Control! With the ability to query ngine names and road numbers, and access to the complete command library, this should allow me to develop a pretty full-featured Alexa skill to run my layout.

  -- Dave

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

I can't think of a use case but I'd certainly think of a use with Home Assistant integration.

Actually I did think of use cases, e.g. blow whistle when motion detected near train room door
Stop train when motion/presence no longer detected
X number of bell hits dims train room lights

And finally, zigbee devices can (such as a simple button) can activate things on the train. Easy to set up interactions for kids/adults alike

https://www.home-assistant.io/

@kwilliamm posted:

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

@kwilliamm

As Homer might say, "doh!"  I had not checked pypi Until right now, and yes, pytrain is already taken. Kind of makes sense, given all the machine learning interest out there. I will add some kind of suffix that makes sense, at least to me. Although suggestions will be welcome!

one of the reasons I took on packaging is that I wanted to keep the restful API and Alexa integration in a separate project and not have the overhead of it in the main py train. I definitely agree. They should be separate projects and they will be. I'm still debating if I wanna combine the restful API and Alexa into one or even break those out separately. Do you have an opinion on that?

I have no experience with DCS. I'm assuming this is what MTA uses? I think I remember hearing somewhere that MTH was very proprietary and actually went after someone that tried to reverse engineer their protocol. I don't have any of their equipment, nor any of their engines, so at least for the time being, i'm not gonna take that on.

Thanks!

  -- Dave

@cdswindell posted:

I have no experience with DCS. I'm assuming this is what MTA uses? I think I remember hearing somewhere that MTH was very proprietary and actually went after someone that tried to reverse engineer their protocol. I don't have any of their equipment, nor any of their engines, so at least for the time being, i'm not gonna take that on.

Mark DiVecchio has done a lot of work with DCS and has decoded their protocol very well.  Check out his RTC page: http://www.silogic.com/trains/RTC_Running.html

This is a pretty impressive program and he does it all via the WiFi.  He has extensive information on the actual protocol to talk to the DCS as well.

@Rollsington posted:

I can't think of a use case but I'd certainly think of a use with Home Assistant integration.

Actually I did think of use cases, e.g. blow whistle when motion detected near train room door
Stop train when motion/presence no longer detected
X number of bell hits dims train room lights

And finally, zigbee devices can (such as a simple button) can activate things on the train. Easy to set up interactions for kids/adults alike

https://www.home-assistant.io/

The way Google and Alexa smart home devices work is by making use of apis of the client applications to control them. Thus, I need to build this into PyTrain in order to build the Alexa integration.

  — Dave

@kwilliamm posted:

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

I think I’ll name the package pytrainrr or pytrain-rr.

  — Dave

I've started to work on better documentation. Lots more to do here, bot have a look at the first few pages  on  Github. You will also see the schematic of the first real control panel I'm building.

  -- Dave

Added support to auto-restart clients when the server is restarted.

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

@gunrunnerjohn posted:

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

Any company needs to prioritize initiatives based on many factors. One would need to accept a physical remote that replicates the Cab-2 and now app didn't make the cut.

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

@David_NJ posted:

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

Nothing like unrealistic expectations!

Just added the ability to "replay" a set of commands at start-up. With this feature, and tools already built into the OS of the Raspberry Pi and MacOS (and probably Windows as well) you can:

• Activate an accessory at a specific time of day
• Start and stop trains based on some external event
• Turn on your layout lights at sundown and off at sunrise (if you pulled an all nighter )
• Blow the horns of all active trains at noon
• and many other...

The PyTrain startup option to do this is -replay followed by the name of a text file containing the commands to run at startup. The command can be run from a PyTrain client or server.

  -- Dave

@gunrunnerjohn posted:

Nothing like unrealistic expectations!

John, can you come up with the pcb gerber files for a remote or is that someone else's bailiwick?  I've spoken with someone about doing the layering of the code for a touch screen... we are sort of at Cab-1L on steroids or Cab-2 lite to keep it reasonable.

@Dennis-LaRock posted:

John, can you come up with the pcb gerber files for a remote or is that someone else's bailiwick?  I've spoken with someone about doing the layering of the code for a touch screen... we are sort of at Cab-1L on steroids or Cab-2 lite to keep it reasonable.

The bigger issue is coming up with a design for the hardware, actually developing the PCB design is just part of the picture.  For something like a TMCC/Legacy remote, the hardware design and debug would be a lot more effort than the actual PCB design.

@gunrunnerjohn posted:

The bigger issue is coming up with a design for the hardware, actually developing the PCB design is just part of the picture.  For something like a TMCC/Legacy remote, the hardware design and debug would be a lot more effort than the actual PCB design.

What if you used my software as the base? Then you are really just hardware debugging the physical switches, pots, and display...

@cdswindell posted:

What if you used my software as the base? Then you are really just hardware debugging the physical switches, pots, and display...

That would be a giant step forward.  The first steps would be coming up with a form factor.  I certainly think it should be a handheld remote.  We'd have to agree on the actual display and exact configuration of the switches so that the hardware matched the finished software.  Clearly, that can be done, just pointing out there's coordination involved between software and hardware development required for such a project.

We don't want to hijack Dave's thread here.... is it worth a thread to try and winnow it out or is there to much to chew on?

Well, I think Dave would be a key player in any such design since he wrote the code!

@Dennis-LaRock posted:

We don't want to hijack Dave's thread here.... is it worth a thread to try and winnow it out or is there to much to chew on?

My own 2 cents is there is a fair amount of discussion to be had as to what features a handheld must control as well as what features would be nice to have. At the end of the day, features will just send a bunch of bytes out the wifi interface to the Base 3 (or a PyTrain server so we have the Ser2 feedback). PyTrain knows how to send out all of the published TMCC and Legacy commands, so it can handle any features the discussion arrives at. I think it would be better to have these discussions in a separate thread. I promise to follow it .

Just to stir the pot, here are some things to ponder:

• Is it acceptable to just support TMCC and Legacy engines (PyTrain doesn't do Bluetooth yet, and many LionChief engines also support Legacy)
• Must it control turnouts? (requires a dynamic LCD and GUI)
• Must it fire routes? (requires a dynamic LCD and GUI)
• Must it operate accessories? (requires a dynamic LCD and GUI)
• Must it be able to build/configure trains? (Gui work and I need to add a few more commands)
• Must it allow editing of road names and numbers?
• Must it allow the addition and configuration of new components (Set address, set command type, etc)

One of the issues with dynamic displays required to implement the more involved GUIs is they may drive the design to something that needs more than a simple 4 line, 20 character LED Matrix display. Inexpensive OLED displays do exist for the Pi, and some even support touch screens. However, this will mean more software. The OLED displays take a few more pins than the LCD screens do (I drive mine off the 2 pin I2C bus on the Pi), which means fewer pins for switches. I do have I2C support for GPIO extenders up and running, but now you need to make room and have power in the handheld for these boards too.

Fun stuff to talk about, and certainly a nice distraction.

  -- Dave

@gunrunnerjohn posted:

That would be a giant step forward.  The first steps would be coming up with a form factor.  I certainly think it should be a handheld remote.  We'd have to agree on the actual display and exact configuration of the switches so that the hardware matched the finished software.  Clearly, that can be done, just pointing out there's coordination involved between software and hardware development required for such a project.

John, agreed!

@David_NJ posted:

Any company needs to prioritize initiatives based on many factors. One would need to accept a physical remote that replicates the Cab-2 and now app didn't make the cut.

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

How nice of you to be willing to buy one 🤣

@gunrunnerjohn posted:

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

@cdswindell posted:

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

I guess we just have to change that.

I have a very limited understanding of the programing and electronic issues in this project.  However, Dave you are doing a phenomenal job in the development of a remote device to replace or upgrade the cab2 handheld.  I hope this thread may wake up Lionel to support the development of a handheld unit or aid in the progress so far.  Although I have my doubts. 

Marty

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

@JD2035RR posted:

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

I just checked in a new program, make_service.py, that installs PyTrain as a service on Raspberry Pis. What does this mean and why should you care? One of my goals is to make the Pi as much as "an appliance" as I can. When you turn on the main power to your layout, it is important to me that all of the embedded Pis in the control panels boot up and start running the PyTrain software with no fuss or further configuration. Lionel's Base 2 and Base 3 do the same thing. Even though they are really special purpose computers with custom programming, when you flip on the main power, they just turn on and start working (most of the time 😉)

The way you make this happen on a computer, like the Pi, is to install your program as a system service. This commit adds the program make_service.py , which creates a system service to run PyTrain as either a server or a client. Once this is done, PyTrain will just fire up every time you power on your control panel(s). The client instances will even wait for your PyTrain server to launch, and the server will wait for the Base 3 to boot.

My goal is to have it all just work!

  -- Dave  

While on the subject of system management (I do realize this is a train site , I've also added a number of commands to PyTrain to simplify the management of a PyTrain environment, AKA a bunch of control panels. Those commands are:

• update: checks for PyTrain updates and installs them as necessary
• shutdown: does a clean shutdown of all Pis. On my master control panel, I plan to have a big button that I will press and hold for a few seconds before I plan to power down the layout. Although Pis were designed to just have their power cords pulled out, no piece of electronics really likes this...
• restart: restarts PyTrain on all clients and servers if something is wonky (which means I didn't do my job 😢)
• reboot: reboots all of the Pis and restarts all services
• upgrade: updates the Linux operating system on each Pi, then does a PyTrain update.

All of these commands automatically relaunch PyTrain whether it is running as a system service (preferred) or from a shell window (for developers and layout operators that want to use the PyTrain CLI to control their layout).

  -- Dave

@cdswindell posted:

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

@JD2035RR posted:

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

Huh, cool! I was unfamiliar with BlueRail. They must have essentially enabled Bluetooth game controller control of their app.

Although I've never done it, you can pair a BlueTooth controller to an iPhone and use it as an "assistive touch" device to control the cursor and operate an iPhone. Someone should order one of these things and see if it can operate the Cab 3!

https://support.apple.com/en-us/111775

  -- Dave

I've started work on the Restful API for PyTrain.

http://127.0.0.1/engine/72 -->

{
  "current_speed": 0,
  "road_name": "Union Pacific GP35",
  "road_number": "0742",
  "tmcc_id": 72
}

I was happy to see I could rather quickly adapt the PyTrain main program to support being called as a provider by another process. From here on out, I want to minimize changes to the main PyTrain GitHub project and do the api-specific stuff in a new GitHub project: PyTrainApi. The one exception is I will probably add the ability to get state info in JSON form...

  -- Dave

Some quick screen shots. I have to think a bit about syntax...

http://127.0.0.1:5000/train/20 -->
{
  "control": "Legacy",
  "direction": null,
  "labor": 12,
  "max_speed": null,
  "momentum": 0,
  "road_name": "Delaware & Hudson",
  "road_number": "0020",
  "rpm": 0,
  "scope": "train",
  "smoke": null,
  "speed": 0,
  "speed_limit": null,
  "tmcc_id": 20,
  "train_brake": 0,
  "year": null
}

http://127.0.0.1:5000/acc/15 -->
{
  "block": "on",
  "road_name": "Upper Main North Power District",
  "road_number": "0015",
  "scope": "power_district",
  "tmcc_id": 15
}

http://127.0.0.1:5000/sensor_track/50
{
  "last_loco_lr": 255,
  "last_loco_rl": 255,
  "road_name": null,
  "road_number": null,
  "scope": "irda",
  "sequence": "slow_speed_normal_speed",
  "tmcc_id": 50
}

http://127.0.0.1:5000/acc/50
{
  "road_name": "Upper Main West",
  "road_number": "0050",
  "scope": "sensor_track",
  "tmcc_id": 50
}

http://127.0.0.1:5000/switch/1 -->
{
  "road_name": "Gantry 1",
  "road_number": "0001",
  "scope": "switch",
  "state": "thru",
  "tmcc_id": 1
}

I'm trying out the flask-restful package. It only took about 40 lines of code to generate this output. If others have suggestions for restful-api frameworks that they've used in Python, I'd love to hear about them.

What's interesting is that I've basically written a wrapper around the PyTrain CLI, I can literally issue any  dmcc/legacy command I want to. I will probably write simplified POST handlers for speed, direction, bell, horn, and any other common engine commands, but then I'll add an endpoint that takes a CLI command, giving me access to the entire Lionel command set.

  -- Dave

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

@kwilliamm posted:

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

Thanks. I was reading about FastAPI last night and was planning to give it a try this morning. I liked that it helps generate the API docs.

  -- Dave

@kwilliamm, well that was, er, fast:

I really like the way it documents the API automatically and provides you the ability to try out each endpoint.

I'm still thinking about syntax and what other features to expose, but with about a half a day's effort, I have a working Restful API. Thank you for the suggestion!

  -- Dave