Skip to main content

Just checked code to support subscribing to system events, and have refactored the command echoing program echo.py to use it. The mechanism uses a simple subscribe/publish mechanism to allow "listeners" to subscribe to "events" they are interested in. Events can be broad, like any changes to any engine, or specific; changes to a specific engine (or switch or accessory or train). There is also a "broadcast" channel you can subscribe to which receives all TMCC events the system reads from the LCS Ser2. This is how echo.py now receives its updates.

I threw together a quick and dirty command syntax to query device state and added it into pytrain.py. I've only wired in switches and accessories, but will work on engine and train state next. I have to figure out how to serve this information up to client pytrains. Once I do that, I'll be able to change speeds more gradually by querying an engine's current speed and ramping up/down to the new requested speed. I also want to wire listeners in to the GPIO support so indicated light states can be kept in sync.

I've asked the Lionel folks how to communicate directly with the Base 3 over wifi. This will be a far better mechanism to get the current state of switches, accessories, and power districts, but as far as I know, engine and train states are most likely perceived by interpreting the TMCC command stream (over wifi or serial).

  -- Dave

My $15 Raspberry Pi Zero W (2017 vintage) arrived yesterday. The short answer is it works with my software and can be used to control trains! The longer answer is it is v e r y  v e r y s l o.  w.

It takes very little power, with is a plus, but it is pretty slow to boot up. I'll post some pictures of it running my Lionel Strasburg switcher once my LCS Ser2 arrives (I've switched locales to Boston for a few months; I miss my layout in NH already.

  -- Dave

Attachments

Images (1)
  • mceclip0
@BlueFeather posted:

Dave, that's a $15, seven year old board.  :-)  You can spend a little more and speed things up fairly easily if that's the bottleneck.

LOL, no,  just wanted to see if it works. IMHO, it boots too slow to be useable in a control panel. You want the panels to be up and operational as soon as the Base 3 is booted.

Because of its small size, the Pi Zero would have been a good choice to build right into a panel. I have a Pi Zero 2 on order at the whopping price of $24, so there may still be hope!

but frankly, that's something a little bigger than the size of a stick of chewing gum can be a fully functioning computer with a GUI, TCP/IP, etc., is still pretty amazing!

  -- Dave

Well, it looks like you're on the trail of something faster.

As someone that's worked with software all my life, I am still fascinated by the array of hardware we now have available and its capability. Here's some timing, using the elapsed time it takes to run my test suite:

Mac M2 Ultra (2023):          1.36 sec
Raspberry Pi 5 (2024): 1.44 sec
Raspberry Pi 4B (2018): 3.11 sec
Raspberry Pi Zero W 2 (2023): 4.31 sec
Raspberry Pi Zero W (2017): 24.27 sec


It's making me wonder if my big, honking M2 Ultra was worth the price

  -- Dave

Last edited by cdswindell

Fairly major update. The code that parses the byte stream from the LCS Ser2 is paying off in spades. I've could echo received commands for a couple of weeks now. This means that no matter how the TMCC command was generated, via a Cab2, the Cab 3, another third-party vendor, or my Python code, I can capture it via the output stream from the LCS Ser2 and echo it to a console.

One of the issues with building control panels is keeping system state in sync. For example, if I have a light on a control panel that reflects if a power district is on or off, I want that light to reflect the correct state no matter how that state is changed. So, if a power district is on, and the corresponding led is lit, it should turn off if:

  • I manually turn off the district via a cab2
  • I manually turn off the district via a cab3
  • It is turned off via a 3rd party app
  • A TMCC Halt command is issued

For this to work correctly, you need:

  • A way to maintain state on the PyTrain server
  • A way to keep that state in sync on all PyTrain clients (the ones powering the panels)
  • An understanding of the dependencies that exist between Lionel commands

This last item is particularly interesting. It's obvious that a power district led should turn off if it sees a "turn power district off" command (Accessory Aux2, in my case), but what about that Halt command? And when you do an engine reset, (the (R) button on the Cab2/Cab3) it sets the engine speed to zero, resets direction to forward, turns the bell off, sets RPM to zero, sets engine labor to 12, and performs an immediate startup, and I may still be missing some; phew!).

With this morning's checkin, the following is available:

  • A state management system has been implemented that observes the command stream from the LCS SER2 and determines device states from it,
  • Commands received on the server (the one with the LCS Ser2 connection) are now propagated to all clients; they in turn, using the same code, keep their own local state caches in sync,
  • I've implemented a command listener system that listens for specific commands (scoped to device type, specific device, device command, and device command data) that calls callback methods in response, and
  • I've started developing a dependency mechanism that understands the relationship between commands (e.g., Halt command should disable power districts)

Devices are engines, trains, switches, and accessories. There is still a lot more work to be done, but with these components, the system is now aware of events happening elsewhere on the layout and from other control mechanisms, and can maintain device state accordingly.

My next step is to wire these dependency-aware event listeners into the code to control LEDS so their state always reflects reality.

I am hoping that one day soon I will learn how to query LCS devices via wifi, but for now, you can do a lot with an LCS Ser2.

  -- Dave

@cdswindell posted:

One of the issues with building control panels is keeping system state in sync. For example, if I have a light on a control panel that reflects if a power district is on or off, I want that light to reflect the correct state no matter how that state is changed. So, if a power district is on, and the corresponding led is lit, it should turn off if:

  • I manually turn off the district via a cab2
  • I manually turn off the district via a cab3
  • It is turned off via a 3rd party app
  • A TMCC Halt command is issued

No offense Dave, but the simple way to do this is to monitor the power at the actual power district and control your panel light with that.  No mistakes, and you can't get a false indication unless a bulb burns out.

However, a TMCC halt command, IMO, probably shouldn't turn the light off.  It doesn't remove power from the track, it just stops all current TMCC/Legacy activity.  However, you can immediately spin the throttle and be on the move again, so in that scenario, I feel the power district light should remain on.

No offense Dave, but the simple way to do this is to monitor the power at the actual power district and control your panel light with that.  No mistakes, and you can't get a false indication unless a bulb burns out.

However, a TMCC halt command, IMO, probably shouldn't turn the light off.  It doesn't remove power from the track, it just stops all current TMCC/Legacy activity.  However, you can immediately spin the throttle and be on the move again, so in that scenario, I feel the power district light should remain on.

None taken, but with my BP2s configured to accept accessory commands, when the TMCC1 Halt command is given, those districts all turn off. Power isn't restored until you reenable that power district (at least on my layout).

I used power district as an example. You encounter the same situation when you reset an individual engine (Numeric 0). The direction is set back to "FWD", speed is set to 0, the bell is turned off, etc.

  -- Dave

State management is now in place and working well, both on the PyTrain server as well as clients. Engine speed, direction, and startup/shutdown status is tracked, and maintained correctly when changed via a Cab 3 as well as impacting commands (engine reset and halt commands).

My next plan is to use the engine speed state to smoothly ramp up or down speed changes.

Stay tuned.

  — Dave

I'm getting pretty close to putting together my first control panel that uses the PyTrain code. I've added several methods to the GpioHandler class that model Lionel LCS modules and accessories, which makes it easier to build out physical devices and associate pushbuttons and switches to Raspberry Pi pins.

For example, here is the code to support a Command/Control Smoke Fluid Loader:

GpioHandler.smoke_fluid_loader(12,
boom_pin_1=12,
boom_pin_2=16,
dispense_pin=21,
lights_on_pin=26,
lights_off_pin=19)


This is all the code a user would need to write to operate the Smoke Fluid loader with Accessory ID #12. Boom rotation is done via a rotary encoder that is connected to Pi pins 12 and 16. Smoke fluid droplets are dispensed via a pushbutton connected to pin 21. And you can turn the unit's lights on and off via a toggle switch connected to pins 26 and 19.

Of course, you could use which ever pins you want...

I've defined the following helper methods:

GpioHandler.route

GpioHandler.switch

GpioHandler.power_district # LCS BP2 in accessory mode

GpioHandler.accessory # LCS ASC2

GpioHandler.culvert_loader # Command & Control

GpioHandler.smoke_fluid_loader # Command & Control

GpioHandler.gantry_crane # Command & Control



These are all items I have on my layout and that I want to build panels for. Other devices (including the pre-command/control variants of those listed above) are easy enough to piece together.

All of these devices are state-aware, that is, if an operation is started via a control panel, it can be completed via a Cab 2, Cab 3, or other 3rd-party software. Indicator lights on the panels will reflect actual state, regardless of how it got there.

My first panel will control 1 switch, 2 power districts, and a gantry crane. I will send pictures as I make progress. What I am most happy about is that each panel will only need one wire; the power wire for the Raspberry Pi itself. All control operations and state management will be done via the PyTrain software, WiFi, and a single Pi connected to an LCS SER2.

  -- Dave

p.s. The $25 Raspberry Pi Zero W 2 looks like a real winner for this application. It is about the size of a piece of chewing gum and can be powered by a small USB power adapter, like they used to send out with your iPhones. I will build the Pi directly into the panel itself, so there will be short wire runs from the physical switches and status LEDs to the Pi's GPIO pins.

I am learning how to interact with the Base 3 over WiFi. I still don't know enough to say if I can dispense with the LCS Ser2 altogether. One interesting observation is that commands sent and received appear to be the ASCII representations of the byte sequences and not the byte sequences themselves. For example, the "keep alive" sequence one needs to send to the Base to keep the TCP/IP connection open is D1 29 27 DF (thanks to a new friend who sent me that info in a DM). I assumed I would be sending 0xd12927df, but instead, I have to send "D12927DF"!! This was a bit of a surprise and makes me a little worried the mechanism I'm using to connect is a debugging backdoor that may be slammed shut one day...

I've also modified my client code so that it requests a complete update from the server when started. This will allow all clients (deployed in separate control panels) to be up to date when booted up, should the server receive state before clients boot.

Back to it...

  -- Dave

@cdswindell posted:

I am learning how to interact with the Base 3 over WiFi. I still don't know enough to say if I can dispense with the LCS Ser2 altogether. One interesting observation is that commands sent and received appear to be the ASCII representations of the byte sequences and not the byte sequences themselves. For example, the "keep alive" sequence one needs to send to the Base to keep the TCP/IP connection open is D1 29 27 DF (thanks to a new friend who sent me that info in a DM). I assumed I would be sending 0xd12927df, but instead, I have to send "D12927DF"!! This was a bit of a surprise and makes me a little worried the mechanism I'm using to connect is a debugging backdoor that may be slammed shut one day...

I've also modified my client code so that it requests a complete update from the server when started. This will allow all clients (deployed in separate control panels) to be up to date when booted up, should the server receive state before clients boot.

Back to it...

  -- Dave

Originally when Lionel released the Legacy code, they made a point of saying you had to use the SER2 to access the full Legacy command set.  I always figured they had done something different in the internal communications that would make it more difficult to use the "raw" data.  Of course, the app has to communicate with the BASE3 over the network, but there might be some sort of "key" that gets changed periodically with releases that will break the communications.

@cdswindell posted:

State management is now in place and working well, both on the PyTrain server as well as clients. Engine speed, direction, and startup/shutdown status is tracked, and maintained correctly when changed via a Cab 3 as well as impacting commands (engine reset and halt commands).

My next plan is to use the engine speed state to smoothly ramp up or down speed changes.

Stay tuned.

  — Dave

That's pretty cool!  Are you keeping some sort of state table for everything?  If so, I'm guessing it is dynamic in nature (record creation and destruction) so as to minimize memory usage.

George

Last edited by G3750
@G3750 posted:

That's pretty cool!  Are you keeping some sort of state table for everything?  If so, I'm guessing it is dynamic in nature (record creation and destruction) so as to minimize memory usage.

George

Yes, I'm keeping a state table for the devices discovered by sniffing the TMCC command flow fro the LCS Ser2. The info is fairly minimal, to reduce the memory footprint. For example, for switches, just the thru/diverged info. For Accessories, a triplet for Aux on/off, Aux 1 on/off, and Aux2 on/off. For engines and trains, for now, I'm only listening for speed, direction, and if the engine has been "started" or "shutdown". I could maintain more info, such as momentum, brake level, smoke, etc, but I'm not sure I'll need that for what I'm doing (but it could be added).

I never really delete any state records, as on a typical layout, you are not removing switches or accessories. I could time out engine/train state records, but they really don't take too much memory.  On my Pi Zero W 2, which I think only has about 512MB of ram, the program only uses about 20mb.

  -- Dave

@cdswindell posted:

Yes, I'm keeping a state table for the devices discovered by sniffing the TMCC command flow fro the LCS Ser2. The info is fairly minimal, to reduce the memory footprint. For example, for switches, just the thru/diverged info. For Accessories, a triplet for Aux on/off, Aux 1 on/off, and Aux2 on/off. For engines and trains, for now, I'm only listening for speed, direction, and if the engine has been "started" or "shutdown". I could maintain more info, such as momentum, brake level, smoke, etc, but I'm not sure I'll need that for what I'm doing (but it could be added).

I never really delete any state records, as on a typical layout, you are not removing switches or accessories. I could time out engine/train state records, but they really don't take too much memory.  On my Pi Zero W 2, which I think only has about 512MB of ram, the program only uses about 20mb.

  -- Dave

Makes sense.  If you limit the amount of information kept per engine and only keep state for detected, i.e. started locomotives, you'll be fine.  Statically accommodating comprehensive records for hundreds of locomotives might eventually be a problem, but you are going about this logically and efficiently.  And you're doing this with a Raspberry Pi, not an Arduino.

George

I'm now able to send TMCC commands directly to the Base3 via wifi (not via the LCS Ser2). Although this is good (), it is tempered by not all TMCC commands are echoed out from the base. It may be possible to query the base for detailed engine state info (is the bell ringing?; is the engine started up?), it may be a lot easier to read the state info as the stream of TMCC commands available via the Ser2.

I may focus for the time being on reading from the Base 3 only the state information I'm most interested in for managing control panel state. This includes switch positions, accessory state, and power district state. But I now have in place the some of the architecture I'll need to perform bidirectional communication with a Base 3.

  -- Dave

I ran into an interesting situation today while coding up my PDI packet parser for the ASC2.  In TMCC land (via the LCS Ser2), accessory commands focus on the Aux 1 and Aux 2 buttons on the Cab2 controller. This makes it very easy to understand which button was pressed. That said, it is Not at all clear, whether buttons are being used to control a momentary action device, such as the little man that goes round and round inside a control tower, or an on/off toggle device like the interior light in that control tower.

With the ASC2, you are presented with very clear on and off states. The Aux1 button sends a series of “on” packets for as long as you keep the button pressed, and sends a single “off” packet once you take your finger off the button. The Aux2 button functions completely as a toggle. The first time you press it, and “on” packet is sent. With the second press, an “off” packet is sent. This is great, however, you have no way to tell which button was pressed!

I want to use the PDI data available on the Base 3 to initialize my system. This mismatch between TMCC and PDI lands makes this challenging.

Any thoughts?

Thanks,

  — Dave

I’m now reading switch state from the PDI events generated by the STM2 and the ASC2, as well as power district events from the BPC2. I still need to add code to query initial states when the system powers up, but I have most of the code in place to do this.

I am taken by how very different the PDI command structure is from TMCC. There is really no overlap. PDI devices all share/support a set of “common” commands (get, set, receive), and actions (config, status, info, reset, identify, clear errors, and firmware) as well as a few device-specific commands and actions. Most of the common commands focus on what I would call "system administration"; reporting comm errors on the PDI bus, getting firmware revisions, etc. These have really nothing to do with running trains or operating accessories,  And, as I mentioned previously, PDI land doesn’t receive any notifications at all about many of the events that occur in TMCC land, meaning I will need my Ser2 when I roll out my first control panel.

  -- Dave

I’ve added support for Python 3.10, 3.11, and 3.12, as well as have started using Tox to run my test suites. This week, I’ll add support to send commands directly to the Base 3, removing the requirement for an LCS Ser2. However, you will still need the Ser2 if you want to issue the extended TMCC commands (like to trigger dialog), and if you want to monitor the TMCC command stream that is made bailable via the Ser2 (not all TMCC commands are broadcast from the Base3).

I am also reading up on how to package the code and put it up on PyPi. At that point, it will be much easier to install and use the main executable, PyTrain!

  -- Dave

I can now read the Engine and Accessory rosters from the Base 3. It will be easy enough to do the same for switches and routes as well. Between this and the code I added earlier to query LCS devices t system startup, I have all the pieces I need to keep system state in sync.

Reading data from the Base 3 also lets me get the road names nd numbers of everything, but I don’t think this will be of too much use in fabricating control panels, at least the ones I plan to build.

The Raspberry Pi’s continue to be able to handle all of the work, and the memory footprint is still pretty minimal.

The next big piece left is packaging the code so I can upload it to PyPi. This will make it much easier for end users to grab the code without having to set up a development environment.

  — Dave

I am now updating the Base 3 database when changes are made to an engine/train's speed, momentum, diesel run level (rpm), engine labor (efx), smoke level, and train brake via TMCC commands. All changes are reflected back from the Base 3. Only speed and momentum changes are correctly reflected on the Cab 3. I've made @Dave Olson aware.

I'm not sure if the Base 3 maintains the fwd/rev direction state of a train or engine (relative to motion, not the direction an engine is facing in a consist). Would be nice to sync this state as well, if possible.

So, to recap progress (and I may repost this separately), here is what's working:

  • All TMCC-style commands can be sent via Ser2 and/or Base 3 except for consist creation and modification (See V1.21 of the Legacy Command Protocol document)
  • All 3 byte Legacy/TMCC2 commands can be sent via Ser2 and/or Base 3 except for consist creation and modification (Also in V1.21 of the Legacy Command Protocol document)
  • Almost all multibyte Legacy/TMCC2 commands have been implemented. They can be sent  via Ser2 and/or Base 3 (multibyte commands include smoke levels, dialog controls, advanced lighting controls, etc, see V1.21 of the Legacy Command Protocol document)
  • Ability to define a sequenced command (maybe I should call this a component command) that allows you to issue multiple commands with specified time delays and repeats, as needed. For example, I've developed ones to perform the grade crossing whistle blows that happen when you use an IR track, and that do a sequenced speed increase that includes engine and tower dialog. These mimic the "official" railroad speed buttons on the Cab 2 and Cab3, but include the tower and engineer dialogs issuing the speed-up (or slow down) dialog, the change speed command, then the engineer acknowledge dialog.
  • All TMCC and Legacy commands broadcast from the Ser2 and echo'd to the console.
  • All PDI commands broadcast from the Base 3 can be echoed to the console
  • All broadcast TMCC, Legacy, and PDI commands can be "subscribed to" and callback functions can be invoked when specific actions occur (this feature is used to support state management as well as console echoing)
  • Switch, Accessory, and some Engine/Train state are maintained.
  • State is initialized at startup by querying LCS modules and the Base 3 itself
  • Client/Server mode is supported where the code on one Raspberry Pi can service multiple connected clients. Clients can send and receive commands and state updates as if they were the server module. Only the server needs connection to an LCS Ser2 and/or a Base 3.
  • TMCC state changes are pushed to the Base 3 (and Cab2/3 controllers) automatically
  • A command line interface is provided that allows all TMCC and Legacy trains, engines, switches, accessories, and routes to be operated and monitored. This same program is used to perform TMCC/Legacy/PDI command echoing.
  • Most implemented commands can be issued via the command line interface.
  • Support has been added for the LCS BPC2, STM2, Ser2, and ASC2 modules.
  • Software support exists to issue all of the common LCS  PDI commands (Firmware, Status, Config, Info, Clear Errors, Reset, and Identify). It is a TODO to make these commands available via the command line interface.
  • Helper methods are provided to simplify the association of buttons, LEDs, toggle switches,  joysticks, rotary encoders, and pots to TMCC and Legacy commands and dynamic state. This means you can associate the commands to throw a track switch to buttons on a Raspberry Pi, and have an indicator light that correctly reflects the switch state, even if thrown another way (e.g., the black Lionel controller, your Cab2, derail prevention feature, LCS STM2, etc). This was the raison d'être for the project
  • Higher-level methods exist to control Lionel Culvert Loader/Unloaders, Gantry Crain (some work left to finish here), Smoke Fluid Loader, Generic Route, Generic Switch, Generic Accessory, Power District, and Engine.
  • Ability to define buttons, lights, components, etc. in a startup file that is loaded when the command line program runs; this is where you will "define" the physical components on your control panel and associate them with the TMCC/Legacy operations to perform when the controls are manipulated.
  • All code is in Python3 and runs and has been tested on Python 3.10, 3.11, and 3.12 (the most recent version supported by the Pis is 3.11) and is up on Github today.

Remaining work:

  • Package the code to enable downloading from PyPi. This will allow folks that don't want to download source code to install the executable that makes all this work.
  • Documentation (sigh)
  • More tests
  • Implement logging (remove print statements)
  • Modify sequenced speed commands to look at engine state, so if you give a speed change command, the engine gracefully ramps up to or down to the new speed)
  • Add generic support for rotary encoder
  • Complete Dialog Control commands
  • Build a control panel



  -- Dave

Last edited by cdswindell

Just checked in the rest of the engineer/tower dialog effects on page 13 of the Lionel protocol spec. These are some fun things, like the engineer speaking the current speed, fuel, and water level, and the tower to engineer and engineer to tower arriving, arrived, departing, and departed dialogs. Using randomization and setting up event watchers on the IR sensor tracks would allow users to trigger random dialogs as your train motors along the rails.

  -- Dave

The things I'll do to avoid taking on something I have trepidations about (packaging).

I've implemented support for the seven common PDI commands for the LCS devices (FIRMWARE, STATUS, INFO, CONFIG, RESET, IDENTIFY, and CLEAR_ERRORS). These can be issued from the server pytrain instance (the one with the Base3 connection). The commands let you display information about your LCS devices (modes, voltage, firmware, errors), turn on and off the little red Ident LED, clear out error counts, and reset the module (I have no idea what this does; sounds dangerous!).

I've also returned to the house where my layout is at and tweaked support for the STM2, BPC2, an Sensor Track modules, and I must say, things are working pretty nicely! At power-on, you get a complete list of all engines, trains, switches, and accessories, along with their names, road numbers, and current state.

One thing I need to do is to send a discovery command when a power district is turned on. Switches that are contained in a power district don't report their actual state until power to that track section is turned on. This is a simple addition for me to make.

  -- Dave

@BillYo414 posted:

This is very exciting work! I could only dream of having the time and knowledge to tackle something like this. I'm having a little trouble understanding some of the posts but it gives me a good reason to try out Python. I'm trying to learn C# at the moment for work so I figure learning any code will benefit me.

Happy to answer any questions you have either here or in email!!

  -- Dave

This is starting to smell a lot like @Ryaninspiron's gamification project.

https://ogrforum.com/topic/int...rack-based-game-idea

It seems like that may have been stalled in the PDI communication implementation, but I don't see why it couldn't use the wifi connection.  I believe his goal was to have a pure LCS implementation (no serial), and it sounds like sensor track may have been the last piece of that puzzle.  I assume sensor track signals can be received via LCS wifi here right?

This is starting to smell a lot like @Ryaninspiron's gamification project.

https://ogrforum.com/topic/int...rack-based-game-idea

It seems like that may have been stalled in the PDI communication implementation, but I don't see why it couldn't use the wifi connection.  I believe his goal was to have a pure LCS implementation (no serial), and it sounds like sensor track may have been the last piece of that puzzle.  I assume sensor track signals can be received via LCS wifi here right?

Absolutely. In fact, the Irda/Sensor Track packets are only sent out via PDI/WiFi.

Add Reply

Post
×
×
×
×
Link copied to your clipboard.
×
×