Python code to issue TMCC, Legacy, and PDI Commands

@christie posted:

I was playing with “momentum” in analog controllers over the summer. I found that for it to really look and feel right, you need to be playing calculus games - actively changing the rate of change over time (second derivative? I forget, it’s been an awfully long time…) to ease in and out of acceleration and deceleration. Otherwise it’s just like a sort of delayed speed change effect.  With complete control over the software stack like you are building, I bet you can really do momentum right!

Calculus was an awfully long time ago for me, but you absolutely could build any kind of speed table that you want. I took a look at what the Lionel Cab3 controller does and tried to emulate that. My implementation at this point is all based on linear map, but I guess I was just lazy.

  -- Dave

@christie posted:

I better get going on this before you do everything cool!

I really can’t wait to build my own legacy controller - I just keep getting sidetracked with other projects!!!

The software I've developed implements ALL of the TMCC commands and most of the PDI/LCS commands. you absolutely could!

  -- Dave

Added a "decode" command to PyTrain to convert hex codes seen in the LCS Monitor app (or Wireshark Sniffer) to the corresponding TMCC or LCS Command.

For example:

>> decode D1 28 F8 87 CC 8D DF
0xd128f887cc8ddf --> [PDI Tmcc_Rx [ENGINE 67 ENGINE_LABOR 12 (0xf887cc)] (0x28f887cc)]
>> decode 0xF88703
0xf88703 --> [ENGINE 67 REVERSE_DIRECTION (0xf88703)]

-- Dave

Finally added logging to the code. Log messages can go to both a file and the console at the same time. Logging can also capture command echoing, so you can have a complex transcript of all the TMCC commands happening on your layout, be they from a Cab 2, a Cab 3, or from the PyTrain command line. This really lets you see the commands the Cab 2 and Cab 3 generate when you operate a control; you’d be surprised!

  — Dave

I received my Gantry Crane today and ****ed if the components I’ve wired up to control it did just that! I turned the rotary encoder and the crane cab turned. I pushed up and down on the joystick and the magnet raised and lowered. And I pushed left and right on the joystick and the crane moved left and right!!

I have some tweaking to do, as the code that sends the numeric prefix (<Aux 1> [1, 2, 3]) needs to be synchronized so that state isn’t lost when you operate multiple controls at the same time, but it works!

  — Dave

I've switched gears to working on support for making the system easier to use and more bulletproof. This is involved two changes. The first is I've developed some scripts that allow you to run the software when the raspberry pie boots up. This will be important because when powers turned onto the layout, I want the pies that are powering the control panels to just boot up and be working, like an embedded device.

The second change is to use a discovery service to allow the PyTrain server(s) to register their presence on the local network. This will allow the client applications to not need to fool with IP addresses. They will auto-discover The IP address of the server, and just simply connect to it. Since the Base 3 Doesn't echo out very many of the commands it receives in, I still think it will be advantageous to have an LCS Ser2 connected to the server. That said, I'm investigating if it makes sense to have each client send train commands directly to the Base 3 over WiFi. This should help distribute the load.

I'm working now on actually building the first control panel to operate my gantry crane. I'll send pictures along once there's something to show. By the way, all of the code is checked in.

  -- Dave

Added support to allow engines to be looked up by road number. This makes it easier to address engines, as the need to remember the associated TMCC IDs is eliminated. This is a precursor to adding support for 4 digit TMCC IDs…

I also figured out how to update the Base 3 and Cab 2 in a more efficient way. This support isn’t checked in yet, but will come online this week.

i also am about to fabricate my first controller panel! It will just be a prototype. I’ve switched over to using an I2C matrix keypad, which should give me 19 GPIO pins to wire up to fire commands!

  — Dave

Here's a picture of the completed prototype:

The throttle knob is a $12 rotary encoder intended for CNC machines. It has a nice feel with distinct "clicks" as you rotate. My attempt at soldering leads onto a "soft pot" was a total failure. I hadn't noticed you could buy them with connectors already attached. That Amazon order arrives next week

On the software side, I've added a number of commands intended to make operating a cluster of PyTrain servers (actually, 1 server and many clients). You can now issue a single command that will update the PyTrain software across the cluster, as well as upgrade the Linux OS on all systems with one command from one system. I've also enabled the ability to run a client on the same system as a server. Why, you may ask? My intension is to start my software automatically when each box boots. This way, when you turn the power on to your layout (and control panels), all of the integrated Pi systems will just boot up and run the code. But with this method, where all the systems are "headless", there is no operator console available to issue commands! This is where the need to run a client on the server comes in. When it comes time to upgrade the OS or my code, you cans onto one of the systems (preferably the primary server), run the software in "client" mode, then issue the appropriate command (update to update my software, upgrade to upgrade the raspberry pi OS as well as my software), or reboot, because sometimes...

I also added an ability to automatically find the Base 3 on a local network. I used Wireshark to try to figure out how the Cab 3 finds the Base, and, as best as I can tell, it simply tries all IP addresses on the local network (255 - 3, or 252), stopping once it succeeds. I am using Python's "multiprocessor support" to do this, so on my 24 core development Mac, it finds the base in just a few seconds! On a Pi 5, with just 4 cores, it can take 10 - 20 seconds to find the Base, but as it only needs to do this once on start-up, and only on the "server" instance (clients can get the base address from the server), this may be sufficient. I will probably hard-wire the base address into my server start-up script, but I wanted the option...

  -- Dave

For 30 years we've been stuck with the control surfaces that Lionel gave us or those developed/licensed by a small number of competitors and developers. Now, we can create basically any interface our imagination can think up, or anything our layouts demand. This is amazing work, Dave.

This is awesome work.  Well done, Dave.

Now that they're officially out of the handheld controller game, I hope Lionel takes notice and really open things up with Legacy to allow 3rd parties to bring their own control hardware, and even actively support integration with Base3 and a lot of the other stuff they make.  This is a great way to get technically minded folks into the hobby, too.  I don't really see how it could hurt them unless they ever change their mind about making hardware remotes, but even then, most people would probably just buy theirs again anyway.

Just added an "when_pressed_or_held" handler that can issue one of 2 commands, based on how long a button is held. This is useful for handling things like startup and shutdown, where there are really two flavors of the command; one that does the action without dialog, and another that does it with dialog. The threshold is settable, but .3 seconds seems to work nicely...

  -- Dave

Just added more realistic syncing of Diesel RPM levels as you increase/decrease throttle via the PyTrain methods. I have to do the same for Engine Labor as well (TBD).

Does anyone know offhand if Engine Labor (effects) applies to Steam and/or Electric, as it does to Diesel? I'm only applying RPM effects to Diesel engines (as does the Cab 3).

  -- Dave

I do believe they apply to steam, you use the same controls on the CAB1 to increase the labor for steam.  I can't say for sure how that translates into the Legacy commands, but given you use the same techniques for steam or diesel to increase labor, I would suspect it would be the same commands.

@gunrunnerjohn posted:

I do believe they apply to steam, you use the same controls on the CAB1 to increase the labor for steam.  I can't say for sure how that translates into the Legacy commands, but given you use the same techniques for steam or diesel to increase labor, I would suspect it would be the same commands.

Thanks John.

Yes, the commands are the same, I just didn't know if the concept applied to steam. I would have thought the concept of RPM applied to steam too, but if you pull up a steam engine on the Cab 3, the RPM bars to the left of the throttle disappear...

It's not RPM on steam, it's labor.  Obviously, unless the steam engine changes speed suddenly or has wheel-slip, the RPM won't change.  However, if you pull the train brake down and then run the speed up past where the train brake is limiting the speed, the labor increases.  By the same token, if you have momentum set at high and spin the throttle up quickly, you'll also get increased labor until the actual locomotive speed catches up with the speed setting.

The concept of "working harder" applies, but with a diesel, it's the prime mover ramping up in speed to generate more power, with steam it's the amount of steam fed to the cylinders to produce more power to the wheels.  Same idea, just a totally different way of accomplishing it.

@cdswindell posted:

Thanks John.

Yes, the commands are the same, I just didn't know if the concept applied to steam. I would have thought the concept of RPM applied to steam too, but if you pull up a steam engine on the Cab 3, the RPM bars to the left of the throttle disappear...

I think you’re taking about two different things. RMP definitely is only in diesels, but going into the speed menu on the Cab-2 and now you have labor adjustments labeled EFX. This shows up on diesel, steam, and I’m pretty sure it’s also on electrics.

I haven’t looked for that menu on the Cab3 App. I’ll get back to the base 3 tonight and see if I can find it.

Finally completed support for using I2C digital I/O extender boards to add more buttons. The code creates a new type of gpiozero “Button” that has all of the same behavior. This means that you can use the “when_pressed” and “when_held” call backs, which was central to my design.

The code was developed for the CQ Robots Ocean 16 port board, but should work with any board with a MCP23017 chip. The nice thing about this chip is it provides interrupt support so you do not have to poll the board looking for button presses.

I don’t think I’m going to take the time now to add support to drive LEDs, although I think doing so would be pretty simple.

  — Dave

@cdswindell posted:

Finally completed support for using I2C digital I/O extender boards to add more buttons. The code creates a new type of gpiozero “Button” that has all of the same behavior. This means that you can use the “when_pressed” and “when_held” call backs, which was central to my design.

The code was developed for the CQ Robots Ocean 16 port board, but should work with any board with a MCP23017 chip. The nice thing about this chip is it provides interrupt support so you do not have to poll the board looking for button presses.

I don’t think I’m going to take the time now to add support to drive LEDs, although I think doing so would be pretty simple.

  — Dave

It was pretty simple; support for I2C LEDs is now in place

Code has been modified to use gpio extender ports or regular pi ports in all GpioHandler methods (route, switch, controller, etc).

  — Dave

I finally tackled packaging and now have the code up on Pypi’s test site. And, as I had hoped, you will soon be able to get up and running by typing 3 commands! I’ve also developed a tool to configure a Raspberry Pi to run the code and remove extraneous services and packages not needed to run PyTrain. This helps reduce the system load on the small Pi devices, although it runs fine even without the optimizations.

This wraps up the development efforts for PyTrain. The major outstanding task is completing four digit addressing support. I’m hoping to have a four digit addressable engine next month, at which point I will complete that support.

Next up, I've decided to implement a Restful API. This will allow developers working in other languages to control trains. And it is a necessary component for my next major feature; Alexa Control! With the ability to query ngine names and road numbers, and access to the complete command library, this should allow me to develop a pretty full-featured Alexa skill to run my layout.

  -- Dave

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

I can't think of a use case but I'd certainly think of a use with Home Assistant integration.

Actually I did think of use cases, e.g. blow whistle when motion detected near train room door
Stop train when motion/presence no longer detected
X number of bell hits dims train room lights

And finally, zigbee devices can (such as a simple button) can activate things on the train. Easy to set up interactions for kids/adults alike

https://www.home-assistant.io/

@kwilliamm posted:

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

@kwilliamm

As Homer might say, "doh!"  I had not checked pypi Until right now, and yes, pytrain is already taken. Kind of makes sense, given all the machine learning interest out there. I will add some kind of suffix that makes sense, at least to me. Although suggestions will be welcome!

one of the reasons I took on packaging is that I wanted to keep the restful API and Alexa integration in a separate project and not have the overhead of it in the main py train. I definitely agree. They should be separate projects and they will be. I'm still debating if I wanna combine the restful API and Alexa into one or even break those out separately. Do you have an opinion on that?

I have no experience with DCS. I'm assuming this is what MTA uses? I think I remember hearing somewhere that MTH was very proprietary and actually went after someone that tried to reverse engineer their protocol. I don't have any of their equipment, nor any of their engines, so at least for the time being, i'm not gonna take that on.

Thanks!

  -- Dave

@cdswindell posted:

I have no experience with DCS. I'm assuming this is what MTA uses? I think I remember hearing somewhere that MTH was very proprietary and actually went after someone that tried to reverse engineer their protocol. I don't have any of their equipment, nor any of their engines, so at least for the time being, i'm not gonna take that on.

Mark DiVecchio has done a lot of work with DCS and has decoded their protocol very well.  Check out his RTC page: http://www.silogic.com/trains/RTC_Running.html

This is a pretty impressive program and he does it all via the WiFi.  He has extensive information on the actual protocol to talk to the DCS as well.

@Rollsington posted:

I can't think of a use case but I'd certainly think of a use with Home Assistant integration.

Actually I did think of use cases, e.g. blow whistle when motion detected near train room door
Stop train when motion/presence no longer detected
X number of bell hits dims train room lights

And finally, zigbee devices can (such as a simple button) can activate things on the train. Easy to set up interactions for kids/adults alike

https://www.home-assistant.io/

The way Google and Alexa smart home devices work is by making use of apis of the client applications to control them. Thus, I need to build this into PyTrain in order to build the Alexa integration.

  — Dave

@kwilliamm posted:

Hey Dave. I'm extremely excited to play with this. What an amazing resource you've provided.

Love the idea of a rest API. A couple unsolicited thoughts (please take with a grain of salt, I know you are under no obligation to share your time any more than you already have)

• I wonder though if integration code like Alexa etc should live outside the package. Just thinking about how to keep it easier to maintain over the long term. There might be many other integrations as well in the future that could leverage the HTTP API.
• Given the thoughtfulness already put into this library, do you imagine a DCS control implementation could also live here? It seems what you are creating here could become THE go to library for Lionel & MTH control.
• Naming: I noticed there are already some pypi packages using pylegacy and pytrain etc. ocontrol? Whatever the case, would be cool for "seo" and marketing purposes if the repo and package name were the same.

P.S. have you seen this bluetooth control python library for lionchief?

I think I’ll name the package pytrainrr or pytrain-rr.

  — Dave

I've started to work on better documentation. Lots more to do here, bot have a look at the first few pages  on  Github. You will also see the schematic of the first real control panel I'm building.

  -- Dave

Added support to auto-restart clients when the server is restarted.

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

@gunrunnerjohn posted:

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

Any company needs to prioritize initiatives based on many factors. One would need to accept a physical remote that replicates the Cab-2 and now app didn't make the cut.

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

@David_NJ posted:

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

Nothing like unrealistic expectations!

Just added the ability to "replay" a set of commands at start-up. With this feature, and tools already built into the OS of the Raspberry Pi and MacOS (and probably Windows as well) you can:

• Activate an accessory at a specific time of day
• Start and stop trains based on some external event
• Turn on your layout lights at sundown and off at sunrise (if you pulled an all nighter )
• Blow the horns of all active trains at noon
• and many other...

The PyTrain startup option to do this is -replay followed by the name of a text file containing the commands to run at startup. The command can be run from a PyTrain client or server.

  -- Dave

@gunrunnerjohn posted:

Nothing like unrealistic expectations!

John, can you come up with the pcb gerber files for a remote or is that someone else's bailiwick?  I've spoken with someone about doing the layering of the code for a touch screen... we are sort of at Cab-1L on steroids or Cab-2 lite to keep it reasonable.

@Dennis-LaRock posted:

John, can you come up with the pcb gerber files for a remote or is that someone else's bailiwick?  I've spoken with someone about doing the layering of the code for a touch screen... we are sort of at Cab-1L on steroids or Cab-2 lite to keep it reasonable.

The bigger issue is coming up with a design for the hardware, actually developing the PCB design is just part of the picture.  For something like a TMCC/Legacy remote, the hardware design and debug would be a lot more effort than the actual PCB design.

@gunrunnerjohn posted:

The bigger issue is coming up with a design for the hardware, actually developing the PCB design is just part of the picture.  For something like a TMCC/Legacy remote, the hardware design and debug would be a lot more effort than the actual PCB design.

What if you used my software as the base? Then you are really just hardware debugging the physical switches, pots, and display...

@cdswindell posted:

What if you used my software as the base? Then you are really just hardware debugging the physical switches, pots, and display...

That would be a giant step forward.  The first steps would be coming up with a form factor.  I certainly think it should be a handheld remote.  We'd have to agree on the actual display and exact configuration of the switches so that the hardware matched the finished software.  Clearly, that can be done, just pointing out there's coordination involved between software and hardware development required for such a project.

We don't want to hijack Dave's thread here.... is it worth a thread to try and winnow it out or is there to much to chew on?

Well, I think Dave would be a key player in any such design since he wrote the code!

@Dennis-LaRock posted:

We don't want to hijack Dave's thread here.... is it worth a thread to try and winnow it out or is there to much to chew on?

My own 2 cents is there is a fair amount of discussion to be had as to what features a handheld must control as well as what features would be nice to have. At the end of the day, features will just send a bunch of bytes out the wifi interface to the Base 3 (or a PyTrain server so we have the Ser2 feedback). PyTrain knows how to send out all of the published TMCC and Legacy commands, so it can handle any features the discussion arrives at. I think it would be better to have these discussions in a separate thread. I promise to follow it .

Just to stir the pot, here are some things to ponder:

• Is it acceptable to just support TMCC and Legacy engines (PyTrain doesn't do Bluetooth yet, and many LionChief engines also support Legacy)
• Must it control turnouts? (requires a dynamic LCD and GUI)
• Must it fire routes? (requires a dynamic LCD and GUI)
• Must it operate accessories? (requires a dynamic LCD and GUI)
• Must it be able to build/configure trains? (Gui work and I need to add a few more commands)
• Must it allow editing of road names and numbers?
• Must it allow the addition and configuration of new components (Set address, set command type, etc)

One of the issues with dynamic displays required to implement the more involved GUIs is they may drive the design to something that needs more than a simple 4 line, 20 character LED Matrix display. Inexpensive OLED displays do exist for the Pi, and some even support touch screens. However, this will mean more software. The OLED displays take a few more pins than the LCD screens do (I drive mine off the 2 pin I2C bus on the Pi), which means fewer pins for switches. I do have I2C support for GPIO extenders up and running, but now you need to make room and have power in the handheld for these boards too.

Fun stuff to talk about, and certainly a nice distraction.

  -- Dave

@gunrunnerjohn posted:

That would be a giant step forward.  The first steps would be coming up with a form factor.  I certainly think it should be a handheld remote.  We'd have to agree on the actual display and exact configuration of the switches so that the hardware matched the finished software.  Clearly, that can be done, just pointing out there's coordination involved between software and hardware development required for such a project.

John, agreed!

@David_NJ posted:

Any company needs to prioritize initiatives based on many factors. One would need to accept a physical remote that replicates the Cab-2 and now app didn't make the cut.

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

How nice of you to be willing to buy one 🤣

@gunrunnerjohn posted:

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

@cdswindell posted:

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

I guess we just have to change that.

I have a very limited understanding of the programing and electronic issues in this project.  However, Dave you are doing a phenomenal job in the development of a remote device to replace or upgrade the cab2 handheld.  I hope this thread may wake up Lionel to support the development of a handheld unit or aid in the progress so far.  Although I have my doubts. 

Marty

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

@JD2035RR posted:

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

I just checked in a new program, make_service.py, that installs PyTrain as a service on Raspberry Pis. What does this mean and why should you care? One of my goals is to make the Pi as much as "an appliance" as I can. When you turn on the main power to your layout, it is important to me that all of the embedded Pis in the control panels boot up and start running the PyTrain software with no fuss or further configuration. Lionel's Base 2 and Base 3 do the same thing. Even though they are really special purpose computers with custom programming, when you flip on the main power, they just turn on and start working (most of the time 😉)

The way you make this happen on a computer, like the Pi, is to install your program as a system service. This commit adds the program make_service.py , which creates a system service to run PyTrain as either a server or a client. Once this is done, PyTrain will just fire up every time you power on your control panel(s). The client instances will even wait for your PyTrain server to launch, and the server will wait for the Base 3 to boot.

My goal is to have it all just work!

  -- Dave  

While on the subject of system management (I do realize this is a train site , I've also added a number of commands to PyTrain to simplify the management of a PyTrain environment, AKA a bunch of control panels. Those commands are:

• update: checks for PyTrain updates and installs them as necessary
• shutdown: does a clean shutdown of all Pis. On my master control panel, I plan to have a big button that I will press and hold for a few seconds before I plan to power down the layout. Although Pis were designed to just have their power cords pulled out, no piece of electronics really likes this...
• restart: restarts PyTrain on all clients and servers if something is wonky (which means I didn't do my job 😢)
• reboot: reboots all of the Pis and restarts all services
• upgrade: updates the Linux operating system on each Pi, then does a PyTrain update.

All of these commands automatically relaunch PyTrain whether it is running as a system service (preferred) or from a shell window (for developers and layout operators that want to use the PyTrain CLI to control their layout).

  -- Dave

@cdswindell posted:

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

@JD2035RR posted:

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

Huh, cool! I was unfamiliar with BlueRail. They must have essentially enabled Bluetooth game controller control of their app.

Although I've never done it, you can pair a BlueTooth controller to an iPhone and use it as an "assistive touch" device to control the cursor and operate an iPhone. Someone should order one of these things and see if it can operate the Cab 3!

https://support.apple.com/en-us/111775

  -- Dave

I've started work on the Restful API for PyTrain.

http://127.0.0.1/engine/72 -->

{
  "current_speed": 0,
  "road_name": "Union Pacific GP35",
  "road_number": "0742",
  "tmcc_id": 72
}

I was happy to see I could rather quickly adapt the PyTrain main program to support being called as a provider by another process. From here on out, I want to minimize changes to the main PyTrain GitHub project and do the api-specific stuff in a new GitHub project: PyTrainApi. The one exception is I will probably add the ability to get state info in JSON form...

  -- Dave

Some quick screen shots. I have to think a bit about syntax...

http://127.0.0.1:5000/train/20 -->
{
  "control": "Legacy",
  "direction": null,
  "labor": 12,
  "max_speed": null,
  "momentum": 0,
  "road_name": "Delaware & Hudson",
  "road_number": "0020",
  "rpm": 0,
  "scope": "train",
  "smoke": null,
  "speed": 0,
  "speed_limit": null,
  "tmcc_id": 20,
  "train_brake": 0,
  "year": null
}

http://127.0.0.1:5000/acc/15 -->
{
  "block": "on",
  "road_name": "Upper Main North Power District",
  "road_number": "0015",
  "scope": "power_district",
  "tmcc_id": 15
}

http://127.0.0.1:5000/sensor_track/50
{
  "last_loco_lr": 255,
  "last_loco_rl": 255,
  "road_name": null,
  "road_number": null,
  "scope": "irda",
  "sequence": "slow_speed_normal_speed",
  "tmcc_id": 50
}

http://127.0.0.1:5000/acc/50
{
  "road_name": "Upper Main West",
  "road_number": "0050",
  "scope": "sensor_track",
  "tmcc_id": 50
}

http://127.0.0.1:5000/switch/1 -->
{
  "road_name": "Gantry 1",
  "road_number": "0001",
  "scope": "switch",
  "state": "thru",
  "tmcc_id": 1
}

I'm trying out the flask-restful package. It only took about 40 lines of code to generate this output. If others have suggestions for restful-api frameworks that they've used in Python, I'd love to hear about them.

What's interesting is that I've basically written a wrapper around the PyTrain CLI, I can literally issue any  dmcc/legacy command I want to. I will probably write simplified POST handlers for speed, direction, bell, horn, and any other common engine commands, but then I'll add an endpoint that takes a CLI command, giving me access to the entire Lionel command set.

  -- Dave

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

@kwilliamm posted:

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

Thanks. I was reading about FastAPI last night and was planning to give it a try this morning. I liked that it helps generate the API docs.

  -- Dave

@kwilliamm, well that was, er, fast:

I really like the way it documents the API automatically and provides you the ability to try out each endpoint.

I'm still thinking about syntax and what other features to expose, but with about a half a day's effort, I have a working Restful API. Thank you for the suggestion!

  -- Dave

I’ve finished the first pass at the API. It’s checked into GitHub as PyTrainApi. Documentation is nonexistent yet, but if you want to run it, set up a separate virtual environment, download the code, install the requirements with pip, and type the command:

fastapi run src/pytrain_api/fastapi_ex.py

Note you need to use run and not dev, as the PyTrain log file causes the fastapi server to keep restarting (I have to figure this out, one day). With the server running, hit up:

Http://<server ip address>:8000/

and you will get the doc pages displayed.

If anyone does give this a try, please let me know, I'm definitely looking for feedback on the API syntax.

Now on to the Alexa integration...

@kwilliamm posted:

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

@kwilliamm, I really want to thank you for pointing me towards FastAPI. I finished implementing the functional endpoints I wanted to build around 6:00 this evening. After dinner, I decided I'd see what was involved with adding some kind of password security to what I did. Two hours later, I have OAuth password protection with real JWT tokens! I didn't even know what some of that meant 2 hours ago 😂. Their online documentation/tutorials are some of the best I've seen, and I've been in the business since the 1970s...

PyCharm just added support for uv. For now, I'm going to stick to what I have (GitHub workflow pushing new versions to pypi) but I will have a look once I've further down the road with the Alexa integration.

Oh, and as I use Netgear routers and have a DynamicDNS entry already defined pointing to where I live in Boston, I've even used the API remotely to trigger actions on my little switcher engine in my office (a Chesapeake & Ohio NW2). It was very cool to see it move down the tracks and blow its horn via a remote web session!

Thanks again!!

  -- Dave

P.S. for others that may read this post, this framework let me develop a complete REST API to my app with about 3 days of effort, and this includes the time to learn the tool and rework PyTrain to support the API.

Working with the Alexa code is no where near as much fun as working with FastAPI 😢. I am finding ways to do it, but my goal was to develop and publish an Alexa skill others could use as well. There is a lot of configuration needed to set up an HTTPS proxy server in front of the PyTrainApi. Maybe another approach will become clear as I proceed, but this one may not be for the feint of heart 😵‍💫.

@cdswindell posted:

Huh, cool! I was unfamiliar with BlueRail. They must have essentially enabled Bluetooth game controller control of their app.

Although I've never done it, you can pair a BlueTooth controller to an iPhone and use it as an "assistive touch" device to control the cursor and operate an iPhone. Someone should order one of these things and see if it can operate the Cab 3!

https://support.apple.com/en-us/111775

  -- Dave

I wonder if one of these would work.  I could see using the iphone proper for most things, then use this widget to control your current train and getting a few key buttons for speed, dir, horn, whistle, halt, etc...

I have my engines starting up and shutting down via Alexa!! It's a start...

  -- Dave

@cdswindell posted:

I have my engines starting up and shutting down via Alexa!! It's a start...

  -- Dave

Alexa Skill can now:

• Start up and shut down engines with or without dialog
• Set engine speed, with and without dialogs
• Reset engines
• Blow whistle/horn
• Ring bell
• Change direction
• Stop immediate
• Open front/rear couplers
• Send Halt command
• Throw switches
• Fire routes

Now onto setting speed, etc...

  -- Dave

The PyTrain API project is up and live on PyPi. You can download it here. The project includes the single executable, pytrain_api, which is all you need to start serving commands. The package includes a Uvicon web server, which launches the PyTrain API on port 8000.

You will need an API Key to make calls. I need to figure out how I want to do this long term (if others download and use the API), as you really don't want someone taking over your layout! For now, use the API key:

e54d4431-5dab-474e-b71a-0db1fcb9e659

This will change, but it will work for now.

Once you install the package (from a virtual environment, of course), hit the URL:

http://<your host ip>:8000/pytrain/v1

to get a list of the supported APIs, You can even try them out from this web page! To do so, hit the green [Authorize] button at the top right of the page and enter the key from above. All of this functionality is provided automagically by FastAPI; really cool stuff!

  -- Dave

The API and PyTrain DB commands now report consist information (what components make up the train).

@cdswindell posted:

The PyTrain API project is up and live on PyPi. You can download it here. The project includes the single executable, pytrain_api, which is all you need to start serving commands. The package includes a Uvicon web server, which launches the PyTrain API on port 8000.

You will need an API Key to make calls. I need to figure out how I want to do this long term (if others download and use the API), as you really don't want someone taking over your layout! For now, use the API key:

e54d4431-5dab-474e-b71a-0db1fcb9e659

This will change, but it will work for now.

Once you install the package (from a virtual environment, of course), hit the URL:

http://<your host ip>:8000/pytrain/v1

to get a list of the supported APIs, You can even try them out from this web page! To do so, hit the green [Authorize] button at the top right of the page and enter the key from above. All of this functionality is provided automagically by FastAPI; really cool stuff!

  -- Dave

Dave,
Not sure if this would help
import secrets
import os

def generate_api_key(length=32):
"""Generates a secure API key."""
return secrets.token_hex(length)

def save_api_key(api_key, filename=".env"): you can change the filename here
"""Saves the API key to a file, handling existing files."""
if os.path.exists(filename):
with open(filename, "a") as f:
f.write(f"API_KEY={api_key}\n")
else:
with open(filename, "w") as f:
f.write(f"API_KEY={api_key}\n")

if __name__ == "__main__":
new_api_key = generate_api_key()
print("Generated API Key:", new_api_key)
save_api_key(new_api_key)
print("API key saved to .env file.") and here

The above code will auto gen an api key results of what it does below

biqu@SBM4P-CB1:~$ python3 api_gen.py
Generated API Key: d51887d41f748d25bd8cc409372cffb7b64683451b81f2e950997e324ca22cc6
API key saved to .env file. will list your filename

@cdswindell posted:

The API and PyTrain DB commands now report consist information (what components make up the train).

I'm guessing this means when there's a lash-up that's been assigned a "TR" number in the system? All of a sudden, I had visions of putting IR sensors on every piece of rolling stock and reporting on entire consists and the locomotives they are being pulled by... 😂

@J.Dooley posted:

I'm guessing this means when there's a lash-up that's been assigned a "TR" number in the system? All of a sudden, I had visions of putting IR sensors on every piece of rolling stock and reporting on entire consists and the locomotives they are being pulled by... 😂

LOL, yes, multi-engine consists defined with the Cab 2/Cab 3 TR button. The Base 3 makes the information available as to what components make up the consist, along with which direction they're facing, and if dialog and horn sounds are masked or not. Adding support to decode/report this was on my "to do" list.

  -- Dave

I’ve added a couple of new features, including:

• can now send PDI commands directly to LCS ASC2 and BPC2 (via Base 3 only). The ASC2 command lets you explicitly turn on and off a relay, rather than depending on the timing of how fast multiple acc aux2 commands come in. Also better at reporting accessory state.
• pytrain clients can now send pdi commands (proxies thru a pytrain server to the Base 3)
• added -repeat option to support sending accessory commands (to simulate holding down aux1 key)
• added -duration option, which allows you to do things like "blow the horn for 10 seconds", and "rotate the men inside the control tower for 20 seconds". This could be simulated by using the "-repeat" command, but the -duration option is much easier to use.
• added relative speed command control for accessories (TMCC-style)
• sensor track state now remembers the last engine/train that traversed as well as the direction (left to right, right to left). I’m toying with adding some form of automated block control and this will be useful to know…
• fixed bugs...

  -- Dave

@cdswindell posted:

I’ve added a couple of new features, including:

• can now send PDI commands directly to LCS ASC2 and BPC2 (via Base 3 only). The ASC2 command lets you explicitly turn on and off a relay, rather than depending on the timing of how fast multiple acc aux2 commands come in. Also better at reporting accessory state.
• pytrain clients can now send pdi commands (proxies thru a pytrain server to the Base 3)
• added -repeat option to support sending accessory commands (to simulate holding down aux1 key)
• added -duration option, which allows you to do things like "blow the horn for 10 seconds", and "rotate the men inside the control tower for 20 seconds". This could be simulated by using the "-repeat" command, but the -duration option is much easier to use.
• added relative speed command control for accessories (TMCC-style)
• sensor track state now remembers the last engine/train that traversed as well as the direction (left to right, right to left). I’m toying with adding some form of automated block control and this will be useful to know…
• fixed bugs...

  -- Dave

I've added these features to the PyTrain REST Api.

  -- Dave

After writing complicated code to use rotary encoders and potentiometers to control the Lionel Crane Car, Gantry Crane, and Smoke Fluid Loader, I've realized its much simpler to just use a momentary contact switch-based joystick. These devices have just 2 states per position; on and off. They can directly map to Cab-1 RELATIVE SPEED or the Boost/Brake command. Using the gpiozero library, you can code it up so that if the joystick is held "on", the command can be fired repeatedly. I'll send along a video shortly, but this code is much simpler and works much more reliably!! I still allow the use of a rotary encoder to rotate the crane housing, but it also just sends a relative speed command with no acceleration factor. It doesn't look like a cab rotates any faster or slower on the Cab-3, so there isn't a need for complexity.

  -- Dave

Here is a video of my TMCC Crane Car being controlled by a 2 axis joystick. And another of the TMCC Smoke Fluid Loader being controlled by a rotary encoder. Since making this video, I modified the code to speed up the boom movement by sending commands every 0.020 seconds (the Cab 3 sends commands every 0.010 second). These sorts of changes are very easy to do in software. I may even make it a parameter so users could control the timing themselves without recoding.

  -- Dave

P.S. I just added a parameter to these classes to allow you to set this time without recoding

Well that is simply amazing to me! Excellent work!

It's also pretty cool that they're both connected to the same Pi. What length of wire do you estimate you can put between your controllers and the Pi?

@BillYo414 posted:

Well that is simply amazing to me! Excellent work!

It's also pretty cool that they're both connected to the same Pi. What length of wire do you estimate you can put between your controllers and the Pi?

Do you mean the length of wire connecting the joystick to the pi? Probably something measured in yards, given it is simply carrying a 3.3v signal to ground. My goal is to keep that distance short because I will mount the control panels close to where the layout components will live and use a separate Pi in each panel (I just bought 10 new Pi 5 computer modules!!)

  — Dave