Python code to issue TMCC, Legacy, and PDI Commands

@David_NJ posted:

Any company needs to prioritize initiatives based on many factors. One would need to accept a physical remote that replicates the Cab-2 and now app didn't make the cut.

If you believe you can create one that is cost effective, I'm sure there are folks that would buy it. Let us know when we can order one. I'd like it to be under $200 and perform all the functions of the Cab-2 and smartphone apps that go through the Base3 or direct to equipment that supports BT, etc.

How nice of you to be willing to buy one 🤣

@gunrunnerjohn posted:

Dave, all the work you guys are doing with these protocols makes you wonder why Lionel and MTH can't even make a remote.

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

@cdswindell posted:

Well, software and hardware are two different animals, as you know, and my software only handles 2 protocols (TMCC/Legacy and PDI). And I don't have to answer to management, support legacy (as in older) products, or do any user support (yet 🤣)!

I guess we just have to change that.

I have a very limited understanding of the programing and electronic issues in this project.  However, Dave you are doing a phenomenal job in the development of a remote device to replace or upgrade the cab2 handheld.  I hope this thread may wake up Lionel to support the development of a handheld unit or aid in the progress so far.  Although I have my doubts. 

Marty

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

@JD2035RR posted:

What about using a MFI iOS controller (bluetooth video game controller) to send commands to the Raspberry Pi?  It could also hold the phone to show the Cab3 app.  I understand that there may be (a lot of) additional code to incorporate something like this.  However, the gaming controllers are inexpensive, durable, allow a wide range of input types, and can navigate menus and data input rather quickly even without a number pad.  Food for thought for Dave and others in the trenches on this project.  Thanks for making everything public, Dave.  Very fun and exciting stuff.

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

I just checked in a new program, make_service.py, that installs PyTrain as a service on Raspberry Pis. What does this mean and why should you care? One of my goals is to make the Pi as much as "an appliance" as I can. When you turn on the main power to your layout, it is important to me that all of the embedded Pis in the control panels boot up and start running the PyTrain software with no fuss or further configuration. Lionel's Base 2 and Base 3 do the same thing. Even though they are really special purpose computers with custom programming, when you flip on the main power, they just turn on and start working (most of the time 😉)

The way you make this happen on a computer, like the Pi, is to install your program as a system service. This commit adds the program make_service.py , which creates a system service to run PyTrain as either a server or a client. Once this is done, PyTrain will just fire up every time you power on your control panel(s). The client instances will even wait for your PyTrain server to launch, and the server will wait for the Base 3 to boot.

My goal is to have it all just work!

  -- Dave  

While on the subject of system management (I do realize this is a train site , I've also added a number of commands to PyTrain to simplify the management of a PyTrain environment, AKA a bunch of control panels. Those commands are:

• update: checks for PyTrain updates and installs them as necessary
• shutdown: does a clean shutdown of all Pis. On my master control panel, I plan to have a big button that I will press and hold for a few seconds before I plan to power down the layout. Although Pis were designed to just have their power cords pulled out, no piece of electronics really likes this...
• restart: restarts PyTrain on all clients and servers if something is wonky (which means I didn't do my job 😢)
• reboot: reboots all of the Pis and restarts all services
• upgrade: updates the Linux operating system on each Pi, then does a PyTrain update.

All of these commands automatically relaunch PyTrain whether it is running as a system service (preferred) or from a shell window (for developers and layout operators that want to use the PyTrain CLI to control their layout).

  -- Dave

@cdswindell posted:

I must be missing something here, as I'm not sure what this accomplishes. We could run the Cab 3 app on the iPhone already, right? Are you looking to use the joystick on the controller to control engine speed?

Thanks,

  -- Dave

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

@JD2035RR posted:

The idea is to provide a tactile hand held remote (rather than a touch screen phone) to control engine speed, quilling whistle, bell, boost/brake, start up sequences, etc.  - I'd say everything other than programming engines which would be simpler through the app itself.  The phone would show the Cab3 App with engine data, throttle position, etc.

I understand this is out of scope from what you set out to do, I was just offering a hardware option that might be able to work before having to cobble something together.

There is another train control app called Bluerail (not related to TMCC/Legacy at all). The Bluerail app allows you to control the app directly using a MFI controller.  The Cab3 app to my knowledge doesn't allow MFI/bluetooth control of the app - which is why I was thinking perhaps you could use the bluetooth controller input to the raspberry pi to activate the code you have written for TMCC/Legacy.

Here's a video demonstrating it for Bluerail.

There is https://youtu.be/plinKYtuwrM

Huh, cool! I was unfamiliar with BlueRail. They must have essentially enabled Bluetooth game controller control of their app.

Although I've never done it, you can pair a BlueTooth controller to an iPhone and use it as an "assistive touch" device to control the cursor and operate an iPhone. Someone should order one of these things and see if it can operate the Cab 3!

https://support.apple.com/en-us/111775

  -- Dave

I've started work on the Restful API for PyTrain.

http://127.0.0.1/engine/72 -->

{
  "current_speed": 0,
  "road_name": "Union Pacific GP35",
  "road_number": "0742",
  "tmcc_id": 72
}

I was happy to see I could rather quickly adapt the PyTrain main program to support being called as a provider by another process. From here on out, I want to minimize changes to the main PyTrain GitHub project and do the api-specific stuff in a new GitHub project: PyTrainApi. The one exception is I will probably add the ability to get state info in JSON form...

  -- Dave

Some quick screen shots. I have to think a bit about syntax...

http://127.0.0.1:5000/train/20 -->
{
  "control": "Legacy",
  "direction": null,
  "labor": 12,
  "max_speed": null,
  "momentum": 0,
  "road_name": "Delaware & Hudson",
  "road_number": "0020",
  "rpm": 0,
  "scope": "train",
  "smoke": null,
  "speed": 0,
  "speed_limit": null,
  "tmcc_id": 20,
  "train_brake": 0,
  "year": null
}

http://127.0.0.1:5000/acc/15 -->
{
  "block": "on",
  "road_name": "Upper Main North Power District",
  "road_number": "0015",
  "scope": "power_district",
  "tmcc_id": 15
}

http://127.0.0.1:5000/sensor_track/50
{
  "last_loco_lr": 255,
  "last_loco_rl": 255,
  "road_name": null,
  "road_number": null,
  "scope": "irda",
  "sequence": "slow_speed_normal_speed",
  "tmcc_id": 50
}

http://127.0.0.1:5000/acc/50
{
  "road_name": "Upper Main West",
  "road_number": "0050",
  "scope": "sensor_track",
  "tmcc_id": 50
}

http://127.0.0.1:5000/switch/1 -->
{
  "road_name": "Gantry 1",
  "road_number": "0001",
  "scope": "switch",
  "state": "thru",
  "tmcc_id": 1
}

I'm trying out the flask-restful package. It only took about 40 lines of code to generate this output. If others have suggestions for restful-api frameworks that they've used in Python, I'd love to hear about them.

What's interesting is that I've basically written a wrapper around the PyTrain CLI, I can literally issue any  dmcc/legacy command I want to. I will probably write simplified POST handlers for speed, direction, bell, horn, and any other common engine commands, but then I'll add an endpoint that takes a CLI command, giving me access to the entire Lionel command set.

  -- Dave

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

@kwilliamm posted:

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

Thanks. I was reading about FastAPI last night and was planning to give it a try this morning. I liked that it helps generate the API docs.

  -- Dave

@kwilliamm, well that was, er, fast:

I really like the way it documents the API automatically and provides you the ability to try out each endpoint.

I'm still thinking about syntax and what other features to expose, but with about a half a day's effort, I have a working Restful API. Thank you for the suggestion!

  -- Dave

I’ve finished the first pass at the API. It’s checked into GitHub as PyTrainApi. Documentation is nonexistent yet, but if you want to run it, set up a separate virtual environment, download the code, install the requirements with pip, and type the command:

fastapi run src/pytrain_api/fastapi_ex.py

Note you need to use run and not dev, as the PyTrain log file causes the fastapi server to keep restarting (I have to figure this out, one day). With the server running, hit up:

Http://<server ip address>:8000/

and you will get the doc pages displayed.

If anyone does give this a try, please let me know, I'm definitely looking for feedback on the API syntax.

Now on to the Alexa integration...

@kwilliamm posted:

@cdswindell great idea to separate the API implementation from the main package.

I recommend using the FastAPI framework. And also using pydantic to validate requests.

Separately you might consider using UV as the package manager and as the recommended installation method for users of pytrain.

Using uv can help make sure python version and other dependencies are consistent across installs and upgrades over time.

@kwilliamm, I really want to thank you for pointing me towards FastAPI. I finished implementing the functional endpoints I wanted to build around 6:00 this evening. After dinner, I decided I'd see what was involved with adding some kind of password security to what I did. Two hours later, I have OAuth password protection with real JWT tokens! I didn't even know what some of that meant 2 hours ago 😂. Their online documentation/tutorials are some of the best I've seen, and I've been in the business since the 1970s...

PyCharm just added support for uv. For now, I'm going to stick to what I have (GitHub workflow pushing new versions to pypi) but I will have a look once I've further down the road with the Alexa integration.

Oh, and as I use Netgear routers and have a DynamicDNS entry already defined pointing to where I live in Boston, I've even used the API remotely to trigger actions on my little switcher engine in my office (a Chesapeake & Ohio NW2). It was very cool to see it move down the tracks and blow its horn via a remote web session!

Thanks again!!

  -- Dave

P.S. for others that may read this post, this framework let me develop a complete REST API to my app with about 3 days of effort, and this includes the time to learn the tool and rework PyTrain to support the API.

Working with the Alexa code is no where near as much fun as working with FastAPI 😢. I am finding ways to do it, but my goal was to develop and publish an Alexa skill others could use as well. There is a lot of configuration needed to set up an HTTPS proxy server in front of the PyTrainApi. Maybe another approach will become clear as I proceed, but this one may not be for the feint of heart 😵‍💫.

@cdswindell posted:

Huh, cool! I was unfamiliar with BlueRail. They must have essentially enabled Bluetooth game controller control of their app.

Although I've never done it, you can pair a BlueTooth controller to an iPhone and use it as an "assistive touch" device to control the cursor and operate an iPhone. Someone should order one of these things and see if it can operate the Cab 3!

https://support.apple.com/en-us/111775

  -- Dave

I wonder if one of these would work.  I could see using the iphone proper for most things, then use this widget to control your current train and getting a few key buttons for speed, dir, horn, whistle, halt, etc...

I have my engines starting up and shutting down via Alexa!! It's a start...

  -- Dave