Alexa Smart Speaker TMCC Control (Updated 8/31/20)

The voice command vocabulary is an interesting topic.  My first thoughts are to reuse the names of button press sequences on the Cabs which you already use for the switches.  When you implement synonyms and aliases then lots of interesting things can happen including groups of commands (i.e. "Hiawatha 3 depart station" which maps to "tower talk cleared for departure, start rolling at speed step 2 with bell for 10 seconds, then accelerate to yard speed").

The IR sensor interface is sorely needed since Lionel is lagging on providing triggers for their sensor tracks.  Keep us updated.

Routes would be cool. I assume that could be done as well. 

@MartyE posted:

Routes would be cool. I assume that could be done as well. 

Absolutely, infact since you just mentioned it I will add the feature and post a demo clip on here.  Routes make a good next step after switch basic switch throws. 

Ryan, the second largest O guage manufacturer is in need of a buyer, Are you in???

Nice work.....again!

This is very impressive and it could open up the hobby to new groups of people. Awesome work!

@Ryaninspiron posted:

Next on my project list is to finish recreating the discontinued IR sensor car from scratch. (progress is happening)

I like the sound of this one.  I had given some thought to seeing if that could be done.

@gunrunnerjohn posted:

I like the sound of this one.  I had given some thought to seeing if that could be done.

The answer to the IR is yes but it will probably have to be a bit more manual to setup than just hitting the TX IR Button on the Cab2. I figure I will probably use a cheap little OLED screen and buttons to have you directly set the name and engine ID from the boxcar. or maybe have that little screen connected to the boxcar with a ribbon cable.  (let me know if that sounds acceptable to you). If not it could take me twice as long to get the remote based programming up. Not to mention the requirement of at least an R2LC or R4LC to receive the signal adding to the cost of each unit. (many times more expensive than a screen). On a personal note there I would love a circuit to decode the legacy signal without those boards.

@Tracker John posted:

The IR sensor interface is sorely needed since Lionel is lagging on providing triggers for their sensor tracks.  Keep us updated.

I certainly will do, not too far off now. 

I am about halfway through decoding that protocol with the hardest part in front of me. For example I just about to be at the point where I can hard code the RoadNumber and Name to my module but I am still a little bit off from letting the operator change that on their own. (and a long ways off from setting it with the cab 2 and not a screen as I mentioned above.)

Sounds like some combination of app based display and voice command would be the most usable overall implementation. I will definitely stay tuned to this intriguing effort whose time , I am happy to see, has come.

I reviewed the IR Sensor track PDI comms interface specification in the LCS Partner docs.  Nothing about the IR interface data exchange between the engine and track.  The sensor track passes a lot engine-provided information to the Command Base in an ACTION_DATA packet when the engine passes over the track.  That data is generated by the Legacy firmware in the engine.   Question is what is really necessary for a sensor car to pass to the track to get the sensor track to generate that packet?  Engine ID and engine type?  

To quote a line of some Star Trek episode, "This is like nothing we've seen before."

@Roger L. posted:

To quote a line of some Star Trek episode, "This is like nothing we've seen before."

Thanks, and I cant wait to get my hands on that new Star Trek LionChief set! To give you a preview of what I have in mind for that, you need to know that the Alexa speaker can also be set to activate on the word COMPUTER instead of Alexa. I can also set my Alexa code to respond to any activation phrase, so instead of "the yardmaster" it can be "Mr.Sulu" You, can guess what I have in mind for this, It involves Alexa sending commands directly to the LC loco through my LC Gateway board. No more spoilers, but I have a fun video in mind once that set comes out.

I have now added the route and track power controller access to the voice skill for my LC Gateway Project.

The route portion was pretty easy, however the Track Power Controller took a bit of extra time to develop for a few reasons. First I thought I had absolute speed steps on the TPC but apparently I was wrong and it is relative only. So I read the manual and found out the quickest way to get the power on with this is to have my code switch it to command mode (which lets the boost button act like an on switch). The next delay was realizing that track commands use a different command structure. Eventually I worked it all out and got it working. Next up should probably be engine commands.

Still working on the different ways to phrase these requests. Funny enough that is much harder than writing the actual code. 

@Ryaninspiron posted:

The answer to the IR is yes but it will probably have to be a bit more manual to setup than just hitting the TX IR Button on the Cab2. I figure I will probably use a cheap little OLED screen and buttons to have you directly set the name and engine ID from the boxcar. or maybe have that little screen connected to the boxcar with a ribbon cable.  (let me know if that sounds acceptable to you). If not it could take me twice as long to get the remote based programming up. Not to mention the requirement of at least an R2LC or R4LC to receive the signal adding to the cost of each unit. (many times more expensive than a screen). On a personal note there I would love a circuit to decode the legacy signal without those boards.

I think the plug-in touch-screen would be fine.  That way you could build more than one boxcar and not have to replicate the screen for each one.

I thought many times about the RF reception of the TMCC/Legacy signal direct.  However, I still don't know if you'd run into a Lionel patent if you tried to sell something like that.  I would hate to put all the time into it getting it working and find out I couldn't really use it.

@MartyE posted:

Routes would be cool. I assume that could be done as well. 

Done  See my previous post above for the video.

I can do these things a little faster with cab11 but you are certainly creating the future for our train layouts. Very exiting.

@Tracker John posted:

I reviewed the IR Sensor track PDI comms interface specification in the LCS Partner docs.  Nothing about the IR interface data exchange between the engine and track.  The sensor track passes a lot engine-provided information to the Command Base in an ACTION_DATA packet when the engine passes over the track.  That data is generated by the Legacy firmware in the engine.   Question is what is really necessary for a sensor car to pass to the track to get the sensor track to generate that packet?  Engine ID and engine type?  

I feel like I should probably get a dedicated forum topic for this setup so we can can discuss in detail. but yeah, I am used to working on Lionel's undocumented protocols after I reverse engineered LionChief for my gateway board. Unfortunately the data sent via IR is very different from the TMCC or LionChief protocols.

Do you have access to the non public partner docs? I never did see any explanation anywhere about the PDI packet structure. I have the impression that it is just higher speed 9 bit serial data right? It's possible I could have a much easier time implementing this if I saw what data was included in these ACTION_DATA packets you mentioned. My belief so far is that the sensor car is simply sending out all the data same data as an engine would. You assign it the same TMCC ID as The engine you want to copy the information from and the legacy base uploads it via the track signals to the sensor car. The sensor car over IR appears to send an Eng ID, Type, Sound System, Name, and RoadNumber.

@romiller49 posted:

I can do these things a little faster with cab11 but you are certainly creating the future for our train layouts. Very exiting.

True, and I am certainly not trying to make something faster than the remote, Cab1L or Cab2 can both do these things faster. Just the same way as flipping a light switch is faster than using your voice, even though I automate all the lights in my house with Alexa to save getting up to do it . But that's not where I am trying to go with this, it's certainly not about being lazy like with my light switches.

I am honestly doing this to get a more immersive run session for the solo engineer. I feel it's a bit more lonely to run trains in the basement alone in silence than it would be to at least feel like your talking to a coworker on your simulated rail line, Rail-Sounds are neat but they leave you to be a quiet listening observer at the end of the day. Many people like talking (generalization) and honestly I am already getting more in the zone/enjoyment of of this by asking my not so imaginary yardmaster to throw a switch for me than silently pressing some buttons with nothing but beeps for feedback on the remote. More of a simulated radio system, a solution for the engineer who can't convince their friends to play yardmaster and man the switches for them all the time (Que the waterworks and sad violin music).

On the note of interactivity I am about to add a process where you check into a virtual yard office. where you tell the yardmaster what engine your running, then when you tell the engineer your going to normal speed, the engine the yardmaster knows your in will just start moving to normal speed. This is what I mean by immersive. It lets you focus on actual prototypical conversions(or at least I hope they can be prototypical of some era at least.) Maybe better yet it can help you imagine your the engineer.

When you sign into the yard office your selected engine will start up, when you sign out it then it will shutdown. You can also say tell the yardmaster I'm changing to engine six and carry on from that engine's perspective. Immersive. I also plan on having the App confirm your requests by calling back to you with your selected company and roadnumber: "Copy That UP 9599", "Copy That Erie 482".

This voice portion of my LC Gateway Project is me letting my imagination turn into real conversations with a smart speaker using my knowledge of programming. I'm glad you raised that point since it let me get to the real point of all this.

@BOB WALKER posted:

Sounds like some combination of app based display and voice command would be the most usable overall implementation. I will definitely stay tuned to this intriguing effort whose time , I am happy to see, has come.

As for a display to all of this Amazon does make a version of the Echo with a screen on it called the Echo Show. I could look into getting one of those and writing the code to display a cheat sheet/status screen on that. Or maybe I can use an Amazon Fire TV stick to let you use any TV with an HDMI port as a screen. A phone app should also be pretty doable though, I already know how to program Androids. I'm pretty sure Apple apps are a whole different ball game than what I'm used to. Maybe I can just leverage the existing Alexa app to keep this contained in one ecosystem though.

@RickO posted:

Ryan, the second largest O guage manufacturer is in need of a buyer, Are you in???

Nice work.....again!

Are you proving the financial capital required for such a move? if so I'm in.  It would be cool if the community could gather and become the backers of a co-op purchase of that company though. You get a stake based on how much you invested.

@BillYo414 posted:

This is very impressive and it could open up the hobby to new groups of people. Awesome work!

Thank you thank you, I am only going to keep going on this for now. I wan't a fully immersive experience as I just went over in my big post a few rows up.

@BOB WALKER posted:

This chart was in an article of mine published 3 years ago. It's exciting that young talented designers are taking an active interest in our hobby and introducing the latest technologies.

I love how you showed a picture of an Amazon Echo smart speaker .

Glad I could help fulfill the future you described 3 years ago. I actually have one of those tall echo's too. I can make a video including it just for you.

You actually just made me think of what comes after voice and I have figured it out. Unfortunately the tech I have in mind is a bit expensive right now but I just thought of a way to go way beyond voice control in terms of immersiveness. Whenever I reveal it it will feel like a good leap into the future considering that voice is now present day technology. Just need to figure out the code to program it in the meantime.

A chart predicting voice control was in an article of mine published 3 years ago. It's exciting that young talented designers are taking an interest in our hobby and introducing the latest technologies.

@BOB WALKER posted:

A chart predicting voice control was in an article of mine published 3 years ago. It's exciting that young talented designers are taking an interest in our hobby and introducing the latest technologies.

Thank you, I saw you were having issues with that doc attachment so I posted it in picture form with my reply above. Thanks again.

You are quite welcome. Here's the page from OGRR magazine that included the chart.

@BOB WALKER posted:

You are quite welcome. Here's the page from OGRR magazine that included the chart.

That's a great article, and I agree with it fully. 

This is very cool!

Can the user reprogram the association of a particular TMCC command, or does the configuration have to be pre-programmed? It would be great if the user was free to associate any TMCC or Legacy command with a voice sequence.

And what hardware do you need to connect to the Command Base serial?

Nice job Ryan. Way above my skill level, but interesting and interested so see where you take this!

Charlie

Wow.  Great work.

@gunrunnerjohn posted:

I think the plug-in touch-screen would be fine.  That way you could build more than one boxcar and not have to replicate the screen for each one.

I thought many times about the RF reception of the TMCC/Legacy signal direct.  However, I still don't know if you'd run into a Lionel patent if you tried to sell something like that.  I would hate to put all the time into it getting it working and find out I couldn't really use it.

Exactly, one screen, many boxcars, simple ribbon cable and socket to connect and use to set it up, then disconnect and move on. You certainly won't need to do it too often and it keeps the cost even lower since the all additional cars can share one setup screen.

True,  I was forgetting about that. Your definitely right.

@Professor Chaos posted:

This is very cool!

Can the user reprogram the association of a particular TMCC command, or does the configuration have to be pre-programmed? It would be great if the user was free to associate any TMCC or Legacy command with a voice sequence.

And what hardware do you need to connect to the Command Base serial?

I realized some way to setup shortcuts like that will be needed. In the terms of the "Alexa Skills" they call that persistence. It will require me to add a support database to the skill to store shortcut names and action lists. I can certainly get it done with a voice based menu system but that will take a little time.

It would certainly be easy for me to just add some pre built shortcuts like one called depart station. That would set off a standard series of departure actions with whatever engine ID you mention. (Just like the prepackaged actions in the LCS Sensor Tracks). But the real solution is what I mentioned first. A cloud based database to store shortcuts, That way the command still works a week after you set it up.

The only hardware needed (aside from an Amazon Echo) is a RS232 to TTL serial adapter hooked up to a microcontroller with a WiFi adapter. All the magic is in the really complex software that has given me some real dark circles under my eyes over the last few days.(which itself is built from code I spent weeks on a few years ago.) The bulk of the voice code is all stored inside Amazon's Cloud Servers. I basically had to become a mini Amazon server expert to get this all figured out.

@Charlie posted:

Nice job Ryan. Way above my skill level, but interesting and interested so see where you take this!

Charlie

Thank you, Trust me it's a pretty big time investment to get it all working but the results are pretty fun and worth it in my book. The results literally speak .

I have quite a few more things in mind with this. This forum and my YouTube channel will be the place to get the latest.

@DaveGG posted:

Wow.  Great work.

Thanks, I appreciate it.

And it took me all day yesterday just to connect 3 "primitives" (a cone and 2 cylinders, for a primitive boiler shell, baby steps!) together in FreeCad.

I have an Alexa in the train room, but all it does is tell me time and play RR sounds when I ask it.

I'm still waiting for them to develop the Star Trek medical wand.  Wave it over you and instantly know what the ailment is and the cure.

Good job

@Bob Delbridge posted:

And it took me all day yesterday just to connect 3 "primitives" (a cone and 2 cylinders, for a primitive boiler shell, baby steps!) together in FreeCad.

I have an Alexa in the train room, but all it does is tell me time and play RR sounds when I ask it.

I'm still waiting for them to develop the Star Trek medical wand.  Wave it over you and instantly know what the ailment is and the cure.

Good job

I hear you on that FreeCad. Some fine details can be a bit of effort to figure out. One time, I designed and 3D printed something on there only to have it fall apart on me because I never merged the shapes. They were only touching on my screen but that wasn't good enough it turns out. These days I use Fusion 360 which for now at least is free for home/hobbyists.

SciFi is on the way. Wait till SpaceX starts flying their star ship with a crew of 50!

Thank you

@Ryaninspiron posted:

True, and I am certainly not trying to make something faster than the remote, Cab1L or Cab2 can both do these things faster.

I am honestly doing this to get a more immersive run session for the solo engineer. I feel it's a bit more lonely to run trains in the basement alone in silence than it would be to at least feel like your talking to a coworker on your simulated rail line, Rail-Sounds are neat but they leave you to be a quiet listening observer at the end of the day. Many people like talking (generalization) and honestly I am already getting more in the zone/enjoyment of of this by asking my not so imaginary yardmaster to throw a switch for me than silently pressing some buttons with nothing but beeps for feedback on the remote. More of a simulated radio system, a solution for the engineer who can't convince their friends to play yardmaster and man the switches for them all the time (Que the waterworks and sad violin music).

...

This voice portion of my LC Gateway Project is me letting my imagination turn into real conversations with a smart speaker using my knowledge of programming. I'm glad you raised that point since it let me get to the real point of all this.

This project and your work are really a neat idea. It could also help those with disabilities enjoy running their trains more.

@BOB WALKER, nice article.  Now the question is, what's next down the road?    Bluetooth control of trains via apps on phones and iPads is here, controlling your layout (trains, accessories, switches) via the iPad is here thanks to LCS and we are experimenting with voice control.  How much further down the road before we get to Augmented Reality being integrated into the hobby?   Think about it, you hold your phone/iPad etc up to a loco or freight car and all of the details about it pop up (Fuel,  water, load, destination etc). This could make the hobby much more interactive.  What about battery power?  I'm sure it's coming at some point. 

what's next?

@Casey_Jones posted:

@BOB WALKER, nice article.  Now the question is, what's next down the road?    Bluetooth control of trains via apps on phones and iPads is here, controlling your layout (trains, accessories, switches) via the iPad is here thanks to LCS and we are experimenting with voice control.  How much further down the road before we get to Augmented Reality being integrated into the hobby?   Think about it, you hold your phone/iPad etc up to a loco or freight car and all of the details about it pop up (Fuel,  water, load, destination etc). This could make the hobby much more interactive.  What about battery power?  I'm sure it's coming at some point. 

what's next?

Hey, stop reading my mind , you just mentioned my next idea after voice control. That next generation tech will be a bit harder for me to get working right away but I have some ideas on how I can get started. I have the hardware to test it (Hololens) but the software tools are still a lot rougher than they are for mature systems like voice control. After my IR boxcar hits the next milestone where I release a video on it. I plan on developing and testing out the next next future technology, so to answer your question now that I'm going to set my mind on it, it wont be as long as you think. I'm not ready to go public with the details on what it will look like yet but I have some fun ideas I want to implement on that next generation technology.

@rplst8 posted:

This project and your work are really a neat idea. It could also help those with disabilities enjoy running their trains more.

Thank you, and your certainly right. I have a few different alternate input devices in mind but figured starting with voice would have pretty wide appeal. Plus it's a feature I have personally wanted for a little while now.

Aside from the high interest (which includes me) , there is always the intellectual property issue of "prior art". Having previously worked in patent licensing, I looked into this and was surprised to find that there are current active patents for voice control of trains. I also found that earlier relevant cited patents date back to 1994. Any future commercialization of this concept will ultimately have to deal with this situation.