OK, Controller lights wrong color:
Is it possible that you have a pair of wires swapped at the connection to the switch?
Switch sometimes buzzes and does not move:
Anti-Derail is holding the switch in position, this is caused by the inner outside rail shorting together with the track connecting to it. Note: Inner outside rail is the one forming a small V with the outer rail of the other diverging track.
This can be caused by misalignment or non level alignment of the connecting track to the switch.
Fix - Level the adjacent track with the switch and make sure the rail ends cannot touch. The electrical connections are underneath anyway. Note one rail has no contact, this is the anti-derail rail.
Additional note: I actually create an isolated rail one or 2 joints away on that rail and connect it to the switch. This gives the switch more time to settle in the new position if a train is coming fast. This is also helpful if you are running conventional and track voltage is low. I run my switches on Aux Voltage for this reason and to avoid bulb burnout / melt when running in Command control with long times of max Track voltage.