If you're talking about the list that is displayed along the left margin, go to Tool/Settings/Tracks Panel. In the Favorite Libraries options on the right side of the window, you can set the number of slots you want to use for quick reference and the display order. I have mine set to the 5 I use most when helping folks with their designs. To initially set the list, set the number of slots, then select each library you want displayed and lastly change the order to Fixed. You'll still be able to select other libraries, but they won't get added to the Favorites list.
If you're talking about the actual libraries, you'll need to delete the libraries you don't want to see/use. Depending on how you installed SCARM, these are located in a folder something like C:\<user>\Program Files (x86)\SCARM\lib. I'd recommend you create a new folder and name it something like lib-backup. Move the libraries you don't want to the new folder so you can move them back if you need to. When you update SCARM with a new version, these will likely be added back to \lib, so you'll have to go through the process of moving them again.