I've posted a couple photos of paintings, etc on this as well as a video of a moroccan sport, Tbourida, or the Fantasia, where teams , brightly dressed with swords on their backs, hilt down, charge and in unison discharge their muskets as close together as they can. Still happens now.
Here's another video or two:
https://www.youtube.com/watch?v=27PEwSRgvD0
https://www.youtube.com/watch?v=hGhmEoisQEI
https://www.youtube.com/watch?v=4NV1q8OkgAE
https://www.youtube.com/watch?v=6SBSx4BKd1g
Swords clearly shown hilt down on left sides of riders.
It's a lot easier to draw from there as opposed to hilt up. how they do not fall out other than friction, I do not know.