The verse in Devarim 22:5 prohibits women having “male articles” upon her, but does not specify what those articles are. Targum Onkelos translates this as referring to “keli zayin”, which can mean weapons or armor. In the midrash halacha (Sifri 226:5), Rabbi Eliezer ben Ya’akov derives from this verse that a woman should not wear “keli zayin” and go out to war. Rambam (Avoda Zara 12:10) rules that a woman should not wear armor or anything like it, which could include a weapon. It is not clear how Rambam relates to the connection between wearing armor, bearing a weapon, and going out to war.
Eight hundred years ago Sefer Hasidim (200) teaches that it is permissible for women to go armed and even dressed fully as men if that will protect them from being raped.
In more recent times, both Rav Ovadya Yosef (Yechave Da’at 5:55) and, as you note, Rav Moshe Feinstein (Orach Chayim 4:75:3) have ruled that a woman may arm herself in self defense. Both rabbi’s responses appear to apply to any situation where a woman needs a weapon for self-defense.
Here are more details, and responses to the points you raised:
1. Pistols: Rav Moshe writes that they, and small weapons like them, are not the type of weapons used in war, so Rabbi Eliezer ben Ya’akov’s statement may not apply to them. He takes “keli zayin” to mean military weapon. However, he brings this as only one of a series of reasons to permit a woman to carry weapons in self-defense, so it is not clear that he would permit a woman to carry a pistol if there were no need for one.
2. Location: At the end of his responsum, Rav Moshe makes it clear that the fundamental halachic considerations are (1) that this is not a case of a woman going out to war as Rabbi Eliezer ben Ya’akov specified (a matter which merits its own discussion), and (2) that preserving one’s life trumps other laws. Rav Moshe adds that a woman need not desist from traveling to dangerous places in order to avoid the need to bear a weapon in self defense. We learn from this that his response applies to any situation where there is a genuine risk to a woman, not just when she is already in a dangerous setting. Rav Ovadya speaks about guard duty, which also can be seen as permitting bearing arms when the threat is more indirect.
3. Additional Considerations: Rav Moshe notes, somewhat elliptically, that where there is a problem of self-defense, it is not considered a men’s garment at all, since the need in that case is equal for men and women. Rav Ovadya adds (as we quote in Tzitzit I: Keli Gever) that Bach (Yoreh Deah 182) permits cross-dressing when there is a significant practical reason for it, like protection from the elements. The case of protection from harm as in self defense can also fall under this category, even if a woman is not in a bona fide war zone.
Please let us know if you have any more questions.