Thank you for asking a really good question!
For whether to use 了 (le) in the sentence you mentioned: the action of "going to the gym" started last month, and it is unclear whether it is still persisting, so we don't add 了 (le) unless it has now stopped being the case. Here, the emphasis is on two parts: the time (last month), and the fact that "going to the gym" started. If we were to say "I went to the gym last month," which implies the action has already happened and ended, then we would use 了 (le) and come up with 上个月，我去了健身房。 (Shàng gè yuè，wǒ qùle jiànshēnfáng。).
Whether we can use 过 (guò) is also an excellent question! 去过 (qùguò) is grammatically correct, but in a different situation. To say you have v + 过 (guò) is to put emphasis on that you have had done or experienced something in the past. So if someone were to question you whether your gym membership is a waste, you could say that it's not, because "I did go to the gym last month" or "I had been to the gym last month." And that will be 上个月，我去过健身房。 (Shàng gè yuè，wǒ qùguò jiànshēnfáng。).
We hope to have you answered your question, and look forward to hearing back from you if you have any more of them!