Just for your own personal use? So you don't accidently request from them again ?

I keep a handwritten list by my computer-- thankfully , it's very short! I don't know of any "official" way to do so though

Nope there isn't.  You can put a post in note near your computer.  You could try adding them to your buddy list and then tagging the buddy list as a bad swapper. But since you need their permission to add them, it might not work.  With so many members, chances are you won't get them again anyway.  If they are a habitual bad swapper then most likely PBS will get rid of them.