My library has a method of designating books mystery/suspense (a red skull on endcap of book) or Romance (three hearts on endcap).

Often times I'm confused as to why an author would choose one designation over the other.  I have read many books that are more mystery/suspense than romance that are designated as romance and books designated as mystery/suspense that are more romance.   What do you think the line between the two designations is?  I'm thinking it might be sex.  What do you think.

Yeah, it's probably the sex.

I don't think the author chooses. I think the publisher or editor determines what the cover itself says. However. your library may or may not go by what the book is listed as officially. They might just put it where people would expect to find it.

I'd suggest that the key difference is the h/h factor. Is there a continual building of / focus on their relationship throughout, with key conflict being about them as a couple, with the ending focused on them as a couple? If so, you've got romantic suspense. If one of the two is simply incidental to the mystery/suspense plot, but the two do not establish any kind of ongoing bond by the end of the book, I'd categorize that as a mystery/suspense/thriller story.

Colleen