Maybe the distinction is reading level. Lit fi seems to be written at a higher reading level than genre fiction. Again, there are exceptions (e.g. Hemingway), but in general lit fi seem to be more demanding texts to read.*

Then you have some titles or authors which seem to have moved from genre into literary if they stand the test of time and become classics, like Fahrenheit 451 or Childhood's End.

It's all very puzzling. I'm sure there are dissertations that have been written on this very topic LOL!

*Of course, that might just be because the average quality of fiction today is pretty deplorable...