Gaining

Obviously you can live your life out in response to what your family, friends, and lover think, changing nothing relative to your weight . But will a life like that be good enough for you? Can you truly live happily that way?
If you can answer those questions with a yes, then maybe leaving things alone will work for you.
But if, through serious introspection on the subject, you come to a different conclusion, it will be time to take the heat by challenging your family's & friends' point of view, changing partners, and finding someone who will appreciate the more voluptuous you.
There are, after all, excellent fat admiring resources out there today to help you make the right romantic connection.

I feel about the same way, here are my three reasons why I can't really gain now.

1. Scared of health issues to my health
2. Social stigma of people that are around me
3. Family members would look down on me.
4. Its freaking expensive to gain

Each of your arguments against are strong & reasonable. And from your response it sounds like chances are a purposefully significant gain is not in your future - and that's OK.
I do wonder what percentage of people sexually stimulated by their own fat never take the plunge for the same reasons you haven't? I expect a very significant portion keep this particular interest forever in the closet, never seriously considering indulging because of all the stigma's attached to it.