Gaining

I am in the same exact boat as you except I'm a little heavier even after losing 30 pounds. I recommend lactaid if you haven't tried it before as it can help with some of the unpleasantness of dairy. Also peanut butter helps and that doesn't have any dairy.

Not really a solution per se, but if you want a high fat and calorie snack that might help you out since you are lactose intolerant... Nuts! Peanuts, cashews, pistachios, any kind of nut really.
They are really high in fat and minus cashews it is all good fat too.