Egg white protein powder is alright if you like the taste of raw eggs! You have too add a boat load of carbs/sugar/sugar substitue etc. to get rid of that. It is also more expensive.
Milk contains whey so maybe it is the amount you take in at one time. Have you been tested for an actual allergy? I used to think I was allergic (because of post nasal drip) but tests were ngative and just to make sure I layed of any dairy for about a month with no difference.
Anyway, if you can handle milk you may be o.k with a protein blend with a small amount of whey/casien/egg/soy. Something like that.