Basically, Pelosi is only polarizing to the already polarized. She is not considered the public face of the Democratic Party in the House because Obama is the public face of the Democratic Party generally. To the extent she has low approval ratings, that is a function of general low approval ratings of Congress, and Republicans are not doing any better than Dems in general public perception.
Bottom line remains what I have been saying for a year. This will be the first election since 2002 that is fought primarily on local issues, not national issues. Republicans will do best if they can identify with local voters on local issues and tap specific anti-incumbent complaints rather than run on a general anti-Obama/Pelosi/Reid ticket.