Yeah, but
1) you get points for losing, and your team only needs to play 10 games a week (and you need to play 3) for you to get points. You rack up points fairly quickly.
2) you get matched to people of your level. Well, initially, you'll have an average rating, so will probably get creamed, but as your rating drops, you'll fight easier people.
3) you can get pretty uber gear from grinding battlegrounds. Hell, you can even buy blue pvp gear for gold now. Good players in cheap pvp gear can beat crap players in pimped out gear.
4) the best way to get better at arenas, is to... play arenas.
Nothing ventured, nothing gained, and all that jazz.
Not to mention that even if you hate PvP, some of the PvP items are really nice in PvE (especially weapons). I know people who hate PvP, but they got an arena team together, and just stood and danced for 10 matches a week, and now have sexy season-2 weapons for raiding.