Steering wheel noise?

I have a 2003 Camry LE, and lately have experienced a noise in the steering wheel or column that sounds like rubber friction.
 
The sound is more from the wheel than from teh coumn itself, and lubricating the joint between the wheel and teh column didn't seem to help out at all. If anything, it did temporarily, as I did this Saturday and by this AM (Monday) my wife was complaining of the same thing.
 
I guess what I am really asking is how hard of a job is it to remove the steering wheel from, and then reassemble, the steering column??
 
Thanks....