Some liberal activists have called for the 70-year-old Obama appointee to retire political power changes hands in January.