Don't be afraid of losing, be afraid of playing a game and not learning something.
When you trade, the key concern is not always the value of the pieces being exchanged, but what's left on the board.
If you are interested in improving, think of a draw offer as an offer to remain ignorant of what you would have learned in the remainder of the game.
Don't worry about your rating, work on your playing strength and your rating will follow.