Also, worldbuilding touches all aspects of your story. It touches plot and character as well. If you don't know the culture your character comes from, how can you know what he's really like? You must know your characters on a much deeper level than you would if you just shrugged your way into a cookie cutter fantasy world.
Patrick RothfussChronicler picked up his pen, but before he could dip it, Kvothe held up a hand. "Let me say one thing before I start. I've told stories in the past, painted pictures with words, told hard lies and harder truths. Once, I sang colors to a blind man. Seven hours I played, but at the end he said he saw them, green and red and gold. That, I think, was easier than this. Trying to make you understand her with nothing more than words. You have never seen her, never heard her voice. You cannot know.
Patrick Rothfuss