כי כל שאר וכל דבש לא תקטירו ממנו אשה לה’ (ויקרא ב:יא)

Since the Korbanos serve as atonement for the aveiros one has sinned, if a person didn’t have a yetzer hara, he wouldn’t sin and wouldn’t need to bring a Korban.  Fruit and honey allude to the yetzer hara; therefore, they cannot have a connection to Korbanos.  Since it serves the opposite purpose of the Korban, and it shouldn’t be like “טובל ושרץ בידו” – one who immerses in the mikvah with an impurity in his hands.