Even if they can grow, olive trees don't even produce fruit for decades. There are sayings about planting an olive grove for your grandchildren for a reason.