That's right. The plant would not grow. It is true that a plant needs to take in carbon dioxide in order to survive and that it expels oxygen as the waste product of photosynthesis. But a plant doesn't only store the food it produces—it uses some to feed itself. And when a plant feeds on its own food, that food is broken down in the same way that it's broken down in an animal's body (including yours): with oxygen. The oxygen is needed to break down the carbohydrate molecules and release the energy stored in those molecules.
While a plant is photosynthesizing, it's producing more than enough oxygen to break down its own food. But if you were to take away the oxygen surrounding the plant as well as the light it needs for photosynthesis, the plant would in effect starve.