At 100 yards, assuming you mean to zero at 100, then go shoot in different conditions also at 100, then no, it doesn't matter much.
What does matter, at least to some degree, is recording the density altitude when you zeroed your rifle, so that can be used in your ballistic calculations for longer range shooting.
For example, you're going to take a 900 yard shot at 6,000 ft DA; it can make some difference whether you zeroed at 6,000 ft DA, or at -1,000 ft DA. (My location does often have negative DA values in winter, being very near sea level with cold weather.)
Changes resulting from different density altitude values are fairly small at 100 yards, but that error grows with distance. The farther you shoot (and the more DA changes), the more it matters.