Paul, well said.
We know the original screen-used ANH was molded and then castings were made to provide the helmets for ESB and ROTJ. With any new pull from a mold, it's subject to molding, casting, curing issues, how it's pulled, sanding preparation for painting, etc. (and how
much something was sanded too.) Then, along the way in filming, props are mishandled, dropped, repaired, patched, repainted, etc. What part all these things play - whether requested or a natural byproduct of being produced by the propmakers - in terms of how they contribute to how the audience perceives the character is interesting. The causes and effects of both intentional and unintentional changes are speculative. One cannot say that Lucas intended some facet or feature of the mask to be, say, exactly 1.3 mm narrower than in the previous movie because with ESB and ROTJ. On the other hand, as Paul indicated, the totality of changes (subtle and overt) do seem to lend themselves to the development of the character across the OT. Sometimes things are intentional, and sometimes things are just dumb luck - which anyone with the right frame of mind would try to chalk up to being, instead, a stroke of genius!
The recent documentary of the making of the ROTS Vader seemed to indicate that Lucas sought to improve Vader with each film. His feeling that the ROTS Vader being the epiphany of the helmet's evolutionary process is obviously disagreed with by many fans of the original trilogy.
There are two aspects of that development I enjoy discussing: the development of his role from ANH through ROTJ, and the emotional subtext (though it's more behind the scenes, and later exemplified by the new context provided by the prequels) in conjunction with how the differences of his costume played into that subtext.
In ANH, we see his mask and armor rough and raw. Marketing materials and press coverage described him as the "Dark Lord of the Sith" even though many didn't know what the "Sith", yet he was clearly the fearsome antagonist. However, he is a bit like a thug, on the end of Tarkin's leash, and even commanded by Tarkin.
In ESB, Vader has gained prestige. He now commands a Super Star Destroyer, and he has his own personal chamber which almost doubles as a throne. We see him reporting directly to the Emperor (who we see for the first time). There are definitely noticable changes in paint finish (paint was sprayed on and the paint job more refined. By contrast, the ANH's metallic gray was brushed on by hand). You also notice the widow's peak is gone (in ANH it helped created the angry look and sense of meanace) and the mask's own eyebrows are more noticable.
Anyways, in ESB, Vader is clearly at the top of his game. There was definitely something that made him appear more elegant overall. (It reminds me of C3PO in tahat when you see 3PO throughout ANH, he had a weathered matte finish, but at the end ceremony, he was given a full presentation polish.)
In ROTJ, we see him side by side with the Emperor, but now we see him also tortured on the inside. When he steps off the shuttle into the landing bay of the Death Star II, you see remarkable level of light reflections off of his dome. (For some reason, I've always felt that ROTJ's light balance and brightness levels seem to make Vader look very dark.) I agree with Paul that Vader looked less aggressive. Along these lines, while in ESB the dome's rim was positioned such that you saw the mask's eyebrows, ROTJ you see the dome positioned above the eyebrows. Whether intentional or not, this somehow this played into the character's sadness. Lucas described the six episodes as being about "The Tragedy of Darth Vader."
Now there are changes that took place from ANH to ESB to ROTJ that perhaps were not done specifically for contributing to the portrayal of the character. The chin triangle was widened in ESB to allow Prowse to breathe better. Apparently in ANH, there was originally no chin triangle, but was made as Prowse had complained he was having difficulty breathing through just the teeth holes.
The armored parts of Vader in ESB and ROTJ appeared glossier than in ANH. The ANH dome had a bit more gloss than most people realize, but you can also see the surface effects of fingerprint grease not properly wiped off. Also, our perception of gloss is based on how a gloss surface reflects light - as well as the nature of the surrounding lighting. In ANH, we see very dark interiors of the Death Star which had very soft overhead lighting and lots of lateral lighting from the wall cutouts. In the Tantive shots you had overhead lighting that was diffused, but the white interiors of the ship bounced light everywhere. With ESB, however, the lighting was more contrasted and you could really appreciate the wetness or gloss of the dome. The reflections looked deeper. It was more refined looking. Similar with ROTJ when we see him in the Death Star II. However towards the end when Palpatine is frying Luke with the Force Lightning, we see Vader's armored parts covered with white powdery oxidation (powdered burnt Luke, and/or smoke from having his hand cut off) which lent a little to a battle-ravaged look. This is simply a surface effect and not a structural effect.
Vader looked really sad (emotionally and aesthetically) when Luke unmasked him. By this time, we as the audience is empathizing with what's left of the human amid the machine. Here was one fearsome Dark Lord of the Sith, and now you felt sorry for him - a weak old man barely able to breath under the imprisonment and horrific weight of all those cybernetics.
Unfortunately, anything we can appreciate from Vader's portrayal was utterly overshadowed by those lousy little Ewoks!
Much to the discontent of hardcore fans, Lucas sought to improve upon the appearance of Vader for ROTS. The Costume Props Supervisor for ROTS, Ivo Coveney, discussed how Vader's face was asymmetrical, and that with the result of 3D scanning and computing efforts, a mirrored replacement facial piece was created to achieve the first-ever symmetrical Vader. And we know how the ROTS Vader turned out. Somehow, Vader appeared younger and cleaner. Was this the intentional result of simply trying to make Vader symmetrical? It certainly played into the character. Anakin had just joined Sidious and was at the beginning of his apprenticeship. Twenty years later, we'd come to Episode IV: ANH where he had the look of having seen battles for two decades.