While theoretically possible, this would be far from trivial to implement.
The replay doesn't quite cache your skin, MC skins are a bit more complex: When you connect to a server, that server queries Mojang to check if your account is legit and also gets a URL to your skin with a digital signature for that information in return. It then sends those two things to all connected clients with the tab list packet. The clients can then check if the skin URL is actually from Mojang using the signature (to prevent servers from faking it) and only then will the client go and fetch the skin from the given URL.
What happens when you change your skin is that Mojang simply assignes a new URL to the new skin and the old URL keeps pointing at the old skin, hence the player in the replay keeping their old skin.
So, to change your skin in a replay, you'd have to go through the whole fetch-url-from-Mojang dance, then parse the replay and replace the old url+signature with the new one.
To sum it up, it's probably easier if you can find some third party mod which allows you to set custom skins for players (I'd imagine it somewhat similar to how can set custom, local names for people on certain chat platforms) if such a mod exists.
