I wouldn't worry much about swapping characters.. you won't lose or gain anything really meaningful unless you do the swap over and over. I'm trying to nail that fixed down, but it seems for some thing to be a game bug (for example, the light armor proficiency remaining). Anyway, if you swap 1 or 2 times you are not game breaking.
About the relic: when you get 3 of them (there are 4) use them in the reliquary (near death shrine).