Basically, when a Protean mon uses Curse, it targets the user, changes the user's type to ghost and then the Damage Over Time effect of a GhostUser!Curse is applied on the user.
I wonder how Protean + Roost would work. My guess is that it'll change the user's type to Flying, and then remove the Flying type and become Normal type. I have this reasoning because from Gen V onwards, a pure Flying type using Roost becomes Normal type (unlike ??? in Gen IV).
Thank Sleepy Jirachi for this find.