You may want to use some self etching primer. I used regular primer, then painted, and most the paint came off. So I stripped them down and used self etching primer, repainted, and have had very good results.
what kind of paint are you using? Ive painted wheels with the duplicolor wheel paint. Just got them really clean and shot it with no primer. It did really good, had it for a year and stripped them down to change the color.
ive used the engine paint aswell. It will look much better with clear coat. I didnt prime with that either and it held up good. Engine paint is very durable and hard to strip if you change you mind on the color.