The goal of sanding wood that has primer on it is to smooth down the grain that has risen while still leaving the primer intact so you do not lose the added adhesion that the primer will provide for the top coat. To sand wood with primer on it you should consider a finer grit sandpaper (like a 220 grit sand paper).