As "wishie" pointed out on IRC the alsa-lib plugin currently seems to ask for specific sink. In result, module-volume-restore will never restore the sink for alsa-lib clients, because it only becomes active if the sink name is left empty by the client. Because many alsa apps close the device when seeking this results in a strange behaviour when the user uses pavucontrol to move a stream to another device: after the seek it is back on the old device.
Ideally, the alsa plugin would not ask for a specific device unless this is really necessary. I am not sure however, if this is actually an issue with the plugin or in the app making use of it.