Hello Dave,
Many thanks for the suggestion! I hadn't considered that he might be a member of the Dockyard Police Band. It's a possibility, although the cap badge is very different. I know that the Police Band often played an important role with the dockyard - playing at launchings , etc., so I suppose he could have posed in his bandsman's cap rather than his normal helmet.
I have considered another alternative. The Dockyard Police operated a Water Branch to patrol the waters around the dockyard. In Devonport the police officers serving in the Water Branch lived in an old hulk, HMS Leda, with their families. Initially their patrols were undertaken in rowing boats, although they later had steam pinnaces. I wouldn't think that a police helmet would have been the most appropriate headgear for operating boats around the dockyard and that a cap might have been better suited. That might also explain why the cap badge is more like a variant of the helmet plate (sloping anchor and crown) than the bandsmans badge you have described.
However, although I have a lot of information concerning the living conditions and the work undertaken by members of the Water Branch I have not been able to find any photographs of them at work, so the above is just guesswork on my part. I suppose it might be worth trying to find some pictures of the river police operating on the Thames to see what sort of headgear they were wearing in the late-19th century.
Pete