When you want to revoke a user being able to access a locked feature you will need to do this from within the padlock feature.
At the bottom of the page where you would grant access to a user is where you can see all the users you have granted access to.
Search for the user you want to remove and the click the red Remove button to revoke all their access permissions.