I removed a signer who hasn't signed yet. Will they still have access to the document if they click the link?

Not. If the signer tries to enter the signature link again, they will get a “document not found” error.