During the Tenancy: You are perfectly within your rights to keep a set of landlord keys to the property during the tenancy, but use them sensibly. Always give the tenant at least 24 hours notice of any visits you want to make. You have a legal right to enter the property to carry out checks on its condition but if you conduct these excessively, it may be construed as an infringement of the tenant’s legal right to quiet enjoyment or could even be seen as harassment. You should make your right to enter clear in the tenancy agreement, but ensure the clause is reasonable, e.g. ‘the tenant cannot unreasonably deny the landlord access’ etc. That way, if the tenant consistently denies you entry to the property, they will be in breach of the contract and you will have a good chance of getting an injunction against them and/or receiving compensation for any damage. If you turn up for the inspection and the tenant is not in, you should be fine to enter, so long as you gave notice and can prove it!
What if the tenant changes the locks? The tenant has no general right to change the locks. It is therefore wise to put a clause into the agreement to say it cannot be done. That way, if they do change them, it will be a clear breach of contract and you will be able to charge the tenant for changing them back. Note that in extreme cases, the law may side with the tenant (for example, if the landlord is clearly guilty of aggressive harassment).
The end of the tenancy: Obviously the security of the property is a top priority so make it very clear in the tenancy agreement how and when the keys should be returned (usually on the day of check-out). You could also provide a reminder when you inform the tenant of the check out procedure. If they fail to adhere to this, you are within your rights to charge the tenant for changing the locks.
Please remember that this is just a general guide. If you are in any doubt, always seek specialist legal advice.