Yup, regardless of whether you replied to their tweet initially or not, people will always be able to message you directly by using your twitter handle.
There are only two instances where things might get complicated:
1) If your account is locked so that your tweets are protected.
Only people that you've approved to follow you can see your tweets. So if the person you just replied to isn't on your follower list - they wouldn't have seen your response.
2) Direct messages.
If you're following someone, they can direct message (or DM) you. But to reply, they need to be following you back!
