Yes you can do it by going to the tools bar menu of the internet explorer window and there you have clear history option click on it and click at delete cookies . There will be a check box for delete offline cookies check it all the sites you visited will be cleared
