Tirefour Castle is situated on a rocky height on the east coast of the island of Lismore. In clear weather Ben Nevis can be seen to the north, Ben Cruachan to the east and the Paps of Jura to the south. There is a steep slope on the northwest side and on the southeast side. The latter slope turns into a rolling plateau that ends in a steep cliff. The broch can easily be approached from the northeast and the southwest.