To be fair, I have always found this particular part of the hall to be oddly bare. It was my best guess for where it might be and as it happens,

[He holds the door open for John to see the nice fire and seating area with food already waiting.]

I was right. Though a voyeuristic castle does sound like an interesting hypothesis.