Oh no. They literally are at the door. They live about 15 miles away. It's not like they come into the house. They just turn up uninvited. Typically with a 'I'm so stressed with the kids' dialogue. This is only because we lock the door. There have been two occassions, where they have tried to open the door, to either find out the door was locked (frantic wrangling of the door) or to find out it was left on the latch with a chain.
Your response seems completely fair and justified.
And for gods sake do not give them a key and keep that door locked, they might have just been trying to snoop and compare how they are doing versus you...but something is off. They should have asked to use your laptop if they just wanted to game or websearch on it.