summer house

noun, a secondary residence occupied during the summer (COD); see also summer home, summer place

“You remember the Smiths that lived next door to us in the summer house?”