That’s a really interesting question. If it was a fort of the First Men prior to or during the Long Night, I wouldn’t be surprised if there wasn’t any evidence – 8,000 years in an exposed location will obscure a lot. And w/o something like the Wall, maybe it was the best the First Men could do at the time, because they do seem to have wanted to use high ground against the undead.
As for the Others, it’s tricky – we haven’t seen the Others create any kind of structure before. Certainly they don’t need to shelter from the cold, so maybe that’s not part of their culture/society at all? The only thing we’ve seen that’s close to them using a trap is with the wights in Jon VII of AGOT, but that’s not building-related.
It’s certainly a possibility, but I’m withholding judgement for the moment.