1) We have far more polls over a longer period of time on Trump. The more polls there are, the lower the likelihood that they are all wrong.
2) The Brexit vote was nationwide, rather than broken down by geographic region for an electoral college-like scheme. Because we conduct the presidential election on a statewide basis, we poll at the state-level, and polling is against Trump across states.
3) The degree to which a state leans for or against Trump in the polls is consistent with the partisanship of the state. For example, Georgia, as a Republican state with an increasingly urban and large African-American population, only has a 4 point Trump advantage. Contrast that with a 14 point Trump advantage in Louisiana. So, when we see a 4 point Clinton advantage in Florida, which Obama has won in the last two elections, combined with Trump's weakness as a candidate, we do not need to discount that as a Brexit-type polling error because our polling across states is consistent with overall partisan patterns. Add that to Clinton locking down New Hampshire and Colorado, and she has probably won.
4) A Brexit vote and a candidate vote are different. This should go without saying, but there really hadn't been many precedents for a vote to leave the EU. That isn't so much a polling issue as a voter response issue. Voters simply didn't know how to respond because they hadn't cast votes on the issue before. It wasn't necessarily lying, then, so much as inability to predict their own behavior. Presidential elections happen every four years. And, they are mostly partisan affairs. The difference is that this year, one candidate is the worst nominee ever, and his polling numbers reflect that.
The polls could be wrong. If so, though, it would be a polling error of a magnitude that would make the Brexit failure look like a minor glitch.