Dragoknight, the main reason only two Enterprises have been destroyed is because Star Trek relies heavily on Deus Ex Machina in order to get out of any trouble. When all hope seems lost, they suddenly realise they have some sort of magic crystal that when mixed with tachyon waves or photon energy or Klingon blood makes them invincible.
Realistically speaking (as realistic as this discussion could get) if both ships were being fully manned by competent crews, the Star Destroyer would win hands down. Star Destroyers are warships, whereas the Enterprise seems to be a jack-of-all-trades, master-of-none ship that is suitable for exploration, small skirmishes and diplomatic missions. However, Star Destroyers often carry with them around a hundred starfighters, including bombers that would make turn the Enterprise into scrap metal. The show Star Trek has shown time and time again that the Enterprise can be easily crippled by a ship with superior starfighting capabilities.
Furthermore, a Star Destroyer is over twice the size of the Enterprise and carries enough firepower to lay waste to the surface of the planet it is orbiting over.
So, Star Destroyer wins.