E

The damage formula can be found in the article about combat mechanics in the AOW Wikia.
At first, it checks if the attack hits. The chance depends on the difference between attacker's Attack stat and target's Defence stat. Neither of those stats is relevant on its own. Default chance to hit is 50%. Each point of difference changes it by 10%: for example, if my Attack is 6 and target's Defence is 4, (6-4=2), the chance to hit is increased by (2*10%=20%), additively, to become 70%. However, there is always a chance of a "critical miss" of at least 10%, so the chance to hit cannot be more than 90%. The chance to hit also cannot be less than 10%, which is the chance of a "critical hit", which deals maximum damage (read below).
If the attack hits, the system decides how much damage is done. The maximum damage is the Damage stat of the attacker (let it be 5 in our case), the minimum damage is typically 1, and on most attacks, a random natural amount between the minimum and maximum damage is picked. Maximum damage is always dealt on a critical hit. So, in the above example, there is 70% chance to hit, from which there is 60% chance to deal a random amount of damage from 1 to 5, and 10% chance to deal 5 points of damage (maximum).
If the same target with Defence stat of 4 is attacked by an attacker with an Attack stat of 2, the difference is -2, and the chance to hit is 30%, from which 20% attacks deal a random amount of damage between 1 and maximum, and 10% are still critical hits dealing maximum damage.
If the attacker has an Attack stat of 1 (minimum), and the target has a Defence stat of 10 (maximum), the difference is -9, but there is still 10% chance to deal attacker's maximum damage on a critical hit.
If the (Attack-Defence) is 5 or more and maximum chance to hit (90%) is already reached, for example, Attack of 7 against Defence of 1, extra Attack points are also not wasted. For each point of difference between Attack and Defence above 4, minimum damage is amplified by 1. To say it mathematically, minimum damage is the larger of 1 and (Attack-Defence-3). So, if an attacker with Attack of 7 and Damage of 5 strikes someone with Defence of 1, difference is (7-1)=6, so minimum damage is (7-1-3=3), and maximum damage is still 5. There is 80% chance to deal a random amount of damage between 3 and 5, 10% chance to deal exactly 5 damage, and still 10% chance to miss.
When you try to hit someone, you hit once with all the checks mentioned above, then your target retaliates if it can and attacks you, making its own checks, then you strike again and your target retaliates again, doing checks each time. There is no retaliation if you kill the target with your strike or apply a condition that makes your target unable to attack (like Stunned, Frozen etc.), or this condition was already applied. Some abilities (most notably the Round Attack and ranged attacks) have no retaliation.
Things to note:
Some abilities use their own Attack and Damage, not the Attack and Damage stat of the unit, most notably all of the ranged attacks and attack spells.
Some abilities don't deal damage at all, but make a check if they hit, they can be called "touch attacks". At first, they make a check to hit using attacker's normal Attack stat against target's normal Defence. Then, if it hits, they make another check, using Attack stat of the ability against target's Resistance. That makes touch attacks very hard to apply, and actually dangerous to use due to a chance to suffer retaliation, but the reward is often worth it, especially if you have somehow disabled your enemy.
To answer the question in the title, damage formula is very complex, but I have put it into a spreadsheet that you can find here. The first sheet has chances to hit and minimum damage for a given difference between Attack and Defence. The second one is expected damage for a given Damage stat and given the difference between target's Defence and attacker's Attack. The third sheet lists increments of average expected damage if you boost Attack to the given value from the lower one, compared to the increment of average expected damage if you boost Damage stat, used to choose the way to invest skill points. Note that while one point in Attack costs 5 skill points, one point in Damage costs 10, that's why there is a column with the halved increment of increasing Damage. The fourth sheet holds a list of ranged abilities in AOW with their Attack and Damage values, again, different from Attack and Damage from the attacker. The average damage of those abilities against a given defence stat is calculated on the fifth sheet.
The sixth sheet now has an outline of all damage spells found in the game. The seventh sheet calculates average damage of each spell against a given Defence. The eighth list divides the average expected damage by mana cost to find the most effective spells. As you can see, for example, Stoning and Turn Undead are always weaker than Ice Shards, and they also deal less damage per mana point spent -- there is totally no reason to use Stoning or Turn Undead spells if you have access to Ice Shards.
Age of Wonders has a very complex mechanic that is hard to analyze, but once you do it, it can be just boiled down to meta moves.