K
This is a rather broad question, since it depends on the map and on the situation. In general, if your feudal aggression can take out villagers or win you the water, you usually end up ahead.
I'm just going to handle land maps, and I'll break it into closed and open maps.
Closed
On closed maps, the superior strategy is usually to skip the feudal age. This is because closed maps are very easily walled off, and feudal units take too long to break through walls. By default, go for a fast castle or fast imp, but don't be afraid of jumping at a chance for feudal aggression if it seems like you could get it to work.
Black Forest: It all depends on the initial wall-off. If you were aggressive in walling and managed to wall such that the opponent doesn't have a wall to match yours, go for the feudal aggression. Trushing seems to be a popular choice. This messy game commentated by ZeroEmpires features this strategy.
Arena: Feudal aggression is usually impossible. However, the walls in arena look like so:
####
#
#
#
There is a small "gap" in the corner. You could possibly "tower hop" the corner by placing a tower there and un-garrisoning units inside the tower. This video from the "Break the Meta" series by Resonance22 features it.
Those are the two most commonly played closed maps, but on other maps, look for opportunities. It might be possible to trush or go for archers even though your opponent locked themselves into their base. Keep your scout alive to help you make your decision.
Open
On open maps, the superior strategy is usually to go into a feudal war. This is because open maps tend to be hard to wall off, meaning that your army can get in there and take out villagers, preventing your opponent from castling or from actually using the castle age technologies due to lack of resources. On the other hand, your opponent is also likely to fight in the feudal age. With no army, how can you keep their units from slaying your villagers?
Arabia: This is the most popular open map. Almost every player will go for feudal aggression on this map. If your opponent tries to fast castle, your feudal army will take out many villagers, either preventing their castle age technologies from helping them or allowing you to castle and not be behind them. On the other hand, if your opponent manages to stone wall, you have to reach the castle age in order to debilitate them. There are a few options which might make you consider fast castling instead of going for a feudal war:
You drushed. While drushing, many players wall up and castle right after reaching feudal, as the extra villagers makes it easier to obtain the necessary resources. That doesn't mean that you can't go for a feudal army, but a castle age army is just that much stronger that you should probably fast castle unless your drush fails.
You realize that you have an easily walled map. In this case, you want to palisade wall / house wall up. If you can actually do this, go for the fast castle. When you reach feudal, consider stone walling or double-layering the palisades to make it even harder for your enemy to get in. You may need to act fast in the castle age and start churning out knights or mangonels to take out your enemy's feudal army pounding on your gates (don't forget to start pressuring your opponent with the army as well; unused military is a waste of resources). I would only do this if you have a place within your walled off base where you can place a town center, as one of the biggest advantages of the castle age is the ability to produce villagers from multiple town centers rather than just one.
Most open maps behave similarly to Arabia. However, some "open" maps are easily to wall, such as Mongolia. You have to be careful and keep your scout alive to see if your opponent is walling up. It is very costly to invest in feudal military units when your opponent is really fast castling behind walls.