R

Can someone clarify what the real difference is between the two and
why or when we should prefer one method over the other?
(Dynamic) symbolic execution is sometimes called whitebox fuzzing, see, e.g., SAGE. Fuzzing can be grouped into categories:
Blackbox fuzzing: doesn't require any information about the system under test (SUT). There are two strategies to generate input: generate from a grammar/model, or mutate from existing ones.
Whitebox fuzzing: require all the information about the SUT, including APIs, environment etc. It needs to know which instruction is executed, so that it can replace the semantics of the instruction with the symbolic semantics.
Greybox fuzzing: require to know a little information about the SUT, i.e. basic block transition. Currently it is the most successful, see, e.g., this trophy case.
Can someone clarify what the real difference is between the two and
why or when we should prefer one method over the other?
Let consider a program with an input x, having the following condition: if (x > 5 && x < 10) abort();
Suppose x is an integer, i.e. its domain is 232. The probability to randomly generate an input x that triggers the error is 4/232. This probability is a bit higher in clever greybox fuzzing, but in general they suck for this kind of condition.
On the other hand, whitebox fuzzing executes the program with a symbol x = X, and use a constraint solver, e.g. Z3, to solve the constraint. Z3 can solve X > 5 ∧ X < 10 in mili or nano second.
However, there is no free lunch, this power comes with an expensive price.
A recent paper shows that for just one path, i.e. no constraint solving, an execution in KLEE is 3000 times slower than native execution, while angr is 300,000 times slower. Since it is very slow, it is unrealistic to achieve 100% coverage.
Constraint solver is only good for linear arithmetic. There is no efficient solver for the theory of string, floating point arithmetic etc
It is unrealistic to know all information about APIs, environment etc.
The current state-of-the-art is hybrid fuzzing, combining both greybox and whitebox. The idea is to use greybox fuzzer as global search to quickly sample the state space. When it gets stuck, then use the heavyweight whitebox as local search. This is the technique used by all the team in the Cyber Grand Challenge. The hybrid fuzzer QSym , running on a modest machine, discovered some CVE that Google couldn't find with greybox fuzzers running on the cloud.