I have to agree with Camli on how haste works in this game. The best reason I have for this is the 'Attack Haste' associated with Dual Weild. 100% haste does not bring your delay to 0, but rather to 1/2 the stated amount. Here's a formula for calculating it:
HastedDelay = StandardDelay * (1/(1+Haste%))
Assuming mulitiplicative bonuses (as most everything in this game is multiplicative rather than additive), and a StandardDelay of 8 seconds, you would get the following  assuming that these three modifiers were all at max:
Jeweler Class Bonus: 5%15%
Gem Haste Aura: 4%13%
Wizardry: 40%80%
HastedDelay = 8.0seconds * (1/(1+15%)) * (1/(1+13%)) * (1/(1+80%)) = 3.42 seconds between gem draws.
