Ground based sky imaging and irradiance sensors are used to quantitatively evaluate the impact of cloud transmittance and cloud velocity on the accuracy of short-term direct normal irradiance (DNI) forecasts. Eight representative partly-cloudy days are used as an evaluation dataset. Results show that incorporating real-time sky and cloud transmittances as inputs reduces the root mean square error (RMSE) of forecasts of both the Deterministic model (Det) (16.3%̃ 17.8% reduction) and the multi-layer perceptron network model (MLP) (0.8% ̃ 6.2% reduction). Four computer vision methods: the particle image velocimetry method, the optical flow method, the x-correlation method and the scale-invariant feature transform method have accuracies of 83.9%, 83.5%, 79.2% and 60.9% in deriving cloud velocity, with respect to manual detection. Analysis also shows that the cloud velocity has significant impact on the accuracy of DNI forecasts: underestimating the cloud velocity magnitude by 50% results in 30.2% (Det) and 24.2% (MLP) increase of forecast RMSE; a 50% overestimate results in 7.0% (Det) and 8.4% (MLP) increase of RMSE; a ±30° deviation of cloud velocity direction increases the forecast RMSE by 6.2% (Det) and 6.6% (MLP).