Quantum asymmetry and coherence are genuinely quantum resources that are essential to realize quantum advantage in information technologies. However, all quantum processes are fundamentally constrained by quantum speed limits, which raises the question on the corresponding bounds on the rate of consumption of asymmetry and coherence. In the present work, we derive a formulation of the quantum speed limit for observables in terms of the trace-norm asymmetry of the time-dependent quantum state relative to the observable. This version of the quantum speed limit can be shown to be directly relevant in weak measurements and quantum metrology. It can be further related to quantum coherence relative to the observable, and we obtain a complementary relation for the speed of three mutually unbiased observables for a single qubit. As an application, we derive a notion of a quantum thermodynamic speed limit.