You are not exactly correct in what you are saying. In a real world experiment there will be a finite rise-time when dI/dV changes from one value to another, particularly if it is measured as a voltage going from one value to another. If d2I/dV2 were being correctly represented based on the dI/dV graph shown, it would be a very short square pulse-like wave-form. However, in a real world set-up again there will be a finite time in which the voltage signal representing d2I/dV2 will take to reach its value, and again to return to zero. The duration spent at the value is very short hence the waveform looks like a triangular peak rather than a square pulse.
If the graph for dI/dV were represented ideally as a square step, then d2I/dV2 would be a single point with zero everywhere else. This diagram is more to illustrate an idea, that you will get a peak in your d2I/dV2 signal when you have a change in your measured conductance. These are real world phenomena which will be subject to the effects detailed above.
Please feel free to point out where this argument may be wrong. John