According to the IBM Dictionary of Computing (which cites International Organization for Standardization Information Technology Vocabulary as the source), response time is:
The elapsed time between the end of an inquiry or demand on a computer system and the beginning of a response; for example, the length of the time between an indication of the end of an inquiry and the display of the first character of the response at a user terminal.
There is also the concept of perceived response time, which is the time a user senses as the beginning of input and the end of the response. It is actually possible (though not usual) for perceived response time to be too fast (it can be mildly disconcerting if a system responds almost instantly). However, this is not the usual complaint.