The output voltage does not change as the load increases in an ideal voltage source. The voltage across the load is always the same, no matter its magnitude. This cannot exist, the source is not a superconductor.
The output voltage, also known as the terminal voltage, decreases as the load increases. It can be modelled by a series-connected internal resistor, which, if the current is higher, will have a higher voltage across it, so less across the load. The internal resistance is given in the data sheet, for example INR18650-29E cell: 35mΩ.
If there is nothing connected to the power supply, there is no current flow, this is an open circuit. In this case, the maximum voltage can be measured as there is no voltage across the internal resistor. It can also be said that an infinite resistance is the load. However, if there is a short circuit, meaning the positive and negative leads are directly connected, then there will be maximum current due to the absence of a load.
All of the diagrams up until this point show a voltage generator. Its function is to provide a constant voltage under a varying load, allowing the current to fluctuate. The default mode of operation for power supplies is CV, or Constant Voltage. A 230V outlet also maintains its voltage regardless of the load plugged into the socket.
There are cases where a current source is needed that provides a constant current (CC), so that the voltage is able to fluctuate. For example the charging algorithm for batteries usually consists of first CC and at the end CV phases. Under constant light conditions, a solar cell can also be considered as a current generator.
A practical current generator also has an internal resistance that affects its output. It can be modelled with a resistor connected in parallel, which causes the current to the load to decrease as the voltage increases.