What does it mean for an algorithm to converge?

I keep coming across this term when reading about reinforcement learning, for example in this sentence:

If the problem is modelled with care, some Reinforcement Learning algorithms can converge to the global optimum


or here:

For any fixed policy Pi, the TD algorithm described above has been proved to converge to VPi


My understanding of the word converge is that it means several things coming together to the same point, but how can a single thing (the algorithm) do that?