What does it mean for an algorithm to converge?

I keep coming across this term when reading about reinforcement learning, for example in this sentence:

If the problem is modelled with care, some Reinforcement Learning algorithms can converge to the global optimum

http://reinforcementlearning.ai-depot.com/

or here:

For any fixed policy Pi, the TD algorithm described above has been proved to converge to VPi

http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node62.html

My understanding of the word converge is that it means several things coming together to the same point, but how can a single thing (the algorithm) do that?