First of all, this is my favorite part of Sillmarillion. -Astapor
Although Sauron long knew that Men were easier to sway, he sought to bring the Elves into his service, as they were far more powerful. So after about a thousand years after the War of Wrath, Sauron decided that the Valar had forgotten about Middle-earth and he once again turned to evil.
After lying hidden and increasing his power in secret, Sauron put on a fair visage, calling himself Annatar, the Lord of Gifts, and in S.A. 1200 he befriended the Elvish smiths of Eregion, counselling them in arts and magic. He was never welcome in Lindon as Elrond and Gil-galad did not trust him and refused to treat with him (which included Galadriel too), although they never realised who he truly was. Elsewhere he was gladly received, especially in Eregion where the Elvish smiths learned much from him as their thirst for knowledge was great.
Under Sauron's tutelage the Gwaith-i-Mírdain, under the leadership of Celebrimbor, grandson of Fëanor, became more skilled than anyone else, save for Fëanor himself. In the year S.A. 1500, when they reached the very height of their power, the Elves began the forging of the Rings of Power, but Sauron knew all their secrets and in S.A. 1600 - ten years after the completion of the Rings of Power - Sauron created The One Ring to control the bearers of the other Rings. For this he invested most of his own power into the Ring as he forged it.
But the Elves were not so easily ensnared, and as soon as Sauron put on the One Ring they and Celebrimbor were aware of him, and realised they were betrayed. They hid their Rings from Sauron and did not use them. Sauron demanded that the other Rings be given to him, for they would not have been made without his knowledge. The Elves refused, and the War was inevitable.
In this time Sauron became known as the Dark Lord of Mordor. He raised Barad-dûr, the Dark Tower, near Mount Doom; constructed the Black Gate of Mordor to prevent invasion; and raised massive armies of Orcs, Trolls, and Men, chiefly Easterlings and Southrons. Sauron's power reached its zenith 700 years after Mordor's creation, in the 17th century of the Second Age.
The War of the Elves and Sauron was a bloody conflict which destroyed Eregion and devastated much of Eriador. Celebrimbor was slain and his body impaled on a spike paraded at the head of Sauron's legions. The Elves were pushed back almost to the Blue Mountains, while their Dwarf allies (who had also rejected Sauron) retreated behind the walls of Moria where Sauron could not assail them. Sauron was master of almost all of Middle-earth beyond the coasts, but the Númenóreans, the powerful Men descended from the line of Beren and Lúthien, who lived on the island of Númenor in the sea between Middle-earth and Valinor, responded to the Elves' call for aid and sent a relief force. The combined armies rallied and were able to defeat Sauron's armies in Eriador after heavy fighting, and the Dark Lord fled back to Mordor with little more than his own bodyguard and a handful of orcs.
As the Elves had failed him, he had decided to distribute the Rings of Power to Men and Dwarves. The Dwarves also failed him, as they proved too hardy and resistant to their corruptive power; however the nine Men were corrupted and eventually faded, being turned into the Nazgûl (Ringwraiths), his chief servants.
Nonetheless, while Sauron's subsequent power never quite matched the height it had during the War with the elves, many of his most powerful enemies' homelands had been devastated. Relative to his enemies, Sauron's empire was actually in a stronger position than it used to be. His empire continued to expand to dominate barbarian Men to the far south and east. Throughout this, Sauron remained faithful in his old allegiance, building temples to the worship of Morgoth, where human sacrifice was practiced. Because of this, towards the end of the Second Age, Sauron assumed the titles of Lord of the Earth and King of Men.
This offended the arrogant Númenóreans who had already started to fall under the Shadow. The proud Númenóreans came to Middle-earth with great force of arms, and Sauron's forces fled. Realizing he could not defeat the Numenoreans with military strength, Sauron allowed himself to be taken as a hostage to Númenor by King Ar-Pharazôn. There, he quickly grew from captive to advisor and was known as Tar-Mairon; he converted many Númenóreans to the worship of Morgoth, becoming High Priest of the Cult of Melkor. He had the White Tree cut down and in its place raised a great temple in which he performed human sacrifices, persecuting those who were still Faithful. Finally, he convinced the king to rebel against the Valar and attack Valinor itself, claiming they would gain immortality. Eru, the supreme god, then directly intervened: Númenor was drowned under the sea, and the great navy of Númenor was destroyed.
Sauron was on Númenor in the Temple of Melkor and was caught in the ensuing flood. However his spirit survived, although severely weakened by the destruction, and (presumably carrying the One Ring) fled back to Middle-earth.
Sauron's spirit returned to Mordor, where he slowly rebuilt his strength during the time known as the Dark Years, and he was unable to assume a fair shape. From this point on he started to rule through terror and force.
Meanwhile, a few faithful Númenóreans, led by Elendil, were saved from the flood, and they founded Gondor and Arnor in Middle-earth. Sauron still considered them his hated enemies and he launched a pre-emptive attack on Gondor in S.A. 3429. These Men, led by Elendil and his sons, formed the Last Alliance with the Elves of Lindon under the Elven-king Gil-galad, and together they fought Sauron in the War of the Last Alliance. The Alliance advanced against Mordor and defeated Sauron's forces in the Battle of Dagorlad and finally laid siege before Barad-dûr.
The siege lasted for seven years until S.A. 3441, when Sauron left his fortress engaging in direct combat. Elendil and Gil-galad fought Sauron and vanquished him, but both were killed. Isildur, son of Elendil, cut the One Ring from Sauron's finger and claimed it. Later, the Ring betrayed him and was lost for more than two thousand years. After his defeat in the War of the Last Alliance, Sauron had lost his ability to form a physical body for a great while.