There are two easy ways to remove your borders:
Either:
Switch them off in the .ini file. Find bBorderRegionsEnabled in the .ini and change the value to 0. You'll find your .ini in the folder My Games under Oblivion, and it will be a text file. Scroll down until you see Find bBorderRegionsEnabled=1 and change that 1 to a 0. That simple.
If you don't want to do that, try this mod instead: Tesnexus.com
Just plug it in and enjoy border-free Cyrodiil.
You will now be able to enjoy a multitude of over-the-border mods such as Silgrad Tower, Ravenview Village, and a number of castles and towns.