Sharing DSL connection between two pcs


Tata Indicom
Hi, I am from Mumbai. I have a DSL internet connection where I do not use a router or modem, just a cable cat 5e comes directly into my home from a switch provided by my isp on the top floor of the building where I live and I just plug it into my Ethernet port and it works with a user name and password. It is a DSL connection, which uses DHCP. How do I share this DSL connection between two pcs? One laptop with Windows XP Home on it and another desktop with Ubuntu 10.10 on it. I have been told to use a router by my isp to share the connection between these two pcs. I am not sure how to do this? Is it ok that both the pcs (one laptop and another a desktop) have different operating systems? Please provide me step by step guidelines. Thanks