Yes, you need to set up a Linked Server, give that [linkage-object] a "Friendly Name" - which would be "MyRemoteServer" in my example - and then you can do the 4-part naming.
I strongly recommend that you use SYNONYMS for 3 and 4 part naming - then if the remote server name, or the remote database, changes (e.g. from TEST to LIVE) then you can just change the SYNONYM instead of all your code!! It would help if you had all the synonym table DROP / CREATE in a single script, and then you can just FindAndReplace "MyTestRemoteServer" with "MyLiveRemoteServer" and re-run the script (or run the script on your local Test database one-way, and on your local Live database another-way)
Don't JOIN a RemoteServer.RemoteDatabase.dbo.RemoteTable if you can help it (SQL is liable to pull all the data local and then figure out the JOIN at that point throwing away all the irrelevant data ... which has used lots of bandwidth to "pull"). If you need to do that consider using OPENQUERY instead (most particularly if you have JOINs from one REMOTE table to another table ont hat same REMOTE server) , or pull the relevant rows into a #TEMP table locally, create a suitable index to facilitate any JOINs, and then JOIN the #TEMP to your other local tables.