Yup, that works 
SELECT A.Col1, B.Col2, ...
FROM DatabaseA.dbo.TableA AS A
JOIN DatabaseB.dbo.TableB AS B
ON B.ID = A.ID
...
however ... databases tend to move around / get split / get renamed / move to different servers, and you might have Production, Test and Development versions - perhaps sometimes you need to connect DEV of DatabaseA to Production of DatabaseB and so on ...
I prefer to use SYNONYMS for that sort of thing (both for 3-part-naming of tables in other databases, and also 4-part naming for things on different servers).
I create a Schema for each "external" database
IF NOT EXISTS (SELECT * FROM sys.schemas WHERE name = 'DatabaseB')
BEGIN
EXEC( 'CREATE SCHEMA DatabaseB' )
END
and then I create a SYNONYM for each table in DatabaseB that I want to access (from the current database).
IF (SELECT OBJECT_ID('DatabaseB.Table1')) IS NOT NULL
DROP SYNONYM DatabaseB.Table1
CREATE SYNONYM DatabaseB.Table1
FOR DatabaseB.dbo.Table1
then (assuming that the current database is DatabaseA) the above query (repeated here):
SELECT A.Col1, B.Col2, ...
FROM DatabaseA.dbo.TableA AS A
JOIN DatabaseB.dbo.TableB AS B
ON B.ID = A.ID
...
becomes this:
SELECT A.Col1, B.Col2, ...
FROM dbo.TableA AS A
JOIN DatabaseB.TableB AS B
ON B.ID = A.ID
...
If DatabaseB changes name or I want to access the TEST database version instead of the DEV database version, or DatabseB moves to a new server, or TableB in DatabaseB moves to DatabaseX on ServerY then i can just change the SYNONYM definition and I don't have to change a single line of code.
Long story short: avoid hardcoding Database and Server names in your SQL queries.