Jerusalem is built as a city that is compact together:
All Commentaries on Psalms 122:3 Go To Psalms 122
Augustine of Hippo
AD 430
4. "Jerusalem that is being built as a city" (ver. 3). Brethren, when David was uttering these words, that city had been finished, it was not being built. It is some city he speaketh of, therefore, which is now being built, unto which living stones run in faith, of whom Peter saith, "Ye also, as lively stones, are built up a spiritual house;" that is, the holy temple of God. What meaneth, ye are built up as lively stones? Thou livest, if thou believest: but if thou believest, thou art made a temple of God; for the Apostle Paul saith, "The temple of God is holy, which temple are ye." This city is therefore now in building; stones are cut down from the hills by the hands of those who preach truth, they are squared that they may enter into an everlasting structure. There are still many stones in the hands of the Builder: let them not fall from His hands, that they may be built perfect into the structure of the temple. This, then, is the "Jerusalem that is being built as a city:" Christ is its foundation. The Apostle Paul saith, "Other foundation can no man lay than that is laid, which is Christ Jesus." When a foundation is laid on earth, the walls are built above, and the weight of the walls tends towards the lowest parts, because the foundation is laid at the bottom. But if our foundation be in heaven, let us be built towards heaven. Bodies have built the edifice of this basilica, the ample size of which ye see; and since bodies have built it, they placed the foundation lowest: but since we are spiritually built, our foundation is placed at the highest point. Let us therefore run thither, where we may be built ...But what Jerusalem do I speak of? Is it that, he asketh, which ye see standing, raised on the structure of its walls? No; but the "Jerusalem which is being built as a city." Why not, a city, instead of, "as a city;" save because those walls, so built in Jerusalem, were a visible city, as it is by all called a city, literally; but this is being built "as a city," for they who enter it are like living stones; for they are not literally stones? Just as they are called stones, and yet are not so: so the city styled "as a city," is not a city; for he said, "is being built." For by the word building, he meant to be understood the structure, and cohesion of bodies and walls. For a city is properly understood of the men that inhabit there. But in saying "is building," he showed us that he meant a town. And since a spiritual building hath some resemblance to a bodily building, therefore it "is building as a city."
5. But let the following words remove all doubt that we ought not to understand carnally the words, "Whose partaking is in the same." ...What meaneth, "the same"? What is ever in the same state; not what is now in one state, now in another. What then is, "the same," save that which is? What is that which is? That which is everlasting ...Behold "The Same: I Am that I Am, I Am." Thou canst not understand; it is much to understand, it is much to apprehend. Remember what He, whom thou canst not comprehend, became for thee. Remember the flesh of Christ, towards which thou wast raised when sick, and when left half dead from the wounds of robbers, that thou mightest be brought to the Inn, and there mightest be cured. Let us therefore run unto the Lord's house, and reach the city where our feet may stand; the city "that is building as a city: whose partaking is in The Same." ...
6. That city "which partaketh in the same," partaketh in its stability: justly therefore, since he is made a sharer in its stability, saith he who runneth thither. For all things there stand where nought passeth by. Dost thou too wish to stand there and not to pass by? Run thither. Nobody hath "the same" from himself ...
