This was known as exclusive psalmody. The psalms weren't the prose-style you'll find in any print Bible though, they were their own translation called the Metrical Psalms.
the act of singing psalms or hymns
See also: hymnody