From 1970 to 1975, Mamiya Michio was fascinated by avant-garde jazz musicians Ornette Coleman and Cecil Taylor, and composed several works linked to his interest in jazz. Piano Sonata No. 2 was also written during this period and is one such work, though it does not employ jazz idioms. According to the composer himself, this work has a twin-like relationship with his Sonata for Solo Violin, composed in 1970. Mamiya found in these two jazz geniuses an aggressive stance: to tear the violin and piano, two beautiful and traditional European instruments, away from their tradition, and to exhaustively use them for self-expression by any means necessary.

Commissioned by Toshiba Records, the work was commenced in the autumn of 1972, with the first movement completed in February and the second movement in May of the following year. In 1973, it was recorded by Minoru Nojima for Toshiba Records and released in the autumn of the same year, coupled with Piano Concerto No. 2. It received the Arts Festival Excellence Award. Furthermore, on August 16 of the same year, Etsuko Tazaki premiered the first movement at the "Chestnut Hill Concerts" music festival in Connecticut, USA, and on April 7 of the following year, Izumi Tateno gave the world premiere of the complete work in Helsinki.