The store attracted entertainers like Louis Armstrong, Duke Ellington and B.B. King looking for their own earlier work, as well as ordinary lovers of hard-to-find music, such as Richard Parks, who made a documentary about Gershenz and his records.
"I started collecting old hillbilly music in high school and Music Man Murray was on the list of places you had to go to find that kind of stuff," said Parks, whose "Music Man Murray" came out in 2012. "He was the godfather of the used-record store."
As LPs gave way to CDs in the 1980s, customers dropped by with requests that chain music stores couldn't fulfill.
"I get calls like, 'My Uncle Charlie passed away and this was his favorite song and it goes like this, and we want to play it at his funeral,'" Gershenz told an interviewer.
In 1999, Milton Berle bet a friend that Gershenz couldn't locate a 78 called "Cohen on the Telephone", a dialect sketch first recorded in 1913 by comedian Joe Hayman. Uncle Miltie lost, Gershenz gleefully told a Times reporter just before he was to deliver the pristine disc over lunch.
"If I can't find it, forget about it," he said.
In his later years, Gershenz's joyful obsession was bleeding cash. Placing his vast collection on the market in 2010, he found no takers.
Last June, Gershenz finally struck a deal with a buyer in Brazil, whom he never publicly named. Five 52-foot-long tractor trailers carted off the sound archive that started with a 16-year-old boy buying a recording of arias by Swedish tenor Jussi Bjorling.
The record had long since shattered but Gershenz kept the label.
In addition to his son Irv, Gershenz is survived by son Norm; daughter Nada Pedraza; two grandchildren; two great-grandchildren; and an aunt and uncle in Florida. His wife Bobette died in 1999.