The click might be of secondary importance. These differences seem more fundamental:
Case 1: Play and release a single note
Polyphone: after release, the sound diminishes to silence by the end of the release phase
Fluidsynth: after release, the sound continues without diminishing and stops abruptly at the end of the release phase
The difference does not seem to be a matter of time spent, but of the rate of change in loudness during the release phase.
Case 2: Play and release a single note, then play another in the same exclusive class
Polyphone: after the second note begins, the sound of the first stops immediately
Fluidsynth: after the second note begins, the first continues to sound for about 0.75 s
The difference seems to be a significantly different interpretation of "rapidly" in the SoundFont spec.
I hope that is more clear.
And I don't want to sound like I'm complaining, but I'm sure you can imagine the feeling of working for a while in Polyphone, believing I've created something very nice, and then learning that it works badly when I go to render my music in Fluidsynth.
But I have found Polyphone useful; thanks for creating it.