Xrhrft

According to https://en.wikipedia.org/wiki/Group_coded_recording the Commodore 1541 disk drive used a particularly efficient GCR encoding scheme to cram 170K onto the same 5.25" disks that in an Apple drive only stored 140K.

However, for the Amiga 3.5" disks, they reverted to MFM, a lower density encoding. Why abandon something that worked so well?

According to 'Amiga Disk Encoding Schemes MFM? GCR? Please explain!' by Betty Clay,

"[GCR] would appear to let a disk hold far more information than could be stored under most other methods. However, since GCR permits the use of as many as eight on-bits in a row, the drive cannot interpret them at full speed. It is necessary to write or read at only half the normal speed, in order to insure accuracy. When the writing speed is slowed to four microseconds per bit instead of the normal two, the density of the data is only half as much, cutting drastically into the storage advantage."

That would indeed seem to eliminate the advantage of GCR. but then why did that disadvantage not apply to its use for 5.25" disks? Is it a limitation of the rate at which the electronics can process the bits, or of the physics of the interaction of the drive head with the magnetic fields?

On a floppy disk, each 'bit' is a flux reversal — a magnetic event. If those bits are too close together, they'll leak into one another and data will be lost.

Disk controllers use a regular clock and either write a transition or write nothing at each clock tick.

There's also a lower limit on how far apart transitions can be. Disk rotation speed varies according to the whims of the motor, aerodynamic drag, etc, and drives contain automatic gain controls — if they think they aren't seeing data but should be, they turn up their own volume.

So bits need to be regular enough that the controller doesn't have to make too many guesses about rotation speed, and the drive doesn't turn up its gain so far that it's reading noise.

As a result, the bit patterns that drives actually write are a translation of the bytes to be stored into some other encoding, that guarantees bits aren't too far apart, and aren't too close together.

FM and the GCR schemes solve for too far apart differently, but use the same solution for ensuring bits can't be too close together: their data clock is picked so that each tick is far enough apart that two will never be too close. The GCR schemes then do a better job of making sure that they're not too far apart than does FM: FM encoding uses two output bits per input bit, but e.g. Apple's second GCR encoding uses only eight output bits for six inputs.

MFM is a later development than GCR and provides a different solution to the too-close-together problem: it guarantees that there are no sequences that lead to two bits being output consecutively. So, you can double the data clock without fear of magnetic collision. Like FM it also produces two output bits per input, but those two fit into the same physical space as one FM bit. Hence: double density.

MFM is an equally valid improvement for 5.25" drives as it is for any other, and is better than both company's GCRs; the reason that Apple and Commodore each came up with GCR schemes is that they were coming up with something better than FM, not rejecting MFM — both companies released drives before MFM controllers were available.