Google says new TurboQuant compression can lower AI memory usage without sacrificing quality

Google Research announced TurboQuant, google Research announced TurboQuant.

TurboQuant is designed to reduce memory usage, turboQuant is designed to reduce memory usage of large language models.

Reported up to an 8x performance increase in some tests, google reported up to an 8x performance increase in some tests with TurboQuant.

Reported up to a 6x reduction in memory usage in some tests, google reported up to a 6x reduction in memory usage in some tests with TurboQuant.

Targets reduction of the key-value cache size, turboQuant targets reduction of the key-value cache size.