The power of scale is a blessing for ML. Indeed, scale has been the tool behind most major advances, starting with AlexNet. But it is also a curse: it is physically impossible to maintain our current rate of progress if scale is our only tool.
— Tom Goldstein (@tomgoldsteincs) July 5, 2022
Culture Magazine
