The "Attention is logarithmic, actually" article argues that it doesn't make sense to classify attention as a quadratic operation due to the highly parallelized nature of GPUs. He argues that the work-depth model of complexity is more informative for paralle...
Subscribe to Ricardo's Digital Garden’s news feed.
Click on “Follow” and decide if you want to get news from Ricardo's Digital Garden via RSS, as email newsletter, via mobile or on your personal news page.
Subscription to Ricardo's Digital Garden comes without risk as you can unsubscribe instantly at any time.
You can also filter the feed to your needs via topics and keywords so that you only receive the news from Ricardo's Digital Garden which you are really interested in. Click on the blue “Filter” button below to get started.
Title: Ricardo's Digital Garden