Hi!Speculative decodingis a hot topic right now!Here is a blog post I wrote proposingmentored decoding, a new variant of speculative decoding. It maximizes the probability to accept the draft tokens while maintaining the Kullback-Leibler divergence between the resulting distribution and the target distribution below a constant D. Speculative decoding can be seen as a special case of mentored decoding for D = 0.Feedback welcome!Blog postSummary in aTwitter threadThanks to@joaogantewho got me interested in speculative decoding with hisblog post.
2023-12-09T01:59:33Z
I am a bit late, but want to praise that you wrote an excellent blog, which is highly appreciate to many researchers …Would love to follow every blog you write!
2024-01-11T20:17:56Z
Hi@Jung. Many thanks, that’s very kind of you. When I publish a new blog post (quite soon hopefully), I’ll mention it on Twitter, so you can follow me (@vivien000000) there to be notified.