Translations, Transformers, and Tilings
Added 2024-02-01 21:20:41 +0000 UTCHey everyone,
There have been three main projects on deck for the last month: Two new videos, and an effort to get videos into different languages.
Translations
The video above shows some of what I have in mind here. If any of you are multilingual and want to help out, here are a few ways you can do so:
- Take a look at the files in this repository, especially the ones titled "sentence_translations.json", and let me know in the comments below how reasonable the translations are.
- To go above and beyond, you can even offer improvements and corrections to those files by clicking the little pencil icon for "edit this file", and then submitting those changes as a pull request.
- As we start to produce more auto-dubbings, I'd love to run them by native speakers to see how natural they sound. If you'd like to provide input, kindly fill out this Google form, and we may send you some samples in the coming weeks.
Transformers
Here's an early view of the script for a video I'm working on about Transformers (The "T" in "GPT"). I'm often a little hesitant to share scripts, mainly because the planned explanation leans so heavily on visuals that it's not often easy to convey in text alone. The way I usually work is to have some sense of what I'd like things to look like in my head, then as I go through and code up the animations, these almost always change based on what that animation process reveals.
I'll share more as it gets animated, but hopefully, the script gives sense sense of the plan and offers some surface area for early feedback. This will be a long one, comparable to 3 or 4 ordinary YouTube explainers, but I think for this topic it makes sense to keep all the context together in a single package rather than subdividing it into separate pieces.
Escher
Quinn, who is the one who helped out with the barber pole optics project, has been helping to put together a piece about Escher, and some of the fun back and forth where math inspired art, which in turn inspired math, which in turn inspired art. It's about his work with tilings. If you're curious about an early peek there, here's a slightly messy outline, where the text in blue is intended as a draft of narration.
We're having an artist depict some of the scenes in the story, and here are a couple of the time-lapses for the drawings, which I think you'll find fun: Here's one depicting a Gallery where Coxeter discovered Escher's word (with a hat tip to Penrose in the background), and here's one showing Escher doodling around with one of the diagrams he was inspired by in a paper from Coxeter. Much more to come!
Comments
Hi Grant, I found your GPT series on Youtube and the amount of intuition i gained from it, specifically around the embedding vectors in n-dimensional space, was seriously invaluable. Thank you! Since, I've gone very deep on transformers. I've coded along with Andrej in pytorch and trained my own (smaller) GPT2! Feels good to have such a deep understanding now. Reflecting on my mental journey understanding these models and what was confusing for me at first was: - GPT models are decoder only models and skip the cross-attention block. The "Similarity is all you need" paper is so prevalent, and the figure of a transformer (for translation) made it seem like (at least to me) that it is the algorithm used for GPT. I thought the encoder input side was taking the prompt and the decoder was producing the output sequence at first glance. Was glad to see that point marked at the end of your script for the next video. That would clear up a lot of confusion for people learning I think! - Decoders are essentially encoders except for the masked triangular matrix in the attention layer. The intuition behind this is that encoders look at all sequences in the context block almost like a bidirectional LSTM. Very cool once you understand that part. - I know you've seen Andrej's video because you credited it, but he uses an nn.Embeding layer for the positional encodings, while the Attention paper uses sine/cosine waves at different frequencies. I understand the intuition behind the sine waves for the positional encoding lookup table, but not necessarily Andrej's embedding layer, which looks like the network also backprops to learn those weights as well. Still unsure on that :/ I think some interesting points to touch on (if the video isn't long enough) is: - pretraining vs finetuning: "pretraining" for the t+1 sequence prediction (which some people call unsupervised, but really it's supervised learning) and "finetuning" for when you want the trained network to "do something": like classification for example, and then you would use your labelled training data. You would throw another linear layer on the output and then append the corresponding activation function for the task you're doing. It wasn't clear to me the point of finetuning, but now I understand. Cool that you can have somewhat intelligent neural connections prior to supervised learning that you can leverage and bend to your will (that sounds overly sinister lol). Kind of like the early days of deep belief networks and pretraining with restricted boltzmann machines. - touching on other modalities other than text. You briefly mentioned chunks of audio in the first video which I found fascinating. It would be cool to have an audio example since those are really popular lately (like suno.ai, text to speech, etc.) and how those are tokenized into embedding vectors for the transformer. Sorry for the novella, I'm real deep in it right now. Thanks for bringing me to this place of mental epiphany!
Greg Burlet
2024-05-07 02:07:37 +0000 UTCI know I’m really late to this, it’s minor feedback in any case. On the word embedding piece, the examples are great, but I think to the laymen it will still be mysterious how the model has generated those vectors that are so similar to each other and have those properties. I think it would motivate if you say something like “The ___ sat on the throne” to show how techniques like masked words are used to nudge the king and queen vectors together, and then other sentences with “him” or “her” to demonstrate how they get pushed apart in some other direction
Tristan
2024-03-11 14:05:30 +0000 UTCHey Grant, Have not seen any new content in a while and was beginning to experience withdrawal so it is so exciting to see what you are cooking up. Reading your script on transformers is my first exposure to the details behind LLMs and my first time reading one of your written outlines. Although I follow and have a cursory understanding of the concepts involved mainly because of my previous education and my following of your previous videos, the quantum leap in understanding and intuitive comfort with the material that your videos provide cannot be overstated. All the above is to say that I fully expect that it will be well worth the wait!!!
Benjamin Bailey
2024-02-17 14:58:18 +0000 UTCOk. I was able to suggest some changes to the Newton puzzle short and the Refractive index question French translation, after reading the readme section.
Pierre-Yves St-Louis
2024-02-06 03:39:54 +0000 UTCThanks for the notes. Indeed, having a native speaker refine the first-pass machine translations will be critical. I'm still figuring out the best format for community contributions, but just from the early activity on the GitHub we have enough to work with to move forward with some experiments.
3blue1brown
2024-02-05 22:48:48 +0000 UTCGood to know, I'll make a note of it.
3blue1brown
2024-02-05 22:46:29 +0000 UTCHi Grant, I think the translation + dubbing project is a great one to reach an even greater audience. Even though the quality of your audio recordings and pronunciation are excellent, I think many people may be afraid (or may not get many YT recommendations) to watch videos in a language that is not their native tongue. I've just filled out the form, and I'd like to help enhancing french translations. After a quick look at the Gaussian integral captions you provided as an example, I think there's indeed room for improvement, e.g. idiomatic expressions, technical words, gender (dis)ambiguity — maybe you pointed to this particular video on purpose ? The automatic translation tool "chose" that the french statistician and their friend were males, whereas your illustration states otherwise. Among other questions, I think this last point would very much benefit the use of translation guidelines or recommendations shared among contributors, since this is and has been a rather touchy and conflictual topic in France. I'm personally used to mess with git repositories + forges on a daily basis, so I'd be fine with editing raw json files, but I wouldn't bet that this is the case for the majority of potential contributors. No better tool comes to my mind though, maybe that some localization tools could do the trick ? I know that weblate is a good option for open source software but I'm not sure this would fit your needs for long contextful scripts. Also, I don't know how one would handle translations PR but this may (or may not !) quickly get out of control, and perhaps a voting system might make sense at some point while reviewing "competing proposals". Also very excited about the Transformers video, but I think I will wait for it to be published and not look at the script for now. Thanks a lot for all the beautiful content you've been sharing.
Raphaël
2024-02-05 21:53:48 +0000 UTCFor someone familiar with git who has a GitHub account, editing JSON is not absurd at all. For those unfamiliar, the whole forking thing is confusing at first. You may want some sort of check to make sure the proposed edit is not destructive. An LLM in the loop as a sanity check on edits might be good enough for that and could probably be implemented as something like a GitHub action.
Alexis Olson
2024-02-04 21:06:32 +0000 UTCI was looking through the subtitles of the Barber Pole effect part 1 in Urdu and the tone of the translation is very formal. There are also many small issues as well, like Linear got translated to "لاین" which means Line, shining light was translated as "روشنی دالنا" which is the translation of the english idiom "shining light on the matter" (a better translation would be روشنی گوزارنا "passing light through it"). I had my mother try to understand the video just using the subtitles without any audio and she found it pretty hard. Not the physics or maths of the video but just understanding what you are actually saying.
Mohammad Areeb
2024-02-04 08:14:48 +0000 UTCHi! I indeed seems to have done a pull request unintentionally (I was in GitHub.dev). I'm not that familiar with the GitHub environment, which in some sense is a direct answer to your question. I really don't think editing a raw JSON file is problematic, but I would appreciate a simpler editing system than GitHub. I've deleted the forked repository, but when I try again to make changes to the Newton art (which was meanwhile move to a 'shorts' folder), by clicking on the edit icon (in Github) I get this message : You need to fork this repository to propose changes. Sorry, you’re not able to edit this repository directly—you need to fork it and propose your changes from there instead. I get to same message when using GitHub.dev. I will need more instruction to be able to send corrections.
Pierre-Yves St-Louis
2024-02-04 05:23:15 +0000 UTCI looked at Russian translations, and they are pretty bad as well -- they are way too literal. For example, "paper by ..." is translated as "статья" in one sentence, but the next sentence: "The paper begins ..." as "Газета начинается ..." ("Газета" is a newspaper and is completely wrong here). It goes downhill from there, e.g. "pulling their leg" is translated literally as "тянет за ногу", which would make absolutely no sense to a Russian reader who doesn't understand English.
Paul Pluzhnikov
2024-02-03 23:48:09 +0000 UTCYeah, the transformer-circuits material is fantastic. I had a very helpful conversation with Chris Olah while putting together the plans for this one.
3blue1brown
2024-02-03 14:07:23 +0000 UTCThank you! This one will also be very useful as an instructional video for professionals 👍. You have a real talent for explaining things simply yet thoroughly and interestingly
Ahgversluis
2024-02-03 05:11:23 +0000 UTCFor months, I've been secretly hoping you'd do a video on transformers, and wondering if the idea was brewing in your head. Very happy and grateful :)
spacediver
2024-02-03 04:34:48 +0000 UTCNot exactly. I try to cross reference it to what Google translate produces, and if there are glaring disparities I will try and ask why that is the case. Problem is that because I am not fluent, I can’t really verify the accuracy of either translator. Another problem is that some languages have such a dearth of written material (some languages in the Philippines are more often spoken than written down for example) so LLMs have little training data to get the nuances of a language correctly.
Kyle M. Kabasares
2024-02-02 21:11:59 +0000 UTCA transformer video is really exciting! I've found that they're the hardest part of ML to get an intuition for. If you haven't read it already, https://transformer-circuits.pub/2021/framework/index.html is really good. I've actually been working on building a small toy-model generative transformer over the last few days to better understand them. Also, I love the idea of an Escher project! Godel, Escher, Bach is my favorite book; there's so many interesting philosophical and mathematical ideas to explore relating to his art.
Henry
2024-02-02 21:10:01 +0000 UTCThanks! Did you happen to offer those modifications as a pull request? How absurd does it feel to invite contributions by editing the raw JSON file? If I were to invite contributions from the community more broadly, do you think it's worth setting up a better interface?
3blue1brown
2024-02-02 20:55:43 +0000 UTCInteresting to know. Are there are ticks you've found for how to nudge the LLMs to give the kind of translation you want?
3blue1brown
2024-02-02 20:53:48 +0000 UTCFirst of all, let me say I much I think that your translation project is an important one. I've often thought when watching your videos how sad it was that non native English speakers didn't have access to such high quality and clear mathematical content in their language (French in my case). I've seen youtuber that are definitely not native English, do their videos in English to reach a bigger audience, but leaving behind their linguistic compatriot. I've looked at the French translation to the Newton 2023 Newton art puzzle short, and made some modifications to it. Overall it's was very good. Nonetheless, the automatic translations surely need a little bit of love from a native speaker to be perfect. Good luck with all of your project!
Pierre-Yves St-Louis
2024-02-02 20:05:37 +0000 UTCI saw the sketches some 50 years ago when they were still in the back room of the Sketches Museum in den Haag, they made a lasting impression.
Gregor Shapiro
2024-02-02 18:53:19 +0000 UTCHappy to see fellow Sublime text user! I love this concept, Grant! Currently trying to learn my family's (Filipino) dialect since I grew up here in the United States. I am not a native speaker, but one of my interests would be to somehow translate a lot of science and math concepts into native Filipino dialects (there are several) using LLMs. It is a little challenging since it seems that most Filipinos discuss science mainly in a hybrid Tagalog (the national language) + English format, since everyone there mostly also speaks English. I do think that there is a benefit and can aid in promoting scientific literacy if translations and dubbings in more rarely spoken languages are available, especially in regions where most people do not have the privilege of learning English fluently.
Kyle M. Kabasares
2024-02-02 18:33:45 +0000 UTCSure! As I was attempting to improve the translations I realized it's not an easy task, even for a native speaker such as myself.
Alon Altman
2024-02-02 17:19:04 +0000 UTCGood to know! We might play around with how well GPT-4 can correct it to be more fluent. Would you mind if I send you a couple of samples as we do?
3blue1brown
2024-02-02 14:58:47 +0000 UTCI do emphasize it later on in the script, but I can consider bumping that up.
3blue1brown
2024-02-02 14:58:02 +0000 UTCSo true, that was one of my favorite memories of the Netherlands.
3blue1brown
2024-02-02 14:56:51 +0000 UTCDear 3Blue1Brown, I absolutely love your videos. You're the first person that actually got me excited about Linear Algebra. If only someone had shown me visually what a determinant is in university, I would have saved a lot of headaches. Anyway, I work in AI patents and the European Patent Office and I could not be more excited about the Transformers video. Truly cutting edge science! What struck me is that where language processing used to be recurrent, Transformers work on the data in parallel. Maybe this is an insight that would fit in the introduction?
Ahgversluis
2024-02-02 13:18:14 +0000 UTCAre you logged into a github account? I believe you have to be, in order to edit.
Gregor Shapiro
2024-02-02 06:32:44 +0000 UTCÁ propos Escher: Don't forget to point interested viewers to the Escher Museum in den Haag, Netherlands, his original sketches and prints are worth viewing.
Gregor Shapiro
2024-02-02 06:21:16 +0000 UTCI'm a native polish speaker. But I don't want to commit to anything so if it requires some effort on your part, please don't. As to which video - do you have any preferences?
Bartosz Błaszkiewicz
2024-02-02 05:49:18 +0000 UTCJust looked at some of the Hebrew translations (specifically for the prism video) and they are not good. They are very literal translations of English forms of speech that are not replaced with equivalent ones in Hebrew. Especially with technical explanations, I think the translation won't help much. This is especially true since most Hebrew speakers who are not very young children also speak at least some level of English. Oh, and I don't see an "edit this file" button anywhere.
Alon Altman
2024-02-02 02:33:50 +0000 UTCRight you are, thanks for the catch!
3blue1brown
2024-02-01 22:54:58 +0000 UTCHey, your explanation of temperature in the GPT script is flipped, higher temperature means a more uniform distribution
Carlos
2024-02-01 22:53:14 +0000 UTCWe'll play around with how well it works for quality assurance on the translations, and potentially if the sentence timings on the auto-dubbings need adjusting, it may be asking it to produce slightly shorter versions of each sentence, but we'll see. I'm happy with where the subtitle display times are at the moment, that feels to me like more of a traditional programming task than a GPT-4 one.
3blue1brown
2024-02-01 22:32:16 +0000 UTCDo you mean for accepting community edits to the translations? Indeed I think it would be better to have a tool in the browser in that case. Thank you for the offer, but if we decide to go that route, I've got someone who can help out.
3blue1brown
2024-02-01 22:29:40 +0000 UTCMy mistake, it should be fixed now.
3blue1brown
2024-02-01 22:26:57 +0000 UTCBTW, when I open the google form I see a "You need permission" screen, is it only me?
Andrea Bedin
2024-02-01 22:17:53 +0000 UTCi am a c++ programmer. do you want me to try to build some tool for dealing with the json format (and also git and stuff)? cpp is properbly not the best choise of language here. typescript would properbly fit the bill better but i don't have any experience in that sadly.
Asser tores
2024-02-01 22:05:44 +0000 UTCCertainly. You could also pass let me know which video, and which language, and I can generate the initial pass for you
3blue1brown
2024-02-01 22:03:00 +0000 UTCI'm not a native speaker, but skimming through some of the files, the Japanese translations seem very reasonable.
ghostdunk
2024-02-01 22:01:05 +0000 UTCIf I see a missing language can I create one? If I can translate it most of the way, but struggle with some domain-specific terms does it help?
Bartosz Błaszkiewicz
2024-02-01 21:51:43 +0000 UTCJust looking through the Italian translations. My sample size is still not large enough to give an overall quality judgment but I already found some issues there. For example In the refractive index questions episode, in the very first sentence "rate of slowdown" is translated with "velocità di rallentamento", which would be better translated in "speed of slowdown", as if the slowdown was dependent on time and the "acceleration" depends on color. Probably it would be better to just leave "rallentamento" which is slowdown. In the prism episode, "earned a place as an iconic album cover" is translated as "guadagnati un posto come copertina iconica dell' album?", which literally translate in "the iconic cover of THE album", as opposed to "the iconic cover of AN album" as ith should be. So "dell' album" should be replaced by "un album". I found these and other small issues in a couple minutes, so I would guess there could be more (or I'm incredibly unlucky...)
Andrea Bedin
2024-02-01 21:51:29 +0000 UTCSpeaking of gpt-4, have you tried using it to produce translations? I wonder if it could do something like adjust the subtitle display times depending on the relative complexity of a given sentence in a given language.
Mason Boeman
2024-02-01 21:36:25 +0000 UTC
