MIT Course: Generative AI for Constructive Communication

[tim_sw]

https://ai4comm.media.mit.edu/

[mercurialsolo]

The linked courseware (required readings) is open and pretty good to get started.

[akmittal]

Is it available to be taken remotely at own pace?

[pratap103 → akmittal]

Doesn't look like it but the slides and recommended reading are posted and should be able to give you some color.

[hislaziness → akmittal]

From the course page - Classes are in person on Wednesdays from 2:00PM - 4:00PM Eastern Standard Time in E14-526

[revskill]

I want a course that can simulate LLM using Small language model instead. It could run fast on a CPU machine, for the sake of learning.

[benob → revskill]

[1] is a good start, although if you want to train from scratch on CPU, you'll have to downscale as transformers need quite a bit of data before they learn to use position embeddings. For example try a single-layer RNN on Shakespeare texts [2] or a list of movie titles from IMDB [3]. You'll have to fill in the blanks because things have evolved quite a bit since those were used for language models, but you can find some tutorials [4] and examples [5].

[1] https://jaykmody.com/blog/gpt-from-scratch/ [2] https://github.com/demmojo/text-rnn/blob/master/datasets/sha... [3] https://datasets.imdbws.com/ [4] https://pytorch.org/tutorials/beginner/chatbot_tutorial.html [5] https://github.com/pytorch/examples/tree/main/word_language_...

[brokenbutton]

Is it possible to view the lectures online?

[gnicholas → brokenbutton]

It doesn't appear to have an online component, nor is there any mention of OCW, so I'm guessing not. Perhaps they'll record it and post later, though. I was curious to see why this was posted if it's not something that is open to remote participation or viewing, but I guess it was posted for the syllabus/readings?