On: RWKV.F90: Large Language Model in Fortran

[PaulHoule]

,,, wanna see differentiable FORTRAN so much.

[matteogrella → PaulHoule]

Hi there,

Just so you know, I'm the one who wrote RWKV.f90. After making a lightweight autograd package that's a lot like PyTorch, but in Go (I called it spaGO), I am now passionately working on adapting the same concepts to the Fortran language.

But first things first, I want to make sure that using your GPU is as easy as pie. That's my next challenge.

[tn1 → matteogrella]

> I want to make sure that using your GPU is as easy as pie

For Nvidia they made the PGI compilers free with the HPC SDK: https://developer.nvidia.com/hpc-sdk

[_lvbh → matteogrella]

Been using Spago (via Cybertron) and it has been amazing

[gyrovagueGeist → PaulHoule]

ADIFOR was one of (if not the first) large scale automatic differentiation tools

https://www.mcs.anl.gov/research/projects/adifor/