points by yunwal 3 days ago | 0 comments
This is still pointless though. I mean it's valuable from a learning perspective but the parent is saying they want to contribute somehow. You can only learn so much from toy examples
[–] bityard 3 days ago
[–] CamperBob2 3 days ago
The prereqs aren't as extensive as people fear, but they are important. Starting out by building a scratch-trained GPT3-scale model is always an option, but anyone who's going to succeed at that doesn't need to watch any YouTube videos.
For the rest of us, the sense of discovery that we get from training and running toy models is a big part of the reward. This stuff is not an exact science, at least not yet, so time spent building intuition with small-scale models is an incredibly important part of the learning process. Not pointless at all.
Opportunity knocks, but you have to build the door first.