Hacker News
points by CamperBob2 3 days ago | 0 comments
But machine learning stuff feels like it could have room for learning-by-doing applications that aren't entirely pointless, so I wonder if there's some way to approach the topic I'm simply not finding.

That's what Karpathy's videos are for. You end up with a GPT-2.0 equivalent if you follow them to the end.

[–] yunwal 3 days ago
This is still pointless though. I mean it's valuable from a learning perspective but the parent is saying they want to contribute somehow. You can only learn so much from toy examples
[–] bityard 3 days ago
Once you learn how an LLM works, you can likely _begin_ to contribute meaningfully to numerous open-source AI projects: inference engines, training frameworks, etc. Or try to find an area of AI that is lacking research and start your own experiments. Or (potentially) go to work for a company doing similar or adjacent work.

Opportunity knocks, but you have to build the door first.

[–] CamperBob2 3 days ago
The prereqs aren't as extensive as people fear, but they are important. Starting out by building a scratch-trained GPT3-scale model is always an option, but anyone who's going to succeed at that doesn't need to watch any YouTube videos.

For the rest of us, the sense of discovery that we get from training and running toy models is a big part of the reward. This stuff is not an exact science, at least not yet, so time spent building intuition with small-scale models is an incredibly important part of the learning process. Not pointless at all.