• 0 Posts
  • 3 Comments
Joined 1 year ago
cake
Cake day: July 2nd, 2023

help-circle

  • I agree with this. I’m using a 1070ti for image gen and it would be more than capable for handling some LLM stuff. An AMD 7700xt ive found dors well with 7B models on my main rig but im sure you could get away with somthing cheaper or less powerful.

    That said, the amount of text you can genrate or the context length of its answers will depend the model you use and the larger the model, the more power it takes.

    If youre just messing around with it or want it to review or answer small questions, I’d say a 1070ti like I’m using would be just fine. Some folks use even more budget friendly options. If you got a gaming machine with any semi recent GPU, I’d say go for it. Worst case, you can pay for a subscription later if you really want.