The author ported Karpathy's llama2.c code to run inference on TinyStories checkpoints for the PS Vita, achieving approximately 120 tok/s for the 260K model and 1.8 tok/s for the 15M model, and encourages others to explore their project on GitHub.
C programming language, PS Vita hardware support
2025-06-15
- website
Highlight 1
The ability to run sophisticated AI models on a handheld device like the PS Vita is a novel and impressive feat, showcasing innovative use of limited hardware.
Highlight 2
The model download and deletion functionality provides flexibility for users, allowing them to manage storage effectively.
Highlight 3
The app’s performance metrics indicate efficient processing speeds, enabling a practical user experience despite the hardware constraints.
Improvement 1
The application could benefit from a more advanced multi-threading capability to enhance token generation speeds.
Improvement 2
A user-friendly interface and detailed documentation could help improve the onboarding process for new users.
Improvement 3
Introducing additional language support for the model could broaden the app's appeal to non-English speaking users.
Product Functionality
Enhance the app by implementing real-time performance tracking and logging to help users monitor resource usage.
UI & UX
Improve the user interface by creating a more visually appealing design and simplifying navigation for easier access to commands and features.
SEO or Marketing
Focus marketing efforts on targeted campaigns highlighting the unique capabilities of running AI models on a handheld device, leveraging social media and tech forums.
MultiLanguage Support
Include localization options in multiple languages to make the app accessible to a wider audience.
- 1
What models are supported by the psvita-llm app?
The psvita-llm app supports the LLaMA 2 models with 260K and 15M parameters.
- 2
How do I download or delete models on the PS Vita?
The app has built-in functionality that allows users to download or delete models directly from the device.
- 3
What are the performance rates for the models?
The 260K model reaches approximately 120 tokens per second, while the 15M model achieves around 1.8 tokens per second.