Here is a step-by-step process for developing your first AI app using local Large Language Models (LLMs) for Windows:
Install Prerequisites
- Install Windows Subsystem for Linux (WSL) using WinGet:
winget install --id Microsoft.WSL - Install Visual Studio Code and the AI Toolkit extension
Choose an AI Model and Framework
- Decide on a small language model like Microsoft’s Phi-3 mini
- Choose a framework like PyTorch or ONNX Runtime
Install Dependencies
- Install PyTorch and other required packages for training (if using PyTorch)
- Install ONNX Runtime (if using ONNX)
Obtain Training Data
- Collect data relevant to your use case (e.g., GitHub issues, PDFs)
- Preprocess and format the data as required
Fine-tune the Model (PyTorch approach)
- Use the AI Toolkit to create a new project and configure it for fine-tuning
- Provide the training data and fine-tune the model using tools like OLive
Run the Fine-tuned Model
- Set up a service or application to load and run the fine-tuned model
- Implement input/output handling and any necessary pre/post-processing
Use Pre-trained Models (ONNX approach)
- Download pre-trained ONNX models from sources like Hugging Face
- Set up a service or application to load and run the pre-trained model
- Implement input/output handling, including data retrieval and grounding
Deploy and Use
- Deploy the application or service running the local LLM model
- Integrate it into your workflows or applications as needed
Iterate and Improve
- Evaluate the performance of the LLM model and refine as needed
- Explore different models, frameworks, or techniques to improve results
Learn More
– Check out additional resources like OLlama for exploring more LLM models
– Participate in Microsoft learning resources on AI, DirectML, and more
This process covers the key steps involved, including setting up the environment, choosing and fine-tuning or using pre-trained models, building applications around them, and deploying them locally on Windows. The specific implementations may vary depending on the chosen approach (PyTorch fine-tuning or ONNX pre-trained models) and the use case requirements.
