Setup and install LocalAI on Ubuntu 22.04 in the cloud on Azure, AWS or Google GCP. Local AI serves as a self-hosted alternative to cloud-based AI services, offering a customizable and private AI environment. LocalAI can replace popular platforms such as OpenAI, Anthropic’s Claude, Google Vertex AI, Microsoft Azure OpenAI, Cohere, and Hugging Face Inference API. It allows you to run LLMs, generate images, audio with consumer grade hardware, supporting multiple model families. Does not require GPU. Deploy in Azure, AWS or GCP using the following images:
Cloud LocalAI
LocalAI Azure
Deploy LocalAI on Ubuntu 22.04 in Azure
LocalAI AWS
Coming soon…
LocalAI GCP
Coming soon…
Getting Started with LocalAI
Once your LocalAI server has been deployed, the following links explain how to connect to a Linux VM:
Once connected and logged in, the following section explains how to start using LocalAI.
Verify LocalAI Installation
Run the following to confirm LocalAI service is running:
sudo systemctl status local-ai
Access LocalAI WebUI
By default, LocalAI’s web interface runs on http://localhost:8080. Open a browser to interact with the WebUI using your server ip address: http://serverip:8080.
The WebUI allows you to configure settings, upload models, and manage tasks.
http://serverip:8080
And you should see the following screen, ready to start installing your models. These can be installed via the gallery.
Click Gallery and you can select from the list of models to install. Once installed you can then choose your installed model.
Add AI Models to LocalAI
LocalAI requires models to function, so place any model files (e.g., .gguf, .bin) in the configured models directory:
Default model path is often /usr/share/local-ai/models.
To add new models, place them in this directory and restart LocalAI if necessary.
You can also add models via the WebUI within the gallery.
By default, LocalAI operates on port 8080. This is where the main REST API and WebUI are accessible, allowing you to make requests and access the interface.
Port 8080: LocalAI’s primary HTTP server for API calls and WebUI access.
The links below explain how to modify / create firewall rules depending on which cloud platform you are using.
Disclaimer: LocalAI is licensed under MIT license. No warrantee of any kind, express or implied, is included with this software. Use at your risk, responsibility for damages (if any) to anyone resulting from the use of this software rest entirely with the user. The author is not responsible for any damage that its use could cause.
Cloud Solution Architect. Helping customers transform their business to the cloud. 20 years experience working in complex infrastructure environments and a Microsoft Certified Solutions Expert on everything Cloud.