Setup LocalAI on Ubuntu in Azure/AWS/GCP

Setup and install LocalAI on Ubuntu 22.04 in the cloud on Azure, AWS or Google GCP.  Local AI serves as a self-hosted alternative to cloud-based AI services, offering a customizable and private AI environment. LocalAI can replace popular platforms such as OpenAI, Anthropic’s Claude, Google Vertex AI, Microsoft Azure OpenAI, Cohere, and Hugging Face Inference API.  It allows you to run LLMs, generate images, audio with consumer grade hardware, supporting multiple model families. Does not require GPU.  Deploy in Azure, AWS or GCP using the following images:

Cloud LocalAI

LocalAI Azure

LocalAI Azure

Deploy LocalAI on Ubuntu 22.04 in Azure

LocalAI AWS

Coming soon…

LocalAI GCP

Coming soon…

Getting Started with LocalAI

Once your LocalAI server has been deployed, the following links explain how to connect to a Linux VM:

 

 

Once connected and logged in, the following section explains how to start using LocalAI.

Verify LocalAI Installation

Run the following to confirm LocalAI service is running:

				
					sudo systemctl status local-ai

				
			

Access LocalAI WebUI

  • By default, LocalAI’s web interface runs on http://localhost:8080. Open a browser to interact with the WebUI using your server ip address: http://serverip:8080.
  • The WebUI allows you to configure settings, upload models, and manage tasks.
				
					http://serverip:8080
				
			

And you should see the following screen, ready to start installing your models. These can be installed via the gallery.

LocalAI UI Dashboard

Click Gallery and you can select from the list of models to install. Once installed you can then choose your installed model.

LocalAI WebUI

Add AI Models to LocalAI

LocalAI requires models to function, so place any model files (e.g., .gguf, .bin) in the configured models directory:

 

  • Default model path is often /usr/share/local-ai/models.
  • To add new models, place them in this directory and restart LocalAI if necessary.
  • You can also add models via the WebUI within the gallery.

 

Documentation on adding Models can be found on:

https://localai.io/docs/getting-started/models/

Restarting LocalAI can be done via the following command:

				
					sudo systemctl restart local-ai

				
			

LocalAI Support / Documentation

For tutorials, documentation and support refer to the official documentation and Gitrepo

 

https://github.com/mudler/LocalAI

 

https://localai.io/basics/try/

Firewall Ports

By default, LocalAI operates on port 8080. This is where the main REST API and WebUI are accessible, allowing you to make requests and access the interface.

 

  • Port 8080: LocalAI’s primary HTTP server for API calls and WebUI access.

 

The links below explain how to modify / create firewall rules depending on which cloud platform you are using.

 

To setup AWS firewall rules refer to – AWS Security Groups

To setup Azure firewall rules refer to – Azure Network Security Groups

To setup Google GCP firewall rules refer to – Creating GCP Firewalls

Disclaimer: LocalAI is licensed under MIT license. No warrantee of any kind, express or implied, is included with this software. Use at your risk, responsibility for damages (if any) to anyone resulting from the use of this software rest entirely with the user. The author is not responsible for any damage that its use could cause.

Avatar for Andrew Fitzgerald
Andrew Fitzgerald

Cloud Solution Architect. Helping customers transform their business to the cloud. 20 years experience working in complex infrastructure environments and a Microsoft Certified Solutions Expert on everything Cloud.

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x