Connecting to AI

While novelcrafter is a great tool for writing on your own, it really shines when you connect it to an AI. This allows you to get suggestions for your story and characters, and even have the AI write parts of your story for you.

However, you do not get automatic access to an AI when you sign up for novelcrafter. Because of that, we allow you full control over which models you’d prefer and can afford. It also allows you to use your own hardware to run the AI models, which can save you a lot of money.

Currently, novelcrafter supports the most popular AI vendors. We are always looking to add more vendors, so if you have a favorite that is not supported yet, please let us know.

Each vendor has its own strengths and weaknesses, so we recommend trying out a few to see which one works best for you. They also offer access to different AI models, so you might find that one vendor has a model that is better suited for your needs.

Your settings are stored locally in your browser. You will have to enter them again if you use a different browser or device. We do not store any credentials on our servers to protect your privacy.

Learn how to transfer your settings to other machines here.

Online/Cloud

OpenAI

Arguably the most famous AI vendor, OpenAI is the creator of the GPT-3 and GPT-4 language models. Their language models are one of the most powerful available today, and are capable of writing entire stories on their own.

Note

OpenAI requires you to have an account with them and to provide your own API key.

Caution

All models by OpenAI are filtered and moderated. If your story contains content that is not suitable for children, is NSFW (like explicit content), and such, you might want to consider using a different vendor.

OpenRouter

OpenRouter is a great tool that offers not just commercial AI models, but also access to hosted open source models. They offer both, filtered and unfiltered models at great prices.

If you need access to Anthropic’s Claude models they’re a great place to start.

Note

OpenRouter requires you to have an account with them and to top up your account with credits (like pre-paid SIM cards).

Anyscale Endpoints

A bit of a niche vendor, as they’re focused on the enterprise market, but they offer cheap access to Llama2 models by Meta (with new ones being added all the time).

Note

Anyscale requires you to have an account with them and to provide your own API key.

Local/On-Premise

OpenAI-API compatible runners

Due to the popularity of OpenAI’s API, there are a number of open source projects that allow you to call them via an OpenAI-compatible API. This allows you to use your own hardware to run the AI models, and to avoid having to pay for the API usage.

When connecting to these, just add them as another “OpenAI” vendor, and use the right connection settings they provide you with.

Here are a few of the most popular ones:

1. LocalAI: This neat project even allows you to run models without a GPU. It not only supports text generation models, but also image generation and speech-to-text.
2. LM Studio: A beginner-friendly tool that has a nice graphical user interface.

Ollama

In case you have a technical background, Ollama can be simply described as “docker for AI models”. They have an easy install process and allow you to get started within seconds - even as a beginner.

It too works on hardware without a GPU (even older laptops with slow Intel-processors).

Note

When you start the Ollama server, be sure to include novelcrafter to the host list, like so (otherwise it won’t be able to connect properly):

OLLAMA_HOST=0.0.0.0 OLLAMA_ORIGINS=https://app.novelcrafter.com ollama serve

LM Studio

LM Studio is a great tool that allows you to discover, download and run local LLMs with ease. It has a nice graphical user interface, and is easy to use even for beginners.

To set up the connection do the following:

1. Add a new OpenAI connection
2. Rename it to “LM Studio” to keep track of it
3. Use https://localhost:1234/v1 as the base URL (or whatever port you’ve specified in the settings)

Note

Make sure to enable CORS for the local server settings, otherwise novelcrafter won’t be able to connect to it.

Caution

LM Studio only supports a single model at a time. If you want to use multiple models, you might have to use a different runner.

Transferring your settings

Connection settings are stored per machine and never get synced to the cloud. This means that you will have to set up your connections on each machine you use novelcrafter on.

However, you can export your settings and import them on other machines. This allows you to easily transfer your settings to other machines.

When you import your settings, Chrome/Firefox/etc.. might ask you for clipboard access when you do this (hit “allow” to continue).